How to check compatible library version interval for my package in python?

Question

I am developing a private python library and I am using other public libraries in it. I want to define an interval of compatible version for every public library that my custom library uses.

I already implemented tests that check if the library works fine. Just need to find a way to test my code in every combination of every public library.

I tried to use tox but it seems like I have to write every combination of libraries in the tox.ini file so that it can create environments with libraries whose versions are different. Is there an automated way to do that?

For example: My development environment has following libraries and it passes my tests.

• numpy==1.24.3

• matplotlib==3.7.3

I want to know if I can test for combinations of

• numpy==1.24.0, 1.24.1 ... 1.24.x
• matplotlib==3.7.1, 3.7.2 ... 3.7.x

in an automated way.

Answer 1

Thanks everyone who helped me in the comment section. As @sinoroc suggested, using nox solved my problem in a very intuitive way. Even though this is not a common practice, I am sharing the solution that I used for given example in the question in case if there is anyone who needs to test their project like me.

Codes in the noxfile.py are given below.

import nox

@nox.session(name = "test_sample")
@nox.parametrize("numpy", ["1.24.3", "1.24.2"])
@nox.parametrize("matplotlib", ["3.7.3", "3.7.2"])
def tests(session, numpy, matplotlib):
    session.install(f"numpy=={numpy}", f"matplotlib=={matplotlib}")
    session.run("pytest")

Parametrize decorator defines the versions that will be used. Since I used two different versions for two different libraries, there will be 4(2*2) test results acquired from different version combinations.

Note: I have observed that nox uses python -m pip command to install the libraries. Therefore, if the public libraries that we test have common dependencies between them, there may be an override problem where given version of a library is overwritten while installing another. I could not find a solution for it but this is as precise as version testing can get.

Answer 2

You can use generative environments, described in the documentation of tox, see https://tox.wiki/en/4.12.1/user_guide.html#generative-environment-list

From the documentation:

[tox]
env_list = py{311,310,39}-django{41,40}-{sqlite,mysql}

[testenv]
deps =
    django41: Django>=4.1,<4.2
    django40: Django>=4.0,<4.1
    # use PyMySQL if factors "py311" and "mysql" are present in env name
    py311-mysql: PyMySQL
    # use urllib3 if any of "py311" or "py310" are present in env name
    py311,py310: urllib3
    # mocking sqlite on 3.11 and 3.10 if factor "sqlite" is present
    py{311,310}-sqlite: mock

Disclaimer: I am one of the tox maintainers. The above section is in the user guide, which just follows the installation instructions. In case you had trouble finding that piece of information, can you tell us how we could improve the documentation?