Tox throws an error when running sphinx-build through Poetry in Docker container

Question

I am trying to generate Sphinx documentation for a project using Poetry and Tox. I have this configuration:

# tox.ini
[tox]
envlist = py36
isolated_build = True

[tox:.package]
basepython = python3

[testenv]
whitelist_externals = poetry
commands =
    poetry install -v
    poetry run pytest tests/

[testenv:docs]
description = invoke sphinx-build to build the HTML docs
whitelist_externals = poetry
commands = poetry run sphinx-build -d "{toxworkdir}/docs_doctree" documentation/source "{toxworkdir}/docs_out" --color -bhtml {posargs}
           python -c 'import pathlib; print("documentation available under file://\{0\}".format(pathlib.Path(r"{toxworkdir}") / "docs_out" / "index.html"))'

# pyproject.toml
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.poetry]
name = "my-project"
version = "0.0.1"
description = "My project"

[tool.poetry.dependencies]
python = "^3.6"
apache-beam = {extras = ["gcp"], version = "^2.24.0"}
click = "^7.1"
click-log = "^0.3"

[tool.poetry.dev-dependencies]
pytest = "^6.1"
black = "20.8b1"
sphinx = "^3.2.1"
sphinx-autoapi = "^1.5"

Locally, I can run the following command lines to generate the documentation, they all work:

• cd documentation; make html
• tox -e docs

But for CI in Bamboo, I chose to run in a Docker container (FROM python:3.6 with Poetry and Tox installed) to avoid issues with other packages installed on the agent, and then tox -e docs throws an error:

 docs run-test: commands[0] | poetry run sphinx-build -d /app/.tox/docs_doctree documentation/source /app/.tox/docs_out --color -bhtml

  FileNotFoundError

  [Errno 2] No such file or directory
 
   at /usr/local/lib/python3.6/os.py:594 in _execvpe
        590│         path_list = map(fsencode, path_list)
        591│     for dir in path_list:
        592│         fullname = path.join(dir, file)
        593│         try:
     →  594│             exec_func(fullname, *argrest)
        595│         except OSError as e:
        596│             last_exc = e
        597│             tb = sys.exc_info()[2]
        598│             if (e.errno != errno.ENOENT and e.errno != errno.ENOTDIR
 ERROR: InvocationError for command /usr/local/bin/poetry run sphinx-build -d .tox/docs_doctree documentation/source .tox/docs_out --color -bhtml (exited > with code 1)

If I run a docker exec to get an interactive shell in the container and run /usr/local/bin/poetry run sphinx-build -d .tox/docs_doctree documentation/source .tox/docs_out --color -bhtml, the documentation is generated.

I can't find the reason for this error. I found a workaround by putting sphinx dependencies directly in Tox configuration, but I now have dependencies in both files and I'd like to understand what happened:

deps =
    sphinx == 3.2.1
    sphinx-autoapi == 1.5.1
commands = poetry run sphinx-build -d "{toxworkdir}/docs_doctree" documentation/source "{toxworkdir}/docs_out" --color -bhtml {posargs}
           python -c 'import pathlib; print("documentation available under file://\{0\}".format(pathlib.Path(r"{toxworkdir}") / "docs_out" / "index.html"))'

Answer 1

From a quick look, I would say:

The commands setting in [testenv:docs] overwrites the one from [testenv]. So I guess it is assumed that poetry install -v has been ran in the docs test environment but it has not. Anyway sphinx (and other dependencies) are not installed in the docs test environment.

You might want to add some variant of poetry install at the top of the commands setting in [testenv:docs].