Pytest is stopping with no results and Test results shows FileNotFoundError

Question

I was trying to do a test that decompress files in a temp directory (made by a fixture), my function changes the current working directory in order to do some safe operations inside the folder.

Unfortunately, I have discovered that the test is doing something strange that causes the pytest to stop silently, no fail, no exception in the fixture, but it is actually an INTERNALERROR.

Here is a simple test file that shows a simple case:

import pytest


@pytest.fixture
def foo():
    import os

    os.mkdir("/tmp/remove-me")
    os.chdir("/tmp/remove-me")
    os.rmdir("/tmp/remove-me")


def test_func(foo):
    assert True


def test_other_thing():
    assert True

And here is what I got as test results on VSCode

============================= test session starts ==============================
platform linux -- Python 3.11.4, pytest-7.2.2, pluggy-1.0.0
rootdir: /home/mostafa/Documents/Cellusys/DCH_Project/ack_file_app, configfile: pytest.ini
plugins: mock-3.11.1, cov-4.1.0, anyio-3.6.2
collected 2 items

tests/general/test_z_crash_pytest.py::test_func 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/_pytest/main.py", line 270, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/_pytest/main.py", line 324, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/_pytest/main.py", line 349, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/_pytest/runner.py", line 112, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/_pytest/runner.py", line 125, in runtestprotocol
INTERNALERROR>     rep = call_and_report(item, "setup", log)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/_pytest/runner.py", line 224, in call_and_report
INTERNALERROR>     hook.pytest_runtest_logreport(report=report)
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/_pytest/terminal.py", line 524, in pytest_runtest_logreport
INTERNALERROR>     ] = self.config.hook.pytest_report_teststatus(report=rep, config=self.config)
INTERNALERROR>         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/mostafa/.local/lib/python3.11/site-packages/pluggy/_callers.py", line 34, in _multicall
INTERNALERROR>     next(gen)  # first yield
INTERNALERROR>     ^^^^^^^^^
INTERNALERROR>   File "/home/mostafa/.vscode/extensions/ms-python.python-2023.20.0/pythonFiles/vscode_pytest/__init__.py", line 206, in pytest_report_teststatus
INTERNALERROR>     cwd = pathlib.Path.cwd()
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/usr/local/lib/python3.11/pathlib.py", line 907, in cwd
INTERNALERROR>     return cls(os.getcwd())
INTERNALERROR>                ^^^^^^^^^^^
INTERNALERROR> FileNotFoundError: [Errno 2] No such file or directory

It is not related directly to the fixture, but to pytest method of doing the tests, here is another show case of the bug:

def func():
    import os

    os.mkdir("/tmp/remove-me")
    os.chdir("/tmp/remove-me")
    os.rmdir("/tmp/remove-me")


def test_func():
    func()


def test_other_thing():
    assert True

I have recorded a video for the behavior of the UI on this failure

Answer 1

I believe it is a bug in pytest, that is making assuptions about the current working directory for some reason.

As a work around, you have to save your cwd before entering the test that manipulate the cwd, and then restore it.

I have made this decorator to guard your cwd:

def guard_cwd(func):
    import os

    def wrapper(*args, **kwargs):
        cwd = os.getcwd()
        retval = func(*args, **kwargs)
        os.chdir(cwd)
        return retval

    return wrapper

The following examples will work perfectly:

@guard_cwd
def foo():
    import os

    os.mkdir("/tmp/remove-me")
    os.chdir("/tmp/remove-me")
    os.rmdir("/tmp/remove-me")


def test_func():
    foo()


def test_other_thing():
    assert True

It will work also with the fixtures (but keep the @pytest.fixture above it)

import pytest


@pytest.fixture
@guard_cwd
def foo():
    import os

    os.mkdir("/tmp/remove-me")
    os.chdir("/tmp/remove-me")
    os.rmdir("/tmp/remove-me")


def test_func(foo):
    assert True


def test_other_thing():
    assert True

Hope this could help some one.