Docker-in-Docker Pytest Failing with temp paths

Question

We do all our development in containers, and one of our applications manages docker via docker-py in production. So testing becomes docker-ception (docker in docker). We mount the host docker.sock via compose for development (ie mount the volume /var/run/docker.sock:/var/run/docker.sock which docker-py uses directly.

Many tests use the tmp_path pytest fixture (or library-specific temp paths) and those tests fail with a collection of random errors - mostly file not found, module not found etc.

Running the same procedures by hand and outside the dev container work.

Why come?

Answer 1

TLDR

the /tmp folder needs to be mounted as a volume in the dev environment. adding

volumes:
  - /var/run/docker.sock:/var/run/docker.sock
  - /tmp:/tmp

makes the temporary assets from testing available to the child container.

Long-winded answer

Docker-in-docker using the host docker.sock creates the containers as siblings, not nested children. This means the docker filesystem references from parent->child won't work because they are accessing the same filesystem as the host.

               #### /tmp files in parent not accessible to child ####

# on host machine                    # dev-container                                           
/                                    /                                                    
|__/home/sven/Repos/dev              |_/app  # mounted to /home/sven/Repos/dev in host    
|__/tmp # sub-container looks here!  |_/tmp  # pytest creates tmp files here                                            
|__/lib                              |... etc
|... etc 

When Debugging

• remember to check the filetype of the files inside the child container to see if they actually mounted. Docker will create empty directories for each file it cannot find on the host. So if main.py did not actually mount, there will be a main.py directory in the container.