Overlayfs volume to docker container

Question

I have a set of directories that I would like to overlay mount into a container:

# On host
/opt/a
  - bin
  - lib
/opt/b
  - bin
  - lib
/opt/c
  - bin
  - lib

# In container
/usr/local
  - bin
  - lib
  - cuda

I am able to create an overlayfs mount on the host machine for /opt/{a,b,c} and mount that as a volume to /usr/local in the container, but then /usr/local/cuda will be inaccessible in the container.

I'm able to achieve this directly with systemd-nspawn with the following:

systemd-nspawn --overlay /opt/a:/opt/b:/opt/c:/usr/local <other flags>

This makes all the files available as a merged mount in /usr/local in the container, with changes written to /opt/c on the host.

Is it possible to easily achieve what I want with docker?

Answer 1

No. If you're familiar enough with the host tools to attempt this, consider running it in a chroot environment instead.

Docker has no way to merge image content, volume content, and host-system directories. Mounts work the same way as the normal Unix mount(8) command: the thing you're mounting completely hides whatever was there before and you only see the mounted contents.

It's also unusual to run a container that heavily depends on host-system content. Typical practice is for an image to include all of the application and library code that's needed to run the software; that way you can run it on a different system even if that software isn't present on the host. That is, it's more typical to install the software you need directly in your Dockerfile, with something like apt-get install for Debian/Ubuntu-based images. If the various /opt things are things you'd build from source, you can use a multi-stage build to install them from source, and then in the final phase COPY --from=opt-a-build into the /usr/local tree.