overlayfs inside docker container

Question

Is it possible to mount an overlay fs inside a (privileged) docker container? At least my intuitive approach, which works fine outside of a container, fails:

> mkdir /tmp/{up,low,work,merged}
> mount -t overlay overlay -o lowerdir=/tmp/low/,upperdir=/tmp/up/,workdir=/tmp/work/ /tmp/merged/
mount: /tmp/merged: wrong fs type, bad option, bad superblock on overlay, missing codepage or helper program, or other error.

Additional information:

• Docker version 18.09.1, build 4c52b90
• Kernel 4.19.0-8-amd64
• Debian 10 (host and docker-image)

Answer 1

Found something that worked! Mounting the workdir and upperdir as tmpfs does the trick for me. Like so:

> mkdir /tmp/overlay
> mkdir /tmp/{low,merged}
> mount -t tmpfs tmpfs /tmp/overlay
> mkdir /tmp/overlay/{up,work}
> mount -t overlay overlay -o lowerdir=/tmp/low/,upperdir=/tmp/overlay/up/,workdir=/tmp/overlay/work/ /tmp/merged/ 

I'd still be interested in an explanation why creating an overlay w/o tmpfs fails within a docker container?

Answer 2

How to mount an overlayfs inside a docker container:

https://gist.github.com/detunized/7c8fc4c37b49c5475e68ef9574587eee

Basically, you need to run the container with either --privileged or the more secure --cap-add=SYS_ADMIN.

Answer 3

This is a bit of a guess but I suspect it is because docker is already using overlayfs and overlayfs is refusing to use upperdir as another overlayfs.

I suspect this may be due to whiteout files:

In order to support rm and rmdir without changing the lower filesystem, an overlay filesystem needs to record in the upper filesystem that files have been removed. This is done using whiteouts and opaque directories (non-directories are always opaque).

A whiteout is created as a character device with 0/0 device number. When a whiteout is found in the upper level of a merged directory, any matching name in the lower level is ignored, and the whiteout itself is also hidden.

To delete a file that exists in a lowerdir, overlayfs will create a whiteout file and hides all whiteout files (device number 0,0). This logically means that you cannot create a character device file with number 0,0 inside an overlayfs because that must be hidden by overlayfs itself.

If you were allowed to use an overlayfs as an upperdir it wouldn't be able to create blackout files and therefore wouldn't be able to rm or rmdir any files from the lower layers. Because it can't create a character device file with number 0,0 on another overlayfs.