APT lock contention on multi-stage (parallel) builds with cache

Question

I am using multi-stage build where both stages install APT packages (MWE Dockerfile below). I use --mount=type=cache on /var/cache/apt to avoid re-downloading of packages, in both stages. The cache directory is shared between the stages, which in itself is good, but: as they run in parallel, one or the other will fail with:

43.24 E: Could not get lock /var/cache/apt/archives/lock. It is held by process 0
43.24 E: Unable to lock directory /var/cache/apt/archives/

(the -o DPkg::Lock::Timeout=180 option would not help, that is for dpkg, not APT)

Now, I see three possible ways to solve this, but don't know how to realize either of them (without hacks, that is):

1. make APT wait for the lock to be released; is there an option for that? I could not duckduck anything out.
2. make the cache separate for each stage; I would not mind, as the packages installed are typically different (build vs. runtime), but don't see an option to --mount which would let me specify that.
3. make the build run serially; I really would not mind, that is not the issue. But again, could not find any option for buildkit enforcing that.

Advice or other suggestions would be appreciated.

---

This is a minimal Dockerfile which you can try yourself; it does not do anything useful but should trigger the issue.

FROM debian:bookworm-20230919 as builder
RUN rm -f /etc/apt/apt.conf.d/docker-clean
RUN --mount=type=cache,target=/var/cache/apt apt update && apt -y --no-install-recommends install build-essential
# build stuff, result in /tmp/build


FROM debian:bookworm-20230919 as production
RUN rm -f /etc/apt/apt.conf.d/docker-clean
RUN --mount=type=cache,target=/var/cache/apt apt update && apt -y --no-install-recommends install vim
# install stuff via --mount=target=/build,from=builder,source=/tmp/build

Answer 1

You can limit the sharing setting of a cache mount for situations like this:

RUN --mount=type=cache,sharing=locked,target=/var/cache/apt apt update && apt -y --no-install-recommends install build-essential

sharing: One of shared, private, or locked. Defaults to shared. A shared cache mount can be used concurrently by multiple writers. private creates a new mount if there are multiple writers. locked pauses the second writer until the first one releases the mount.

https://docs.docker.com/reference/dockerfile/#run---mount

Answer 2

How about copying the content across from one stage to the other?

FROM debian:bookworm-20230919 as builder

RUN rm -f /etc/apt/apt.conf.d/docker-clean
RUN apt update && apt -y --no-install-recommends install build-essential

FROM debian:bookworm-20230919 as production

COPY --from=builder /var/cache/apt/ /var/cache/apt/
COPY --from=builder /var/lib/apt/lists/ /var/lib/apt/lists/

RUN apt -y --no-install-recommends install vim

This will force the stages to build serially, but you say you're okay with that.

By copying across the cache and package lists you don't need to run apt update in the production stage. I'm assuming that this was the goal with caching /var/cache/apt/ between stages?