Docker Inside Docker oom_score_adj: invalid argument

Question

On Kubernetes, to build docker images, we use Docker Inside Docker (dind) where a docker daemon will be running as a privileged user

But some times times we have observed that the docker image creation will fail with the below error and I couldn't pin point the exact reason for this,

ERROR: Error writing header for "scripts": io: read/write on closed pipe
ERROR: Error writing tar: io: read/write on closed pipe
Removing temporary directory /tmp/s2i834345991
Removing directory '/tmp/s2i834345991'
Build failed ERROR: An error occurred: Error response from daemon: set shim OOM score: write /proc/9092/oom_score_adj: invalid argument: unknown

Based on docker documentation, by default oom_score_adj is set to -500, i.e the container can consume 50% of the memory, so am assuming that the OS/Kernel is killing the container when the memory usage exceeds 50% usage but I am not sure about it, because the exception says "Invalid Argument"

Did anyone faced this issue earlier and if yes please let us know the exact reason for the same, I went through many articles but couldn't conclude anything.

Answer 1

oom_score_adj errors can occur when Secrets are not correctly base64 encoded.

https://github.com/kubernetes/kubernetes/issues/30861

Answer 2

This error usually comes when binary content in secret is exposed as environment variable.

Kindly check if secret exposed as environment variable and if there is binary content present in the secret.