My /var ran full, the culprit was /var/lib/containers/storage/overlay-containers/<container-hash>/userdata/ctr.log of my gitlab container, which was 11GB.
I searched the documentation/interwebs in vain for any info on
- is it safe to delete the ctr.log (on a running container)?
- is there a
podmancommand to manage the log (e.g. clean, keep last x MB or days)? - do the
--logdriverand--logoptoptions topodman-runcontrol/affectctr.log, or is this some extra log? - if (through (3)) I can turn off logging, would I still see logs when attaching to the running container?
Regarding (1) it appears the answer is "yes", deleting ctr.log and restarting the container just made a new ctr.log.
This is not an answer to 1,2,3 or 4 but it prevents filling the disk space which seems the objective of the question.
see https://www.suse.com/support/kb/doc/?id=000019954