How to limit the count of volume mount re-try when 'FailedMount' happens

Question

When kubelete failed to mount volume as expected, it will keep retrying until succeed or the pod is deleted. Is there any way we can limit how many times to retry and just exit?

Warning  FailedMount  2m18s (x2 over 15m)  kubelet, xxx  Unable to attach or mount volumes: unmounted volumes=[volume1, volume2], unattached volumes=[volume1, volume2]: timed out waiting for the condition

Answer 1

I checked the code for the latest Kubernetes release and I unfortunately was not able to find code responsible for that. Having the exact version that you are using would definitely limit the search.

I'm not aware of way to limit retries, nor I was unable to find way to do that using standard Kubernetes tools like some parameter that is directly responsible for setting this in a form of number of retry or as time period. I do not know what is your goal here and I don`t see much of use case at this point. Having those attempts generally agrees how Kubernetes works.

Possible, but not ideal workaround for this would be to write a simple script that monitors externally events for the volume attachment and aborts the operation after a certain amount of time or attempts.

With kubectl that would be matter of using kubectl get events piped to grep FailedMount and corresponding volume names, this piped to wc -l (or something similar) to obtain number of lines. After that you'll need a condition to check if the number of failed retries has been reached and if that would reach is desired state do kubectl delete.