Running in Kubernetes I have a Deployment with autoscaling based on Keda.
This Deployment runs long running tasks of several hours and moving to jobs is currently not an option.
Is there any way in Kubernetes to lock a node and/or pod while the long task is running to prevent them from terminating due to down scaling during the long tasks are running?
The only option I currently am aware of it using a long terminationGracePeriodSeconds and making sure the pod exists once the task ends.
You can use a podDisruptionBudget to achieve that. It will make sure a certain number of pods should always remain online and nodes containing these pods cannot be deleted as long as they don't have a replacement online.