aws auto-stop-idle does not detect papermill

149 views Asked by At

I am using papermill to parametrize jupyter notebook deployed on AWS Sagemaker. I also used this lifestyle configuration that will auto shutdown if there are no running/idle notebooks. Unfortunately, it does not detect the Papermill process and continues to shutdown after reaching the specified idle time. What do I need to do to keep Sagemaker alive until the completion of Papermill

1

There are 1 answers

0
Gili Nachum On

You could edit the idleness detection script to account for papermill processes.

Alternatively, if you have async jobs, which you can formulate as python code, you could use SageMaker processing jobs to execute them, which will not depend on your notebook instance being up.