How to recover notebook runtime in unhealthy state

Question

A Runtime has been in "Unhealthy" state for ~12 hours and has not recovered.

I need to access it again because it has storage data I need, but there does not seem to be any way to recover an "Unhealthy" runtime (not through the UI, neither through API)

Answer 1

Are you able to access the terminal even in an unhealthy state? if yes you can try copying the files through terminal to a bucket with this command:

gsutil cp -R /content/\* gs://BucketName

The content is default location when you open the directory of the runtime, you can change it to any directory desired.

Reference: https://stackoverflow.com/a/73030233/19378826

Answer 2

A Runtime consists of 2 disks: boot disk and data disk. The runtime is a hidden VM in User Project, unfortunately the visibility is hidden so access is not possible (Currently is not possible to take an snapshot of disks or SSH into it). I have opened a FR to handle this case. We have a logs section that can shed some light of what may have been the issue.