We are running scheduled databricks jobs on a daily basis in Azure databricks and it runs successfully on all days. But today (29th Sept 2020), the job is failing within few seconds with Internal Error. The error message is given below:
Error while fetching notebook snapshot: HTTP request failed with status: HTTP/1.1 403 Forbidden
Has anyone else faced this issue and knows how to solve this?
On
This is due to Service Disruption (Started: September 29, 2020 00:04 UTC and Resolved: September 29, 2020 04:56 UTC) from Azure Databricks.
Here are the details from Status Notification from Azure Databricks Status Page:
One of the affected Infrastructure Component: Authentication
We are investigating an issue affecting user login.
The Azure Databricks Status Page provides an overview of all core Azure Databricks services. You can easily view the status of a specific service by viewing the status page. Optionally, you can also subscribe to status updates on individual service components, which sends an alert whenever the status you are subscribed to changes.
Reference: Azure Databricks Status page
We were able to identify and fix the issue. The jobs were setup under a person's user id who left the organization last weekend. Since his id was not active, it didn't have access to run the job and it was failing. After changing the job owner to another user id, it ran fine