Setup E-mail notification on failure run with DBX deployment

Question

I am deploying workflows to Databricks using DBX. Here I want to add a step which will send an e-mail to [email protected] whenever the workflow fails. The outline of my deloyment.yml file is as below:

deployments:
  - name: my_workflow
    
    schedule:
      quartz_cron_expression: "0 0 5 * * ?"
      timezone_id: "Europe/Berline"

    format: MULTI_TASK

    job_clusters:
      - job_cluster_key: "basic-job-cluster"
        <<: *base-job-cluster

    tasks:
      task_key: "my-task
      job_cluster_key: "basic-job-cluster"
      spark_python_task:
        python_files: "file://my_file.py"
 
    << Insert notification code here >>

I have not been able to find documentation about it, so if you can point me to that I will also be happy.

Answer 1

I found the solution in DBX documentation. For other looking add the following:

email_notifications:
  on_failure: [ "[email protected]" ]

Link to documentation: https://dbx.readthedocs.io/en/latest/reference/deployment/

Answer 2

Update your deployment.yml file to below:

deployments:
  - name: my_workflow
    
    email_notifications:
          on_start: [ "[email protected]" ]
          on_success: [ "[email protected]" ]
          on_failure: [ "[email protected]" ]
 
    schedule:
      quartz_cron_expression: "0 0 5 * * ?"
      timezone_id: "Europe/Berline"

    format: MULTI_TASK

    job_clusters:
      - job_cluster_key: "basic-job-cluster"
        <<: *base-job-cluster

    tasks:
      task_key: "my-task
      job_cluster_key: "basic-job-cluster"
      spark_python_task:
        python_files: "file://my_file.py"