AWS Glue Workflow to trigger email on any ETL job failure

Question

In AWS Glue, I am executing a couple of ETL jobs using workflow, Now I want to inform business via email on the failure of any of the ETL jobs. I need help to get name of failed job and pass it to job which would trigger an email.

Answer 1

Step 1: Create a topic in Amazon SNS, with Protocol as Email as well as Create subscription and confirm subscription

Step 2: Create new CloudWatch Events rule with following custom event pattern under Event Source section;

{
  "source": [
    "aws.glue"
  ],
  "detail-type": [
    "Glue Job Run Status"
  ],
  "detail": {
    "state": [
      "FAILED",
      "ERROR",
      "TIMEOUT"
    ]
  }
}

Step 3: For Cloud Watch Event Rule, under the Targets section, choose Add targets, and then change the default Lambda function to SNS topic and choose name of SNS topic created in Step 1. Finish creation of this CloudWatch event rule.