DataDog new Events API migration fails

Question

I'm upgrading one of our datadog monitors from event alert to event-v2 alert following this migration guide. Our monitors are created using Datadog Terraform provider.

Current monitor using event alert type:

resource "datadog_monitor" "guardduty_high_severity_findings" {
  name = "[tf] [${terraform.workspace}] AWS Guardduty Reporting High Severity Findings"
  type = "event alert"

  message = <<EOT
Some custom message
EOT

  escalation_message = <<EOT
<nil>
EOT

  tags              = ["system:ops", "service:aws"]
  query             = "events('sources:sns priority:all').rollup('count').last('xm') > 0"
  notify_no_data    = false
  no_data_timeframe = 0
  renotify_interval = 0
  notify_audit      = false
  timeout_h         = 0
  include_tags      = true
}

This monitor works as expected and Datadog alerts get triggered once AWS GuardDuty incident is detected.

Migrated monitor using event-v2 alert type:

resource "datadog_monitor" "guardduty_high_severity_findings" {
  name = "[tf] [${terraform.workspace}] AWS Guardduty Reporting High Severity Findings"
  type = "event-v2 alert"

  message = <<EOT
Some custom message
EOT

  escalation_message = <<EOT
<nil>
EOT

  tags              = ["system:ops", "service:aws"]
  query             = "events('sources:amazon_sns').rollup('count').last('xm') > 0"
  notify_no_data    = false
  no_data_timeframe = 0
  renotify_interval = 0
  notify_audit      = false
  timeout_h         = 0
  include_tags      = true
}

However, this gives an error when performing a terraform plan saying that the request has been rejected by DataDog due to invalid query format.

$ terraform plan
...
Error: error validating monitor from https://api.datadoghq.com/api/v1/monitor/validate: 400 Bad Request: {"errors": ["The value provided for parameter 'query' is invalid: invalid operator specified: "]}

  with datadog_monitor.guardduty_high_severity_findings,
  on monitors-static.tf line 106, in resource "datadog_monitor" "guardduty_high_severity_findings":
 106: resource "datadog_monitor" "guardduty_high_severity_findings" {

Appreciate if anyone could help with figuring out what's wrong here.

Answer 1

I found the same issue and it seems that the new type event-v2 alert doesn't support single quotes in the query param anymore. We need to use double quotes everywhere (plus need to escape them).

Example: "events(\"sources:amazon_sns\").rollup(\"count\").last(\"xm\") > 0"

Curiously, the support migration doc doesn't mention this breaking change.