Datadog

EvoHub receives Datadog monitor notifications through Datadog's Webhooks integration. A triggered monitor opens an EvoHub alert, and its recovery notification resolves it. Datadog's default payload works as it is; an optional custom payload gives EvoHub better severity and a stable alert ID.

Set it up

Create the integration

In EvoHub, go to On-Call → Integrations → + Add Integration, choose Datadog, pick an Escalation Policy and click Create Integration. Copy the Webhook URL.

Add a webhook in Datadog

In Datadog, go to Integrations → Webhooks and add a new webhook. Set Name to evohub and URL to the webhook URL. Save.

Mention it in your monitors

In each monitor's notification message, add @webhook-evohub. Datadog then notifies EvoHub when the monitor triggers and when it recovers.

Optional: custom payload

With the default payload, EvoHub reads the monitor's state from the notification title (for example [Triggered] or [Recovered]) and every alert is medium severity. For proper severity, tags as labels and a stable alert ID, turn on Use custom payload on the webhook and paste:

{
  "id": "$ID",
  "title": "$ALERT_TITLE",
  "body": "$EVENT_MSG",
  "priority": "$ALERT_PRIORITY",
  "alert_type": "$ALERT_TYPE",
  "alert_status": "$ALERT_TRANSITION",
  "alert_id": "$ALERT_ID",
  "alert_scope": "$ALERT_SCOPE",
  "tags": "$TAGS",
  "link": "$LINK"
}

The $… parts are Datadog template variables that Datadog fills in.

What EvoHub reads

EvoHub alert	Taken from
Title	`title`, with any `[Triggered on {…}]` prefix removed. A scope such as `{host:web-1}` in the prefix is kept as a `scope` label.
Description	`body`.
Severity	`alert_type` and `priority` — see below.
Labels	`tags`, split into `key:value` pairs; `alert_scope`, when sent, as the `scope` label.
Link	`link` (`$LINK`), the event in Datadog, when sent. It opens from the alert.
Fingerprint	`alert_id` together with `alert_scope` when both are present; `alert_id` alone when there is no scope; otherwise derived from the title (and scope), which is the same on the trigger and recovery notifications.

Severity

`alert_type`	`priority`	EvoHub severity
`error`	`P1`, `P2` or `normal`	critical
`error`	anything else	high
`warning`	any	medium
`info`, `success`	any	info
missing (default payload)	any	medium

Resolve and deduplication

A notification whose alert_status (custom payload) or title prefix (default payload) says Recovered resolves the matching alert. Nothing is opened.
Any other notification — triggered, re-triggered, warn, no data, renotify — opens an alert, or is recorded as Retriggered on the alert that is still open for the same monitor and scope.
Multi-alert monitors: with the default payload, each group (for example each host) gets its own EvoHub alert, because the scope is part of the fingerprint. With the custom payload above, alert_scope ($ALERT_SCOPE) does the same, so each group gets its own alert and resolves on its own. A custom payload without alert_scope keys on $ALERT_ID alone: every group shares one EvoHub alert, and the first group to recover resolves it.

Troubleshooting

Nothing arrives: check that the monitor message contains @webhook-evohub and that the webhook URL includes ?key=.
Recoveries open new alerts instead of resolving: use the custom payload so EvoHub gets a stable alert_id, or make sure the monitor title does not change between trigger and recovery.
Every alert is medium: switch to the custom payload above.

#Set it up

#Create the integration

#Add a webhook in Datadog

#Mention it in your monitors

#Optional: custom payload

#What EvoHub reads

#Severity

#Resolve and deduplication

#Troubleshooting

#Related