# Datadog

EvoHub receives Datadog monitor notifications through Datadog's **Webhooks** integration. A triggered monitor opens an EvoHub alert, and its recovery notification resolves it. Datadog's default payload works as it is; an optional custom payload gives EvoHub better severity and a stable alert ID.

## Set it up

:::steps
### Create the integration
In EvoHub, go to **On-Call → Integrations → + Add Integration**, choose **Datadog**, pick an **Escalation Policy** and click **Create Integration**. Copy the **Webhook URL**.
### Add a webhook in Datadog
In Datadog, go to **Integrations → Webhooks** and add a new webhook. Set **Name** to `evohub` and **URL** to the webhook URL. Save.
### Mention it in your monitors
In each monitor's notification message, add `@webhook-evohub`. Datadog then notifies EvoHub when the monitor triggers and when it recovers.
:::

## Optional: custom payload

With the default payload, EvoHub reads the monitor's state from the notification title (for example `[Triggered]` or `[Recovered]`) and every alert is **medium** severity. For proper severity, tags as labels and a stable alert ID, turn on **Use custom payload** on the webhook and paste:

```json
{
  "id": "$ID",
  "title": "$ALERT_TITLE",
  "body": "$EVENT_MSG",
  "priority": "$ALERT_PRIORITY",
  "alert_type": "$ALERT_TYPE",
  "alert_status": "$ALERT_TRANSITION",
  "alert_id": "$ALERT_ID",
  "alert_scope": "$ALERT_SCOPE",
  "tags": "$TAGS",
  "link": "$LINK"
}
```

The `$…` parts are Datadog template variables that Datadog fills in.

## What EvoHub reads

| EvoHub alert | Taken from |
| --- | --- |
| Title | `title`, with any `[Triggered on {…}]` prefix removed. A scope such as `{host:web-1}` in the prefix is kept as a `scope` label. |
| Description | `body`. |
| Severity | `alert_type` and `priority` — see below. |
| Labels | `tags`, split into `key:value` pairs; `alert_scope`, when sent, as the `scope` label. |
| Link | `link` (`$LINK`), the event in Datadog, when sent. It opens from the alert. |
| Fingerprint | `alert_id` together with `alert_scope` when both are present; `alert_id` alone when there is no scope; otherwise derived from the title (and scope), which is the same on the trigger and recovery notifications. |

### Severity

| `alert_type` | `priority` | EvoHub severity |
| --- | --- | --- |
| `error` | `P1`, `P2` or `normal` | critical |
| `error` | anything else | high |
| `warning` | any | medium |
| `info`, `success` | any | info |
| missing (default payload) | any | medium |

## Resolve and deduplication

- A notification whose `alert_status` (custom payload) or title prefix (default payload) says **Recovered** resolves the matching alert. Nothing is opened.
- Any other notification — triggered, re-triggered, warn, no data, renotify — opens an alert, or is recorded as **Retriggered** on the alert that is still open for the same monitor and scope.
- Multi-alert monitors: with the default payload, each group (for example each host) gets its own EvoHub alert, because the scope is part of the fingerprint. With the custom payload above, `alert_scope` (`$ALERT_SCOPE`) does the same, so each group gets its own alert and resolves on its own. A custom payload without `alert_scope` keys on `$ALERT_ID` alone: every group shares one EvoHub alert, and the first group to recover resolves it.

## Troubleshooting

- **Nothing arrives**: check that the monitor message contains `@webhook-evohub` and that the webhook URL includes `?key=`.
- **Recoveries open new alerts instead of resolving**: use the custom payload so EvoHub gets a stable `alert_id`, or make sure the monitor title does not change between trigger and recovery.
- **Every alert is medium**: switch to the custom payload above.

## Related

- [Integrations overview](https://docs.evohub.io/integrations-overview.md)
- [Escalation policies](https://docs.evohub.io/escalation-policies.md)
- [Alerts and incidents](https://docs.evohub.io/alerts-and-incidents.md)