Skip to content

Alert rules

Alert rules are part of Pro and Team. A free account’s uptime checks show their results but send no alerts.

Every rule Cloud Viewer evaluates for your team. To follow an alert from the notification to the cause, see When something needs attention.

Column Meaning
Notification on iPhone, the title of the push notification, after Critical: or Warning:. Android shows the alert’s summary as the title instead
Rule the rule_key first, then the rule’s internal name. The app shows the rule_key under the alert’s title (Warning · disk_space_low), webhooks send it as alert.rule, PagerDuty as custom_details.rule
Severity critical or warning
Fires when the condition, and how long it must hold before the alert fires

Rules are evaluated every 30 seconds. The thresholds are the same for every team. The only values you set yourself are a check’s latency budget and a job’s maximum age.

For cloud servers in a connected project, from the metrics Hetzner reports for them. Dedicated servers have no Hetzner metrics, so these two rules do not cover them.

Notification Rule Severity Fires when
server offline host_offline · HostOffline critical Hetzner lists the server as running but has sent no metrics for it for more than 3 minutes. Keeps firing for as long as that lasts; resolves when metrics return, or when the server is powered off or deleted
CPU saturated cpu_saturated · CPUSaturated warning Hetzner’s CPU value for the server is above 95 % for 15 minutes
  • server offline never fires for a server that Hetzner reports as off, including one powered off between two syncs, for a deleted server, while the Hetzner API cannot be reached or refuses the project’s token (the token alert covers that), or when Hetzner goes quiet for every team at once.
  • A server powered off without you doing it does not page either: Hetzner reports it as off and gives no reason. An uptime check on its address watches for that.

For servers that run the agent. The agent sends metrics every 30 seconds; this rule fires when they stop arriving.

Notification Rule Severity Fires when
stopped reporting agent_silent · AgentSilent critical A dedicated server, or a cloud server that is not linked to its Hetzner resource, has sent no agent metrics for more than 5 minutes
stopped reporting agent_silent · AgentSilent warning A cloud server’s agent has sent no metrics for more than 5 minutes while Hetzner still delivers the server’s own metrics
  • The alert keeps firing for as long as the server is silent and resolves with the first new report.
  • Critical where the agent is the only signal: a dedicated server, or a cloud server registered with its own agent token rather than a fleet token, which Cloud Viewer cannot match to its Hetzner resource. The alert cannot tell a server that is down from an agent that stopped; an uptime check on the server’s address can.
  • Warning for a cloud server that Hetzner still reports: the server runs, but until the agent reports again there are no disk, memory or log alerts for it. systemctl status cloudviewer-agent on the server shows why.
  • When Hetzner is silent too, only server offline fires, not both. A cloud server that is not linked to its Hetzner resource can get both.
  • It does not fire for a server that has never reported (Waiting for agent) or only sends job reports, a retired server, a server whose tokens are revoked, a team that is not on Pro or Team, a linked cloud server that Hetzner does not report as running (powered off, being rebuilt), or a check target.
  • If every team’s agents go silent at once, the cause is on our side, and the rule holds back instead of paging everyone.

For cloud and dedicated servers that run the agent, from what it measures on the host.

Notification Rule Severity Fires when
disk space low disk_space_low · DiskSpaceLow warning A filesystem is more than 85 % full for 10 minutes
disk almost full disk_space_critical · DiskSpaceCritical critical A filesystem is more than 95 % full for 5 minutes
memory pressure memory_pressure · MemoryPressure warning Memory use is above 90 % for 10 minutes
  • Disk rules apply to each mount point separately. Memory-backed and image filesystems (tmpfs, devtmpfs, ramfs, squashfs, overlay and similar) are skipped.
  • Memory use is total minus available memory, so page cache the kernel can free does not count.

From the warning-level journald lines the agent sends for log context. Only lines that were stored count, after the per-server limit of 120 lines a minute.

Notification Rule Severity Fires when
critical log lines log_critical_lines · CriticalLogLine warning At least one line at priority 0–2 (emergency, alert, critical: for example a kernel oops or an out-of-memory kill) in the last 10 minutes. Fires at once and resolves 10 minutes after the last such line
error-log burst log_error_burst · ErrorLogBurst warning More than 30 lines at priority 0–3 (error or worse) within 15 minutes, for 5 minutes

SSH and sudo lines have no rule.

From your job reports.

Notification Rule Severity Fires when
job failed user_job_failed · UserJobFailed warning The job’s latest report has a non-zero exit code, for 5 minutes
job stale user_job_stale · UserJobStale warning The job’s last successful run is older than the max_age_seconds it declared, for 5 minutes
  • A job that declares no max_age_seconds only gets job failed.
  • Stale counts from the last successful run. A job that has never succeeded never goes stale.

From your uptime checks. A check is Down only when both networks agree. Mixed and No data never alert.

Notification Rule Severity Fires when
unreachable from outside check_down · CheckDown critical The check is Down for 1 minute
certificate expiring check_cert_expiring · CheckCertExpiring warning The certificate has less than 14 days left, for 15 minutes
certificate expires imminently check_cert_expiring_critical · CheckCertExpiringCritical critical The certificate has less than 3 days left, for 15 minutes
slower than its budget check_slow · CheckSlow warning The response time from either network is above the check’s latency budget for 10 minutes. Only for checks with a budget, which you set in the portal

The notification says “unreachable” rather than “down”, because it can appear next to a resource that Hetzner still reports as running.

While the critical alert fires, the matching warning is not sent.

While this fires it suppresses on the same
disk_space_critical disk_space_low server and mount point
check_cert_expiring_critical check_cert_expiring check
Channel Severities Muted alerts
The app: Active alerts, History, the resource’s screen all shown, marked as muted
Push notifications all; critical ones are time-sensitive on iPhone not sent
Email digest warning and critical, or critical only never included
Alert targets: webhook, PagerDuty warning and critical, or critical only, per target never sent

The digest and alert targets never carry alerts with severity info. None of the rules on this page is info. Setting up the digest and targets: Send alerts where you work.

  • Durations: 1 hour, 4 hours or 24 hours (the web app says 1 day). Every app can end a mute early with Unmute; the iPhone and Android apps can also Extend mute.
  • A muted alert keeps firing. It stays visible, marked as muted, and is recorded in the history.
  • Any team member can mute an alert in the apps.
  • Alert history is kept for 90 days.
  • History in the app shows the last 30 days. Fired N× this week counts the last 7 days.
  • The Claude connector’s get_alert_history returns the last 7 days.
  • Load balancers, floating IPs, volumes, networks and primary IPs have no rules of their own. To watch a load balancer or a floating IP, add an uptime check anchored to it in the portal; its alerts then appear on that resource.
  • CPU on dedicated servers: the CPU rule reads Hetzner’s Cloud metrics, which dedicated servers do not have.
  • Load average, disk I/O and network traffic: charted, no rule.