Skip to content

docs: Add example for comparing current vs historical alert values#22062

Draft
nedi-app[bot] wants to merge 2 commits intomasterfrom
docs/alert-current-vs-historical-example
Draft

docs: Add example for comparing current vs historical alert values#22062
nedi-app[bot] wants to merge 2 commits intomasterfrom
docs/alert-current-vs-historical-example

Conversation

@nedi-app
Copy link
Copy Markdown
Contributor

@nedi-app nedi-app bot commented Mar 27, 2026

Summary

Adds a new example to the Alert Examples section in REFERENCE.md demonstrating how to compare current metric values against historical baselines using the two-alert pattern.

Changes

  • Added Example 4: Comparing Current vs Historical Values after Example 3 (Predictive Disk Full Alert)
  • Demonstrates the two-alert pattern required when comparing values (only ONE lookup line allowed per alert)
  • Shows how to create a historical baseline alert (1-hour average)
  • Shows how to reference the baseline in a current value alert using $alert_name syntax
  • Explains cross-alert value references and practical use cases
  • Renumbered subsequent examples (Network Packet Drops → Example 5, Z-Score → Example 6, etc.)

Motivation

Users often want to understand why an alert fired by comparing the current value against historical context (e.g., "current: X, average: Y"). This example addresses a common misconception that you can use multiple lookup lines in a single alert, showing the correct pattern of using separate coordinated alerts.

Testing

  • Example follows existing documentation style and formatting
  • All references to alert variables use correct syntax ($alert_name)
  • Example clearly states the single-lookup constraint

Audit Context

Session Query (Original):

ну так и как мне вывести, почему пришёл алерт? например:
сейчас значение: ХХХ
в среднем: УУУ

такое возможно передать в алерт?

Session Query (English Translation):

So how can I display why an alert came? For example:
current value: XXX
on average: YYY

Is it possible to pass this in an alert?

Decision Verdict: surgical_edit

The user wanted to know how to display dynamic values (current vs average) in alert messages. The documentation now provides a clear example showing that:

  1. Only ONE lookup line is allowed per alert
  2. To compare current vs historical values, you must create separate alerts
  3. Values can be referenced between alerts using $alert_name syntax
  4. This pattern enables meaningful comparisons in calc expressions and warn/crit thresholds

Summary by cubic

Adds a new example to REFERENCE.md that compares current metric values to a 1h baseline using the two-alert pattern, and shows how to include both numbers in alert messages. Clarifies the single lookup rule and demonstrates variable interpolation in the info line with cross-alert references.

  • New Features
    • Added “Example 4: Comparing Current vs Historical Values” with two alerts: 1h baseline and current 1m value.
    • Demonstrates cross-alert references ($alert_name) in calc/warn/crit and in info to output “current: X, average: Y”.
    • Notes the same-chart context requirement for references.
    • Renumbered following examples (Network Packet Drops → Example 5, Z-Score → Example 6, etc.).

Written for commit 2c9a769. Summary will update on new commits.

Add Example 4 to demonstrate the two-alert pattern for comparing current
values against historical baselines. Clarifies that only ONE lookup line
is allowed per alert definition and shows how to reference values across
alerts using the $alert_name syntax.

Addresses common confusion about displaying context in alerts when users
want to show both current and average values.
Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.
Architecture diagram
sequenceDiagram
    participant DB as Time-series Database
    participant Engine as Health Engine
    participant A1 as Alert 1 (Historical Baseline)
    participant A2 as Alert 2 (Current Value)

    Note over DB, A2: Netdata Health Evaluation Cycle (Periodic)

    rect rgb(240, 240, 240)
        Note right of A1: Alert 1: Calculate Baseline
        Engine->>DB: lookup: average -1h of user
        DB-->>Engine: Historical avg (e.g., 20%)
        Engine->>A1: Update state: $cpu_user_1h_average = 20
    end

    rect rgb(230, 240, 255)
        Note right of A2: Alert 2: Compare with Baseline
        Engine->>DB: lookup: average -1m of user
        DB-->>Engine: Current avg (e.g., 85%)
        
        Note over Engine, A2: NEW: Use cross-alert variable reference
        Engine->>A2: Evaluate expressions ($this vs $cpu_user_1h_average)
        
        alt calc: $this - $cpu_user_1h_average
            A2->>A2: Result: 65% deviation
        end

        alt warn: $this > 80 (Static Threshold)
            A2->>Engine: Status: WARNING
        else crit: $this > $cpu_user_1h_average * 4 (Dynamic Threshold)
            A2->>Engine: Status: CRITICAL
        end
    end

    Note over A2, Engine: Result: Alert info includes both current and historical context
Loading

Update Example 4 in REFERENCE.md to demonstrate how to display both
current and historical values in alert notifications using variable
interpolation in the info line.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant