Skip to content

Conversation

@vesalaakso-oura
Copy link

@vesalaakso-oura vesalaakso-oura commented Sep 15, 2025

What is this feature?

I built a smoothing transformer that cleans up noisy time series data. It uses the ASAP algorithm to pick which data points to keep. This works similarly to Datadog's autosmooth() function.

Why do we need this feature?

This was actually requested back in 2019 in issue #15022. I was migrating dashboards from Datadog to Grafana and we noticed that Datadog’s autosmoothing feature, which made graphs easier to read was not available in Grafana, so we had to find a workaround..

Who is this feature for?

Users coming from Datadog who miss the autosmooth functionality

Which issue(s) does this PR fix?:

Fixes #15022

Special notes for your reviewer:

I've included a gif demo of the feature in action 😎

Please check that:

  • It works as expected from a user's perspective.
  • If this is a pre-GA feature, it is behind a feature toggle. (N/A - standard transformer)
  • The docs are updated, and if this is a notable improvement, it's added to our What's New doc. **NOTE, I believe only maintainers can update the CMS.

What I built:

  • Smoothing transformer using the ASAP downsampling algorithm (same one mentioned in the original issue)
  • Always keeps the first and last data points so you get the full time range (otherwise graph might look incomplete)
  • Simple resolution setting (10-1000 points) to control how aggressive the smoothing is
  • Test cases covering all the edge cases I could think of
  • Docs with examples

Demo gif:
smoothing

@vesalaakso-oura vesalaakso-oura requested review from a team as code owners September 15, 2025 09:23
@vesalaakso-oura vesalaakso-oura requested review from alexjonspencer1, ashharrison90 and gelicia and removed request for a team September 15, 2025 09:23
@CLAassistant
Copy link

CLAassistant commented Sep 15, 2025

CLA assistant check
All committers have signed the CLA.

@github-actions github-actions bot added this to the 12.3.x milestone Sep 15, 2025
Added a smoothing transformer to help clean up noisy time series data.
It uses the ASAP algorithm to pick the most important data points while
keeping the overall shape and trends intact.

The transformer always keeps the first and last points so you get the
complete time range. I also added a test for it.
@vesalaakso-oura vesalaakso-oura force-pushed the feature/smoothing-transformer branch from 27d6e18 to e089343 Compare September 16, 2025 05:49
@gelicia
Copy link
Contributor

gelicia commented Sep 16, 2025

Hello @vesalaakso-oura , thank you for this PR. We were wondering if you were aware of the "resample" expression available already in Grafana. It works a bit differently than this PR, but it may solve your needs. Take a look at a demo here and let me know what you think! https://play.grafana.org/d/krmmxss/resampling-demo?folderUid=ae9kn9tsqxe68c&orgId=1&from=now-6h&to=now&timezone=utc

@vesalaakso-oura
Copy link
Author

vesalaakso-oura commented Sep 17, 2025

Hello @vesalaakso-oura , thank you for this PR. We were wondering if you were aware of the "resample" expression available already in Grafana. It works a bit differently than this PR, but it may solve your needs. Take a look at a demo here and let me know what you think! https://play.grafana.org/d/krmmxss/resampling-demo?folderUid=ae9kn9tsqxe68c&orgId=1&from=now-6h&to=now&timezone=utc

Hello @gelicia, We believe resample and ASAP solve different problems. Resample changes timestamps to a consistent interval so series can line up for math but that can blur spikes or hide outliers. ASAP smooths noise while keeping trends and outliers visible (like Datadog's autosmoother). We believe ASAP and resample just complement each other.

@kylebrandt
Copy link
Contributor

kylebrandt commented Sep 17, 2025

Another possible option using SQL Expressions, keeps spike while smoothing. Might not be as robust as the autosmoother example, and takes a couple more params:

(Note, when there are spikes, both series have them, one just visually covers the other).
image

image
WITH w AS (
  SELECT
    time,
    a,
    __value__  AS val,

    /* rolling center & spread over the last 20 rows */
    AVG(__value__) OVER (
      PARTITION BY a
      ORDER BY time
      ROWS BETWEEN 20 PRECEDING AND CURRENT ROW
    ) AS mu,

    STDDEV_POP(__value__) OVER (
      PARTITION BY a
      ORDER BY time
      ROWS BETWEEN 20 PRECEDING AND CURRENT ROW
    ) AS sigma,

    /* neighbors for "extra-picky" local-extremum check */
    LAG(__value__, 1)  OVER (PARTITION BY a ORDER BY time) AS prev_val,
    LEAD(__value__, 1) OVER (PARTITION BY a ORDER BY time) AS next_val
  FROM A
)
SELECT
  time,
  a,
  val AS Value,
  CASE
    WHEN sigma IS NULL OR sigma = 0 THEN mu
    /* keep spikes/dips when z >= 2.5 and it's a local extremum (or at edges) */
    WHEN ABS(val - mu) >= 2 * sigma
         AND (
              prev_val IS NULL OR next_val IS NULL
              OR (val > prev_val AND val > next_val)
              OR (val < prev_val AND val < next_val)
         )
      THEN val
    ELSE mu
  END AS smoothed
FROM w
ORDER BY a, time;

Oh, and needs transform for viz panel:
image

@vesalaakso-oura
Copy link
Author

Hi @kylebrandt , thanks a lot for taking the time to write up the SQL approach and sharing the example. I really appreciate the effort you put into it. 🙇

My main concern is scalability in larger orgs. If every team needs to ship or maintain their own SQL query, it quickly becomes a burden. It also makes it harder to convince teams migrating from datadog, where autosmoothing is just available out of the box. Having a built-in ASAP transformation would not only help internally but could also tip other organisations out there to consider moving away from datadog to the grafana stack.

ASAP itself is well researched. The original paper shows that ASAP can spot meaningful trends and outliers while suppressing noise, so having it as a built-in transformation would be an easier sell.

@vesalaakso-oura
Copy link
Author

Hello @gelicia, is this a feature we can proceed with, or do you have any concerns or alternative ways to implement it?

@gelicia
Copy link
Contributor

gelicia commented Sep 23, 2025

@vesalaakso-oura thank you for your patience, adding a new transformation to the catalog is something we have to discuss internally so we are checking in with folks! I'll let you know what we decide.

@github-actions
Copy link
Contributor

This pull request has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 2 weeks if no further activity occurs. Please feel free to give a status update or ping for review. Thank you for your contributions!

@github-actions github-actions bot added the stale Issue with no recent activity label Oct 24, 2025
@gelicia gelicia removed the stale Issue with no recent activity label Oct 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature request] ASAP smoother

4 participants