[HDX-5132] Formula support for log/trace event sources by wrn14897 · Pull Request #2953 · ClickHouse/ClickStack

wrn14897 · 2026-08-20T19:26:29Z

Why

Chart formulas (letter-ref arithmetic like A / B * 100, shipped for metrics in HDX-5078/5079/5080) were gated to metric sources only. Users building charts on log/trace sources — e.g. an error-rate over two filtered counts — had no equivalent, even though the same display types (time series, table, number) support multi-series event charts today.

Linear: HDX-5132

What

Renderer (@hyperdx/common-utils)

New renderSelectListWithFormulas: compiles formula columns inline into the chart's single-scan SELECT for event sources — operand columns first (dropped when showOperandSeries: false; their aggregates still evaluate inside the formulas), then one derived column per formula compiled from the validated AST over the bare operand aggregate expressions.
Deliberately not routed through the composed metric UNION ALL + pivot path: event series all aggregate over the same scanned rows, so one scan suffices (per-series filters are already aggFnIf). Missing-data semantics match the existing events ratio toggle (divide(a, b)): countIf with no matches reads 0; avg-class aggregates read nan (a rendered gap).
Same behaviors as metric formulas: alias/expression column naming with collision suffixing, quote escaping, HAVING on formula output columns, render-time validation with structured errors, and formulas superseding seriesReturnType: 'ratio'.
Metric formula rendering is unchanged (still the composed path).

Chart editor (@hyperdx/app)

New isFormulaSourceKind (Metric | Log | Trace) gates the Add Formula button, series letter badges, Show input series toggle, save-time normalization, and form validation — previously metric-only. Session (and other) sources remain gated off.
getFormulaConfig no longer requires metricTables, so number formats and value-column counting handle event formulas.

Dashboards → raw-SQL conversion, tile alerts, and the chart renderers needed no changes — they were already formula-aware via config.formulas.

Testing

Renderer unit tests: 9 new event-formula cases (single scan / no UNION ALL, operand hiding, ratio precedence, name collisions, quote escaping, HAVING on formula alias, single-series formula, structured errors) + snapshot
Integration tests (real ClickHouse): time chart with gap semantics, hidden operands, grouped table + HAVING on the formula column, number shape — full queryChartConfig int suite green (99/99)
SQL template test: event formula chart converts to a macro-based raw-SQL template (snapshot)
App tests: log source exercises formula controls end-to-end (badges → add → save); session source verified still hidden; validation/normalization coverage extended to log/trace and a formula-incapable kind
make ci-lint ✓, make ci-unit ✓ (all packages)

Changeset included (minor bump for @hyperdx/common-utils + @hyperdx/app).

changeset-bot · 2026-08-20T19:26:33Z

🦋 Changeset detected

Latest commit: 7d6a488

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages

Name	Type
@hyperdx/common-utils	Minor
@hyperdx/app	Minor
@hyperdx/api	Minor
@hyperdx/otel-collector	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

vercel · 2026-08-20T19:26:34Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hyperdx-oss	Ready	Preview	Aug 21, 2026 2:31pm
hyperdx-storybook	Ready	Preview	Aug 21, 2026 2:31pm

github-actions · 2026-08-20T19:27:09Z

🟡 Tier 3 — Standard

Introduces new logic, modifies core functionality, or touches areas with non-trivial risk.

Why this tier:

Cross-layer change: touches frontend (packages/app) + shared utils (packages/common-utils)

Additional context: touches the query rendering engine lightly (127 lines, under the 150-line bar for Tier 4)

Review process: Full human review — logic, architecture, edge cases.
SLA: First-pass feedback within 1 business day.

Stats

Production files changed: 5
Production lines changed: 207 (+ 567 in test files, excluded from tier calculation)
Branch: warren/formula-support-for-events-metrics
Author: wrn14897

To override this classification, remove the review/tier-3 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

greptile-apps · 2026-08-20T19:29:34Z

Greptile Summary

The PR extends chart formulas from metric sources to log and trace event sources while retaining the existing formula-capable display types.

Compiles event formulas into the existing single-scan aggregate query.
Exposes formula controls, validation, normalization, and value formatting for log and trace sources.
Adds renderer, ClickHouse integration, raw-SQL conversion, and chart-editor coverage.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
packages/common-utils/src/core/renderChartConfig.ts	Adds validated inline formula projections over event-source aggregate expressions while preserving the composed metric path.
packages/app/src/components/ChartEditor/utils.ts	Extends formula validation and save normalization to metric, log, and trace sources.
packages/app/src/components/DBEditTimeChartForm/ChartEditorControls.tsx	Enables formula controls for supported event sources and formula-capable display types.
packages/app/src/components/DBEditTimeChartForm/ChartSeriesEditor.tsx	Displays positional formula-reference badges for metric, log, and trace series.
packages/app/src/source.ts	Applies formula projection metadata and number formatting to event-source formulas as well as metric formulas.
packages/common-utils/src/tests/queryChartConfig.int.test.ts	Covers event formula execution, gap semantics, hidden operands, grouped HAVING, and number output against ClickHouse.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Chart editor] --> B{Source kind}
  B -->|Metric| C[Composed multi-series query]
  B -->|Log or Trace| D[Single-scan event query]
  C --> E[Compile formula from operand series]
  D --> F[Inline operand aggregates]
  F --> E
  E --> G[Formula output columns]
  G --> H[Time series, table, or number renderer]

_{Reviews (5): Last reviewed commit: "feat: formula support for log/trace even..." | Re-trigger Greptile}

github-actions · 2026-08-20T19:33:27Z

E2E Test Results

✅ All tests passed • 308 passed • 1 skipped • 1169s

Status	Count
✅ Passed	308
❌ Failed	0
⚠️ Flaky	0
⏭️ Skipped	1

Tests ran across 4 shards in parallel.

View full report →

github-actions · 2026-08-20T20:48:12Z

Deep Review

✅ No critical issues found.

The event-source formula path reuses the same validated primitives as the shipped metric path (validateFormula → numeric/operator/seriesRef-only AST → compileFormulaAst, with quotedColumnName escaping aliases), so it introduces no new SQL-injection surface, no happy-path crash (operandAt is bounds-guarded by validation), and no change to metric formula rendering. Nothing rises to P2.

🔵 P3 nitpicks (3)

packages/common-utils/src/core/renderChartConfig.ts:1158 — the routing ternary re-tests Array.isArray(chartConfig.select), which hasInlineFormulas already guarantees, so the second check is dead.
- Fix: Gate the branch on hasInlineFormulas alone and drop the redundant Array.isArray re-check.
packages/common-utils/src/core/renderChartConfig.ts:1092 — formula-alias dedupe seeds seen only from operand select aliases, so a formula alias equal to a groupBy key or the time-bucket column name yields two identically-named columns in the SELECT (result rows are keyed by column name downstream); this mirrors the existing metric path.
- Fix: Seed the collision set with the rendered groupBy and time-bucket output names as well, or document that formula aliases must not collide with grouping columns.
packages/common-utils/src/core/renderChartConfig.ts:1054 — each referenced operand aggregate is re-rendered bare and textually embedded per reference, so an expression like A / (A + B) duplicates the same aggregate SQL multiple times, relying on ClickHouse common-subexpression elimination to avoid recomputation.
- Fix: If profiling shows repeated aggregate evaluation, hoist shared operand expressions into the WITH clause; otherwise leave as-is and note the reliance on CSE.

Reviewers (2): previous-comments (returned clean — the only human input is an LGTM approval; all other prior comments are bot output with no requested changes), orchestrator direct-diff analysis of the renderer and chart-editor changes.

Note: The deeper reviewer panel (correctness, security, adversarial, testing, maintainability, performance, TypeScript, project-standards, agent-native, learnings) was dispatched but had not returned before output was required; the findings above reflect only completed results plus direct code-grounded analysis and should not be read as the full panel's verdict.

Testing gaps: event formula combined with the series-limit CTE (renderSeriesLimitCte) under groupBy; Trace source exercising the inline path (description notes only Log was covered end-to-end); HAVING/ORDER BY referencing an operand column when showOperandSeries: false drops it from the projection.

pulpdrew

LGTM

Extends chart formulas beyond metric sources to log and trace sources on time series, table and number charts. Event formulas compile inline into the chart's single-scan SELECT (renderSelectListWithFormulas) instead of the composed metric UNION ALL + pivot path, with the same missing-data semantics as the existing events ratio toggle. The chart editor's formula controls (Add Formula, series letter badges, Show input series) and validation now gate on formula-capable source kinds rather than metric only.

Rebased onto #2953 (event-source formulas) and widened the API gate to match its editor gating: - isFormulaSourceKind (Metric | Log | Trace) moves from the app chart editor into common-utils core/utils (the app re-exports it), so the editor's Add Formula gating and the server-side surfaces share one predicate. - The external/MCP formula source gate now rejects only formula-incapable kinds (e.g. session) instead of everything non-metric, with an updated error message; MCP tool descriptions, the dashboards query-guide prompt, and the OpenAPI spec document the widened support (including a log error-rate example). - Tests: external round-trip of a trace-source formula tile plus a session-source rejection (kind-flip test now flips to session); MCP save + query_tile of a log-source formula tile asserting the inline-computed value with hidden operands; checkAlerts event-formula tile alert proving the threshold evaluates the formula value.

github-actions Bot added the review/tier-3 Standard — full human review required label Aug 20, 2026

vercel Bot deployed to Preview – hyperdx-storybook August 20, 2026 19:28

View deployment

vercel Bot deployed to Preview – hyperdx-oss August 20, 2026 19:31

View deployment

wrn14897 added the automerge label Aug 20, 2026

wrn14897 mentioned this pull request Aug 20, 2026

[HDX-5081] Formula support across API surfaces #2952

Merged

wrn14897 force-pushed the warren/formula-support-for-events-metrics branch from 5df19b8 to ac7ee10 Compare August 21, 2026 01:44

vercel Bot deployed to Preview – hyperdx-storybook August 21, 2026 01:46

View deployment

vercel Bot deployed to Preview – hyperdx-oss August 21, 2026 01:48

View deployment

wrn14897 force-pushed the warren/formula-support-for-events-metrics branch from ac7ee10 to a51f097 Compare August 21, 2026 04:46

vercel Bot deployed to Preview – hyperdx-storybook August 21, 2026 04:49

View deployment

vercel Bot deployed to Preview – hyperdx-oss August 21, 2026 04:51

View deployment

pulpdrew approved these changes Aug 21, 2026

View reviewed changes

wrn14897 removed the automerge label Aug 21, 2026

wrn14897 force-pushed the warren/formula-support-for-events-metrics branch from a51f097 to 67a4cdf Compare August 21, 2026 13:59

vercel Bot deployed to Preview – hyperdx-storybook August 21, 2026 14:02

View deployment

vercel Bot deployed to Preview – hyperdx-oss August 21, 2026 14:03

View deployment

wrn14897 force-pushed the warren/formula-support-for-events-metrics branch from 67a4cdf to 7d6a488 Compare August 21, 2026 14:28

vercel Bot deployed to Preview – hyperdx-storybook August 21, 2026 14:30

View deployment

vercel Bot deployed to Preview – hyperdx-oss August 21, 2026 14:31

View deployment

wrn14897 merged commit b1d8dc1 into main Aug 21, 2026
27 of 28 checks passed

Navigation Menu

Repository navigation

[HDX-5132] Formula support for log/trace event sources - #2953

[HDX-5132] Formula support for log/trace event sources#2953
wrn14897 merged 1 commit into
mainfrom
warren/formula-support-for-events-metrics

wrn14897 commented Aug 20, 2026

Uh oh!

changeset-bot Bot commented Aug 20, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Aug 20, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 20, 2026

Uh oh!

greptile-apps Bot commented Aug 20, 2026 •

edited

Loading

Important Files Changed

Flowchart

Uh oh!

github-actions Bot commented Aug 20, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 20, 2026 •

edited

Loading

Uh oh!

pulpdrew left a comment

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

wrn14897 commented Aug 20, 2026

Why

What

Testing

Uh oh!

changeset-bot Bot commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

vercel Bot commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 20, 2026

🟡 Tier 3 — Standard

Uh oh!

greptile-apps Bot commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Flowchart

Uh oh!

github-actions Bot commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Test Results

Uh oh!

github-actions Bot commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deep Review

Uh oh!

pulpdrew left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

changeset-bot Bot commented Aug 20, 2026 •

edited

Loading

vercel Bot commented Aug 20, 2026 •

edited

Loading

greptile-apps Bot commented Aug 20, 2026 •

edited

Loading

github-actions Bot commented Aug 20, 2026 •

edited

Loading

github-actions Bot commented Aug 20, 2026 •

edited

Loading