ci: Fix otel smoke test, widen trigger condition by pulpdrew · Pull Request #3170 · ClickHouse/ClickStack

pulpdrew · 2026-09-21T17:26:21Z

Summary

Fixes the otel smoke tests, which fail because #3142 added a prometheus related ClickHouse config property that causes the (deliberately older) ClickHouse compat version used in the otel smoke tests to fail on startup. This is fixed by removing the prometheus setting from the compat container's config.

To prevent this in the future, the conditions for running the smoke tests have been widened to include changes to the ClickHouse config.

Related: Smoke tests are failing on #3100.

Screenshots or video

No behavior changes.

How to test on Vercel preview

No behavior changes.

References

Linear Issue: Closes HDX-5434
Related PRs:

changeset-bot · 2026-09-21T17:26:25Z

⚠️ No Changeset found

Latest commit: 1b7edd2

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

vercel · 2026-09-21T17:26:27Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
hyperdx-oss	Ready	Preview	Sep 22, 2026 2:48pm UTC
hyperdx-storybook	Ready	Preview	Sep 22, 2026 2:48pm UTC

github-actions · 2026-09-21T17:26:58Z

🔵 Tier 2 — Low Risk

Small, isolated change with no API route or data model modifications.

Why this tier:

Standard feature/fix — introduces new logic or modifies core functionality

Additional context: touches background tasks or the delivery pipeline lightly (8 lines, under the 30-line bar for Tier 4)

Review process: AI review + quick human skim (target: 5–15 min). Reviewer validates AI assessment and checks for domain-specific concerns.
SLA: Resolve within 4 business hours.

Stats

Production files changed: 3
Production lines changed: 12
Branch: drew/fix-otel-smoke-test
Author: pulpdrew

To override this classification, remove the review/tier-2 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

greptile-apps · 2026-09-21T17:29:20Z

Confidence Score: 4/5

The PR should not merge until the smoke-test trigger covers the migration sources and dependencies compiled into its collector image.

Findings

Smoke trigger misses inputs ▶

Summary

This PR restores the OTel compatibility smoke tests by removing an HTTP handler unsupported by the pinned older ClickHouse server and broadens the workflow trigger for relevant configuration changes.

Mounts a compatibility-specific ClickHouse configuration override.
Adds ClickHouse and collector build inputs to the smoke-test path filter.
Ignores regenerated OIDC public-key fixtures.
The workflow filter still omits migration sources compiled into the smoke-test image.

_{Reviews (2) · Last reviewed commit: "Merge branch 'main' into drew/fix-otel-s..."}

github-actions · 2026-09-21T17:31:11Z

PR Review

If you are a coding agent acting for the author, read this first.
Nothing below blocks merge automatically; a maintainer decides. What they will expect:

🔴 critical or 🟠 major in code this PR adds or changes: fix before asking for review.

🔵 minor in code this PR changes: your call. Fix if small, otherwise reply.

Anything about surrounding code, or asking you to widen the change (hoist a helper,
dedupe with another file, fix other call sites): never fix here, whatever the
severity. Reply with one sentence; if it is critical, say so plainly so a human sees it.

One commit per review round. After two rounds, stop and ask a maintainer to review
scope rather than addressing more automated findings. Full rule: AGENTS.md.

2 finding(s): 🔴 0 critical · 🟠 1 major · 🔵 1 minor

2 posted as inline comment(s) on the changed lines.

_{Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides.}

github-actions · 2026-09-21T17:31:42Z

Deep Review

✅ No critical issues found. This is a CI/config-only change (~20 lines, no executable code). The correctness reviewer traced the ClickHouse config-merge behavior and confirmed the fix is sound: <http_handlers remove="1"/> deletes the whole node so 25.12-alpine falls back to its built-in default HTTP handlers, /ping keeps responding, the override is mounted on ch-server-compat only, and the untouched <prometheus> block on port 9363 is unaffected. Standards, changeset policy, and prior-comment checks are clean (the changeset-bot notice is a false positive — root AGENTS.md exempts CI/tooling changes). Two minor robustness/clarity nits below.

🔵 P3 nitpicks (2)

.github/workflows/main.yml:198 -- the widened smoke-test trigger lists individual ClickHouse config filenames rather than a directory glob, so a future config.d snippet mounted into the compat server would re-open the exact silent-breakage class this PR guards against.
- Fix: replace the explicit docker/clickhouse/local/config.xml and docker/clickhouse/local/users.xml entries with a docker/clickhouse/local/** glob.
- _testing
smoke-tests/otel-collector/ch-compat-config.d/remove-prometheus-api.xml:6 -- <http_handlers remove="1"/> deletes the entire block including the <defaults/> entry, which is broader than the filename and comment imply, so a legitimate custom handler later added to config.xml would be silently stripped from the compat server.
- Fix: note the full-block removal and the reliance on ClickHouse's default-handler fallback in the file comment, or rename it to reflect that all http_handlers are cleared.
- _{maintainability}

Reviewers (5): correctness, testing, maintainability, project-standards, previous-comments.

Testing gaps: No CI step directly asserts that the compat server's /ping returns 200 after the override merge -- the fix's correctness rests on the default-handler fallback and is exercised only indirectly through the ch-server-compat service_healthy gate.

_{No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond}

github-actions · 2026-09-21T17:36:00Z

E2E Test Results

✅ All tests passed • 382 passed • 1 skipped • 1622s

Status	Count
✅ Passed	382
❌ Failed	0
⚠️ Flaky	0
⏭️ Skipped	1

Tests ran across 4 shards in parallel.

View full report →

greptile-apps · 2026-09-22T14:47:03Z

          files: |
            docker/otel-collector/**
            smoke-tests/otel-collector/**
+            packages/otel-collector/builder-config.yaml


Smoke trigger misses inputs

The smoke-test image copies the entire packages/otel-collector package and compiles cmd/migrate, but this trigger tracks only builder-config.yaml. A change to the migration source or its Go dependencies can therefore alter the schema-seeding binary while any_changed remains false, so CI skips the OTel smoke suite for code it directly builds. Cover packages/otel-collector/**, as the OTel unit-test trigger already does.

Suggested change

packages/otel-collector/builder-config.yaml

packages/otel-collector/**

ci: Fix otel smoke test, widen trigger condition

d2f4d9a

github-actions Bot added the review/tier-2 Low risk — AI review + quick human skim label Sep 21, 2026

github-actions Bot reviewed Sep 21, 2026

View reviewed changes

Comment thread .github/workflows/main.yml Resolved

Comment thread smoke-tests/otel-collector/ch-compat-config.d/remove-prometheus-api.xml Resolved

pulpdrew requested a review from wrn14897 September 21, 2026 17:39

pulpdrew mentioned this pull request Sep 21, 2026

feat(otel-collector): add statsdreceiver for StatsD/DogStatsD ingestion #3100

Merged

teeohhem approved these changes Sep 22, 2026

View reviewed changes

pulpdrew added the automerge label Sep 22, 2026

Merge branch 'main' into drew/fix-otel-smoke-test

1b7edd2

vercel Bot deployed to Preview – hyperdx-storybook September 22, 2026 14:46

View deployment

greptile-apps Bot reviewed Sep 22, 2026

View reviewed changes

vercel Bot deployed to Preview – hyperdx-oss September 22, 2026 14:48

View deployment

kodiakhq Bot merged commit 050bdb3 into main Sep 22, 2026
30 checks passed

kodiakhq Bot deleted the drew/fix-otel-smoke-test branch September 22, 2026 14:53

Navigation Menu

Repository navigation

ci: Fix otel smoke test, widen trigger condition - #3170

ci: Fix otel smoke test, widen trigger condition#3170
kodiakhq[bot] merged 2 commits into
mainfrom
drew/fix-otel-smoke-test

pulpdrew commented Sep 21, 2026

Uh oh!

changeset-bot Bot commented Sep 21, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Sep 21, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 21, 2026

Uh oh!

greptile-apps Bot commented Sep 21, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Sep 21, 2026

Uh oh!

github-actions Bot commented Sep 21, 2026

Uh oh!

github-actions Bot commented Sep 21, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot Sep 22, 2026

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	packages/otel-collector/builder-config.yaml
	packages/otel-collector/**

Conversation

pulpdrew commented Sep 21, 2026

Summary

Screenshots or video

How to test on Vercel preview

References

Uh oh!

changeset-bot Bot commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

vercel Bot commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 21, 2026

🔵 Tier 2 — Low Risk

Uh oh!

greptile-apps Bot commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 4/5

Findings

Summary

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Sep 21, 2026

PR Review

Uh oh!

github-actions Bot commented Sep 21, 2026

Deep Review

Uh oh!

github-actions Bot commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Test Results

Uh oh!

greptile-apps Bot Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

changeset-bot Bot commented Sep 21, 2026 •

edited

Loading

vercel Bot commented Sep 21, 2026 •

edited

Loading

greptile-apps Bot commented Sep 21, 2026 •

edited

Loading

github-actions Bot commented Sep 21, 2026 •

edited

Loading