Make the 04780 index-analysis allocation oracle the minimum of several runs - #114608
Conversation
The test compares `MemoryAllocatedWithoutCheckBytes` of a dotted-constant `EXPLAIN indexes = 1` against a no-dots control with a 150% threshold, but it measured each arm with a single query. Whichever query happens to be the first to touch a cache or spin up a thread pool absorbs a transient multi-megabyte allocation (locally reproducible: the first query after server start reports 8-22 MB against a 2.8 MB steady state), and the dotted arm always runs first, so such a one-off lands on it and inflates the ratio arbitrarily. In CI the test failed with a 4216780-byte dotted arm against a 22224-byte control on several unrelated PRs. Measure each arm three times and take the minimum: a genuine quadratic regression is deterministic and shows up in every run, so the oracle keeps discriminating, while a one-off transient can no longer fail the test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Workflow [PR], commit [291359e] Summary: ✅ AI ReviewSummaryThis PR makes the allocation oracle in Final Verdict
|
Build profile diff (arm_release)Comparing ✅ No significant changes. Binary sizes
Only the stripped binary is compared: the official master build keeps debug symbols while PR builds strip them, so the other binaries differ by construction. Compile time of recompiled translation units6 translation units recompiled, 5 s compile time in total, 6 of them have a recent master baseline. |
robot-ch-test-poll2
added
the
pr-synced-to-cloud
robot-clickhouse
added
the
pr-backports-created
`04780_json_subcolumn_index_match_not_quadratic` failed with
FAIL: tokens index analysis over a constant with 100000 dots allocated
4216780 bytes, more than 150% of the no-dots control (22224 bytes)
and the harness's own reruns then passed twice, both with and without
the randomized settings. This branch still measures a single run,
while `master` takes the minimum of three (#114608, 2026-08-13) - and
that commit's comment names this exact pair of byte counts as the
failure it was written for. #114608 is now labelled for 26.6 and 26.7,
so the oracle will stop being a single sample once it lands; this
merge re-triggers the workflow in the meantime.
robot-clickhouse
added
the
pr-must-backport-synced
Backport #114608 to 26.7: Make the 04780 index-analysis allocation oracle the minimum of several runs
Backport #114608 to 26.6: Make the 04780 index-analysis allocation oracle the minimum of several runs
3 participants
Related: #104948
04780_json_subcolumn_index_match_not_quadraticcomparesMemoryAllocatedWithoutCheckBytesof a dotted-constantEXPLAIN indexes = 1against a no-dots control with a 150% threshold, measuring each arm with a single query. A single run is not a stable oracle: whichever query happens to be the first to touch a cache or spin up a thread pool absorbs a transient multi-megabyte allocation, and the dotted arm always runs first in the loop, so such a one-off lands on it and inflates the ratio arbitrarily. Locally the effect is easy to see: the first query after a server start reports 8–22 MB in this counter against a ~2.8 MB steady state for the identical query.The test failed this way on at least 6 unrelated PRs since 2026-08-12 (e.g.
Stateless tests (amd_asan_ubsan, distributed plan, parallel)on #104948 at commit 81300c2:longidx index analysis over a constant with 100000 dots allocated 4216780 bytes, more than 150% of the no-dots control (22224 bytes)— reruns passed; CIDB shows the same failure on #106011, #108522, #114475, #96130, #114476).Measure each arm three times and take the minimum: a genuine quadratic regression is deterministic and shows up in every run, so the oracle keeps discriminating (the minimum can only remove one-sided transient noise), while a one-off transient can no longer fail the test. Verified against the current master binary: the modified test passes repeatedly.
Changelog category (leave one):
Version info
26.8.1.1329(included in26.8and later)26.7.8.10,26.6.6.13