Skip to content

Add validate_group_by_all_key_types to gate the GROUP BY ALL key type validation - #119868

Open
groeneai wants to merge 1 commit into
ClickHouse:masterfrom
groeneai:groeneai/gate-group-by-all-key-type-validation
Open

Add validate_group_by_all_key_types to gate the GROUP BY ALL key type validation#119868
groeneai wants to merge 1 commit into
ClickHouse:masterfrom
groeneai:groeneai/gate-group-by-all-key-type-validation

Conversation

@groeneai

@groeneai groeneai commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Closes: #119852
Related: #110206

Changelog category (leave one):

  • Backward Incompatible Change

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

New setting validate_group_by_all_key_types (default true) gates the key type validation that GROUP BY ALL applies to the grouping keys it expands the SELECT expressions into. Since 26.7 it rejects a Variant or Dynamic grouping key, for example an untyped JSON subpath, that earlier versions accepted with the analyzer enabled (the default), with no way to keep such a query running across an upgrade. Setting compatibility to a version before 26.7, or validate_group_by_all_key_types = 0, restores the earlier behavior. An explicit GROUP BY is unaffected and still rejects such a key, as it did before 26.7. Closes #119852.

Description

GROUP BY ALL adds the SELECT expressions as grouping keys in expandGroupByAll, after resolveGroupByNode has validated an explicit GROUP BY, so before #110206 the expanded keys were never type-checked. #110206 re-ran both the tuple expansion (fixing #83433) and validateGroupByKeyType on them. That tightening is intended and stays the default; what shipped without an escape hatch is the acceptance change, so compatibility could not keep an upgraded workload running. Requested by @ fm4v in #119852, who measured it on 3 of 20 sampled chReplay instances, with an identical result hash: the old acceptance computed correct answers.

Only the validation is gated, keyed on the query having been written GROUP BY ALL. The tuple expansion stays unconditional and the shared validateGroupByKeyType helper is untouched, so explicit GROUP BY, GROUPING SETS, window PARTITION BY and the old analyzer keep rejecting such keys as before. The gate covers both places these keys are validated: the late was_group_by_all block, and resolveGroupByNode, which validates them instead when group_by_use_nulls meets WITH ROLLUP/CUBE (expanded early there, flag already cleared).

Two measured corrections to the issue's suggested shape:

  1. The rejection first shipped in 26.7, not 26.8, so the SettingsChangesHistory entry goes in the 26.7 block. The requested compatibility = '26.7' arm is in the test as a negative arm; '26.6' is the restore arm. Moving it to 26.8 is a one-line change if you prefer.
  2. A second entry in the 26.9 block is required, or Style check fails.

Default true reproduces current behavior exactly, so no existing query changes. Distributed is deliberately out of scope: a shard receives the expanded keys as an explicit GROUP BY, which 26.6 rejected too, so there is nothing to restore there. I can ask for a backport label after merge.

Evidence for the two corrections, and the validation

1. #110206 (9d67075) is in 26.7. gh api repos/ClickHouse/ClickHouse/compare/<tag>...9d67075 --jq .status:

probe status reading
v26.6.5.120-stable...9d67075 diverged not in the 26.6 line
v26.7.1.1315-stable...9d67075 behind in the first 26.7 stable release
v26.8.1.2041-lts...9d67075 behind also in 26.8, as expected

Its test file is present at ref=v26.7.1.1315-stable and absent at ref=v26.6.5.120-stable. It merged 2026-07-16, inside the 26.7 window (26.7 opened 2026-06-23, 26.8 opened 2026-07-22). The issue compared 26.6.1.2103 against 26.8.1.189, so 26.7 was never tested. Recording it under 26.8 would tell compatibility that 26.7 accepted such a key, which it did not.

2. The 26.9 entry. check_style.py::check_settings_changes_history demands every added setting under the current version block whenever another src/ file changes, so a 26.7-only entry fails Style check. 149 settings already sit in more than one block. No 26.8 entry: nothing changed in 26.8.

Validation. The issue's reproducer throws ILLEGAL_COLUMN on the parent commit and returns its 5 rows with the setting off or compatibility = '26.6'; compatibility resolves the setting to 0 for 26.4/26.6 and 1 for 26.7/26.8/26.9. Verified for Variant and for Dynamic nested in Tuple/Array/Map, with the explicit-GROUP BY form still rejected in every case. 04602_group_by_all_suspicious_types gains 10 arms, including the reported JSON-subpath shape; four were confirmed load-bearing by breaking the fix four ways (gating the tuple expansion, gating the shared helper, gating only the late block, moving the history entry to 26.8) and checking the predicted arm went red each time. 100/100 green over 50 randomized runs of 04602 plus 04518_group_by_all_tuple_order_by.


Workflow [PR]
Sync PR [sync-upstream/pr/119868]

…type validation

`GROUP BY ALL` adds the `SELECT` expressions as grouping keys in `expandGroupByAll`, which
runs after `resolveGroupByNode` has already done its tuple unwrapping and key type
validation for an explicit `GROUP BY`. The keys `GROUP BY ALL` expands into therefore went
through neither, until ClickHouse#110206 re-ran both on them: `expandTuplesInList`, which fixes
ClickHouse#83433, and a `validateGroupByKeyType` loop. That loop is an acceptance rule, and it
changed behavior: a `Variant`/`Dynamic` grouping key written as `GROUP BY ALL`, typically an
untyped JSON subpath, is rejected with `ILLEGAL_COLUMN` from 26.7 onward where earlier
versions accepted it.

The tightening is intended, so this keeps it as the default and only makes it reachable by
`compatibility`. Reported in ClickHouse#119852 with a measured upgrade break (3 of 20 sampled
chReplay instances) and an identical result hash once the key was permitted, i.e. the old
acceptance computed correct answers rather than wrong ones; without a gate an upgrade
breaks those queries with no way to keep them running short of rewriting them or enabling
`allow_suspicious_types_in_group_by`, which also loosens the explicit `GROUP BY` and
`ORDER BY` checks.

The setting is read at the two call sites rather than inside `DB::validateGroupByKeyType`:
that helper is shared with the explicit `GROUP BY`, `GROUP BY GROUPING SETS`, the window
`PARTITION BY` and both old-analyzer sites, none of which changed behavior, so gating the
helper would newly loosen all of them.

Two call sites, not one, because `GROUP BY ALL` reaches the validation two ways. Normally
the late `was_group_by_all` block validates. But when `scope.group_by_use_nulls` is set,
which needs `group_by_use_nulls` plus a `WITH ROLLUP`/`CUBE`/`GROUPING SETS` modifier,
`GROUP BY ALL` is expanded early and `expandGroupByAll` clears the flag, so that block is
skipped and `resolveGroupByNode` validates the expanded keys instead. Hence the ALL-ness is
captured once before any expansion and threaded in as a policy; an explicit `GROUP BY`
passes `true` and is bit-for-bit unchanged. The `nullable_group_by_keys` collection sharing
those loops stays outside the guard: it implements `group_by_use_nulls` promotion, which the
setting must not affect, and moving it inside would reintroduce the bug fixed in
b3edb0b.

The history entry goes in the 26.7 block, not 26.8. ClickHouse#110206 (9d67075) merged 2026-07-16,
inside the 26.7 development window, and `gh api .../compare/v26.7.1.1315-stable...9d67075`
reports `behind`, so the rejection first shipped in 26.7; recording it under 26.8 would tell
`compatibility` that 26.7 accepted such a key, which it did not. The second entry in the
current 26.9 block is required by
`ci/jobs/check_style.py::check_settings_changes_history`, which demands every added setting
under the current version block whenever another `src/` file changes.

Closes: ClickHouse#119852

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@groeneai groeneai added can be tested Allows running workflows for external contributors groeneai-origin-request PR origin: a maintainer pinged or directed groeneai labels Sep 14, 2026
@groeneai

Copy link
Copy Markdown
Collaborator Author
Internal second-model review: adjudication log (click to expand)

Pre-publication review by an independent model (engine: codex), plus my own cold review of the
resulting tree. Gate A ran 3 rounds during planning and finished with 0 findings. Gate B ran twice on
the implemented tree: 1 finding, then 0 findings after the fix round that answered it.

# Sev Finding Verdict Evidence / action
1 ⚠️ validate_group_by_all_key_types = 0 is documented as disabling the GROUP BY ALL key type validation, but it has no effect with enable_analyzer = 0 (src/Core/Settings.cpp) AGREE, fixed The gated path is analyzer-only by design: at v26.6.5.120-stable the old analyzer already expanded GROUP BY ALL in TreeRewriter and validated the resulting keys in ExpressionAnalyzer, and #110206 changed QueryAnalyzer.cpp and two tests only, so the old analyzer rejected such a key before 26.7 as well and compatibility has nothing to restore there. Gating it would be a new loosening. The prose was the defect: the setting description, the 26.7 history reason and the changelog entry now all scope the claim to enable_analyzer = 1, matching the form used by the sibling analyzer-compatibility settings.
2 ⚠️ The issue and PR links sat inside the template's HTML comment, so the rendered body carried no visible provenance link and the changelog entry carried no Closes AGREE, fixed Measured over the last 300 pull requests in this repository: none references an issue only from inside an HTML comment. The links are visible lines now.
3 💡 A comment added next to the gated loop restates the nine-line comment directly above it, which already explains why the tuple expansion is required and cites #83433 AGREE, fixed Reduced to one line stating only what is new at that spot.
4 💡 No test arm reproduced the shape actually reported, a GROUP BY ALL over an untyped JSON subpath; every arm used 1::Dynamic AGREE, fixed The subpath's result type is Dynamic, so the code path was already covered, but the reported form is now an arm of 04602_group_by_all_suspicious_types too, with its explicit-GROUP BY counterpart as the negative control.
5 💡 The commit message describes the rejection as one "earlier versions accepted" without the analyzer qualification added everywhere else AGREE, noted, not fixed Developer-facing rather than published documentation, the message is explicitly about analyzer machinery throughout, and its third paragraph already records that the two old-analyzer sites did not change behavior. Recorded rather than amended.

Two documented deviations from the internal implementation plan were re-derived and accepted on the
merits: the WITH ROLLUP arm dropped a positional ORDER BY 1, because argument 1 resolves to the
Dynamic key and trips the unrelated allow_suspicious_types_in_order_by check (row order measured
deterministic over 15 consecutive runs at max_threads 1/4/16, and 100/100 over 50 randomized runs);
and one planned verification probe was replaced because it failed with NOT_IMPLEMENTED whether the
new gate was on or off, so it could not have discriminated a correct guard from a broken one.

Severity: ❌ blocker / ⚠️ major / 💡 nit. DISAGREE verdicts carry recorded evidence and are
terminal per finding. Findings on hunks unchanged by a fix round are auto-dropped.

Session id: cron:clickhouse-review-slot-8:20260913-234400

@clickhouse-gh clickhouse-gh Bot closed this Sep 14, 2026
@clickhouse-gh clickhouse-gh Bot reopened this Sep 14, 2026
@groeneai

Copy link
Copy Markdown
Collaborator Author
Pre-PR validation gate (click to expand)
# Question Answer
a Deterministic repro? Yes, 100%, not a rate. The issue's reproducer on the parent commit throws Code: 44 ILLEGAL_COLUMN. The second, less obvious path reproduces just as deterministically: SELECT d FROM (SELECT 1::Dynamic AS d) GROUP BY ALL WITH ROLLUP SETTINGS group_by_use_nulls = 1.
b Root cause explained? Yes. GROUP BY ALL adds the SELECT expressions as grouping keys in expandGroupByAll, which runs after resolveGroupByNode has validated an explicit GROUP BY, so those expanded keys were historically never type-checked. #110206 re-ran expandTuplesInList (fixing #83433) and validateGroupByKeyType on them; the latter sees isDynamic(type) with allow_suspicious_types_in_group_by = false and throws. The behavior change is the added acceptance rule, not a miscomputation.
c Fix matches root cause? Yes. It gates exactly that validation, at its call sites, keyed on the query having been written GROUP BY ALL. No widened bound, no data reduction, no no-random-* tag, no defensive check. The tightening itself is preserved and remains the default.
d Test intent preserved / new tests added? Yes. The six existing 04602_group_by_all_suspicious_types statements and its SET enable_analyzer = 1 line are untouched, and 04518_group_by_all_tuple_order_by (the #83433 guard) is untouched and green. 10 arms added, four of which were proven load-bearing by mutation (see e); the last two reproduce the reported JSON-subpath shape and were added in review fix round 1.
e Both directions demonstrated? Yes. Default: still Code: 44. validate_group_by_all_key_types = 0 or compatibility = '26.6': the reproducer's 5 rows. compatibility = '26.7'/'26.8': still Code: 44. Four deliberate mutations each reddened exactly the arm predicted for it (gating the tuple expansion → the expansion arm; gating the shared helper → both explicit-GROUP BY arms; gating only the late block → the group_by_use_nulls arm; moving the history entry to 26.8 → the compatibility = '26.7' arm), so no arm is vacuous. Build IDs differ across every measurement.
f Fix is general across code paths? Yes. Both paths that validate these keys are gated: the late was_group_by_all block, and resolveGroupByNode, which validates them instead when group_by_use_nulls meets WITH ROLLUP/CUBE (there GROUP BY ALL is expanded early and the flag is already cleared). Every other validateGroupByKeyType site is deliberately left unconditional because none of them changed behavior: explicit GROUP BY, GROUPING SETS, window PARTITION BY, and both old-analyzer sites. The gate is at the call site, so the shared helper is untouched, and mutation (e) shows the two explicit-GROUP BY arms are what enforce that boundary.
g Fix generalizes across inputs (params/datatypes/wrappers)? Yes, verified rather than assumed. With the gate off, GROUP BY ALL accepts Dynamic, Variant(UInt8,String), Tuple(Dynamic, UInt8), Array(Dynamic), Map(String, Dynamic) and the JSON-subpath shape, while the explicit-GROUP BY form of each stays rejected; with the gate on, all are rejected; non-suspicious keys are unaffected either way. Nullable(Variant(...)) is unreachable, the type system refuses it first (Code: 43). The reported JSON-subpath shape now has its own test arm plus its explicit-GROUP BY counterpart (toTypeName(c1.p1) measured as Dynamic); no per-type arm was added for the remaining wrappers, because it is one code path and the helper checks isDynamic || isVariant plus forEachChild, so the gate cannot be type-selective.
h Backward compatible? Yes. Default true reproduces current behavior exactly, so no existing query changes. SettingsChangesHistory.cpp records the setting in the 26.7 block with previous value false, so compatibility below 26.7 restores the earlier acceptance, plus the mandatory current-version (26.9) entry. Measured: compatibility resolves the setting to 0 for 26.4/26.6 and 1 for 26.7/26.8/26.9. No server setting, no format change, no on-disk change. This PR is the compatibility gate for an incompatibility that already shipped; it does not introduce one.
i Invariants and contracts preserved? Yes. expandTuplesInList stays unconditional on both paths, so #83433 cannot reopen (mutation (a) is the proof that the test catches it). The nullable_group_by_keys collection stays outside the new guard in both loops, so group_by_use_nulls promotion is unaffected when the gate is off: the rollup total's key is still \N, identical to the gate-on control, rather than the type default (that placement is #110915's bug, and it is the easiest mistake in this change). The new parameter is true on every explicit-GROUP BY path, so that behavior is bit-for-bit unchanged. No error or early-return path is added.

Session id: cron:clickhouse-impl-slot-6:20260913-233005

@clickhouse-gh

clickhouse-gh Bot commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Workflow [PR], commit [98b8363]

Summary:

job_name test_name status info comment
Stress test (amd_tsan) FAIL
Cannot start clickhouse-server FAIL cidb
Check failed FAIL cidb

AI Review

Summary

This PR adds validate_group_by_all_key_types so GROUP BY ALL keeps the post-26.7 Variant/Dynamic key validation by default while letting compatibility or an explicit setting restore the pre-26.7 acceptance. I traced both analyzer validation sites, the group_by_use_nulls early-expansion path, the compatibility-history application order, and the query-tree-to-remote-query path, and I did not find an actionable correctness or compatibility bug in the current code or the added stateless coverage.

Missing context / blind spots
  • ⚠️ I did not execute the new queries locally because this checkout does not include a usable build artifact; this review is based on source tracing and the committed tests.
Final Verdict

No actionable findings.

LLVM Coverage Report

⚠️ No coverage measurement for commit 98b8363: incomplete coverage measurement: 1 of 21 shard profiles are missing: LLVM_COVERAGE_FILE_it_7.profdata.

@clickhouse-gh clickhouse-gh Bot added the pr-backward-incompatible Pull request with backwards incompatible changes label Sep 14, 2026
@clickhouse-gh clickhouse-gh Bot added the comp-query-analyzer Semantic analysis + logical/physical planning + rewrites (Analyzer and Planner modules). label Sep 14, 2026
@clickhouse-gh

clickhouse-gh Bot commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Build profile diff (arm_release)

Comparing 98b83638a with master a5efd3a38 (stripped binary size, per-symbol sizes and ThinLTO time; compile times per translation unit against the most recent warmup build that recompiled it).

✅ No significant changes.

Binary sizes

programs/clickhouse-stripped: smaller than the master baseline by the known offset between the two builds, so the difference is not shown. A delta that differs from the offset by more than 50% of it is shown, in either direction.

The official master build is compiled with -g and a pull request build is not, and XRay counts debug instructions towards its instrumentation threshold, so master instruments thousands of functions more and its binary is ~0.4% larger no matter what the pull request does.

Compile time of recompiled translation units

22 translation units recompiled, 178 s compile time in total, 22 of them have a recent master baseline.

Median compile-time ratio to the baselines is ×1.08 (machine-speed difference or a change affecting every TU); per-TU deltas below are relative to that ratio.
The matched translation units cost +12.5 s (+8%) in total before that adjustment.

Job report

@groeneai

Copy link
Copy Markdown
Collaborator Author

CI finish ledger - 98b8363

CI is complete on this head (344 check runs, aggregator finished 04:24:38Z). Every failure below has an owner: a fixing PR (mine or external), or a full-effort fix task whose fixing-PR link will be posted here when it opens. CH Inc sync is the only exemptible check and it passed, so nothing here is exempt.

Check / test Reason Owner / fixing PR
Stress test (amd_tsan) / Cannot start clickhouse-server startup readiness timeout: the server process was alive but did not answer SELECT 1 within the 120s readiness budget, so the stress workload never ran. The paired Check failed row is the job level companion of the same event, not a second failure. Not reachable from this diff, which adds a setting, its history entries and test arms. a fix task is created (investigating at full effort, fixing PR link to follow on this PR)

Mergeable Check and PR are aggregate statuses and reflect that single failure. The 171 cancelled check runs on this head belong to the dispatch that was superseded when the can be tested label re-triggered the workflow; cancelled is not a verdict.

Session id: cron:our-pr-ci-monitor:20260914-043100

@groeneai

Copy link
Copy Markdown
Collaborator Author

Fixing PR for the Stress test (amd_tsan) start_server readiness timeout: #119891

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

can be tested Allows running workflows for external contributors comp-query-analyzer Semantic analysis + logical/physical planning + rewrites (Analyzer and Planner modules). groeneai-origin-request PR origin: a maintainer pinged or directed groeneai pr-backward-incompatible Pull request with backwards incompatible changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GROUP BY ALL over a JSON subpath rejected with ILLEGAL_COLUMN since 26.8 without a compatibility gate

1 participant