Repository navigation
Commit e0c1933
feat(optimizer): annotate types for CovarPop/Samp, PercentileCont/Disc, and WithinGroup [CLAUDE] (tobymao#7656)
* feat(annotate_types): annotate CovarPop/Samp, PercentileCont/Disc, and WithinGroup
Registers constant DOUBLE return types for COVAR_POP, COVAR_SAMP, and
PERCENTILE_CONT. PERCENTILE_DISC is also registered as DOUBLE by default,
but the new _annotate_within_group method overrides this when wrapped in
a WITHIN GROUP clause: for PERCENTILE_DISC it reads the type from the
ORDER BY sort expression (the engine picks an existing row, so the result
type matches the sort column). PERCENTILE_CONT always interpolates and
therefore always returns DOUBLE regardless of sort column type. [CLAUDE]
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(annotate_types): extract _type_from_order helper from _annotate_within_group [CLAUDE]
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(annotate_types): fix return type of _annotate_within_group and add int-input covariance tests
_set_type returns exp.Expr, not the subtype — separate the call from the
return to satisfy mypy. Also add COVAR_POP/COVAR_SAMP tests with integer
inputs to assert DOUBLE is returned regardless of input type.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(annotate_types): PercentileDisc returns sort-expr type, not always DOUBLE
Remove exp.PercentileDisc from the unconditional DOUBLE block — it picks an
actual row value so its return type matches the sort expression. Add a DuckDB
typing override to handle the bare QUANTILE_DISC(col, fraction) form, where
`this` is the data column. Add tests covering int and bigint sort columns.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(annotate_types): WithinGroup(PercentileDisc) falls back to UNKNOWN when sort type unavailable
Previously fell through to _annotate_by_args(e, "this") which would propagate
UNKNOWN from the PercentileDisc node itself — explicit is better.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(annotate_types): remove spurious UNKNOWN filter in _type_from_order
The annotator runs in a single pass so there is no later opportunity to
resolve an UNKNOWN sort type. Filtering it out caused _annotate_within_group
to set UNKNOWN anyway via the None fallback — with no benefit. Let UNKNOWN
propagate naturally; None now means only that no order expression exists.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style(annotate_types): remove redundant comments from _annotate_within_group
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(annotate_types): WithinGroup always returns DOUBLE in Hive/Spark/Databricks
Empirically verified against Databricks and confirmed via Spark docs examples:
PERCENTILE_DISC returns DOUBLE even for integer inputs in these dialects, unlike
Snowflake/BigQuery which return the sort expression's type. Override in hive.py
propagates to Spark2, Spark, and Databricks via inheritance.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(annotate_types): inline _type_from_order into _annotate_within_group
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(annotate_functions): add DuckDB PERCENTILE_DISC cases
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* style(annotate_types): ruff format
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(annotate_types): Hive/Spark WithinGroup annotates by `this` arg [CLAUDE]
The Hive override hardcoded WithinGroup as DOUBLE, which silently mistyped
non-percentile ordered-set aggregates (e.g. LISTAGG, MODE) that propagate
their inner expression's type. Switch to annotate-by-`this` and register
PercentileDisc as DOUBLE so the existing percentile cases still return
DOUBLE in Spark/Hive/Databricks regardless of the sort-expr type.
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>1 parent 6084d34 commit e0c1933
5 files changed
Lines changed: 81 additions & 0 deletions
File tree
- sqlglot
- optimizer
- typing
- tests/fixtures/optimizer
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1029 | 1029 | | |
1030 | 1030 | | |
1031 | 1031 | | |
| 1032 | + | |
| 1033 | + | |
| 1034 | + | |
| 1035 | + | |
| 1036 | + | |
| 1037 | + | |
| 1038 | + | |
| 1039 | + | |
| 1040 | + | |
| 1041 | + | |
1032 | 1042 | | |
1033 | 1043 | | |
1034 | 1044 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
98 | 98 | | |
99 | 99 | | |
100 | 100 | | |
| 101 | + | |
| 102 | + | |
101 | 103 | | |
102 | 104 | | |
103 | 105 | | |
| |||
114 | 116 | | |
115 | 117 | | |
116 | 118 | | |
| 119 | + | |
117 | 120 | | |
118 | 121 | | |
119 | 122 | | |
| |||
369 | 372 | | |
370 | 373 | | |
371 | 374 | | |
| 375 | + | |
372 | 376 | | |
373 | 377 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
49 | 49 | | |
50 | 50 | | |
51 | 51 | | |
| 52 | + | |
52 | 53 | | |
53 | 54 | | |
54 | 55 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
52 | 52 | | |
53 | 53 | | |
54 | 54 | | |
| 55 | + | |
55 | 56 | | |
56 | 57 | | |
57 | 58 | | |
| |||
63 | 64 | | |
64 | 65 | | |
65 | 66 | | |
| 67 | + | |
66 | 68 | | |
67 | 69 | | |
68 | 70 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6470 | 6470 | | |
6471 | 6471 | | |
6472 | 6472 | | |
| 6473 | + | |
| 6474 | + | |
| 6475 | + | |
| 6476 | + | |
| 6477 | + | |
| 6478 | + | |
| 6479 | + | |
| 6480 | + | |
| 6481 | + | |
| 6482 | + | |
| 6483 | + | |
| 6484 | + | |
| 6485 | + | |
| 6486 | + | |
| 6487 | + | |
| 6488 | + | |
| 6489 | + | |
| 6490 | + | |
| 6491 | + | |
| 6492 | + | |
| 6493 | + | |
| 6494 | + | |
| 6495 | + | |
| 6496 | + | |
| 6497 | + | |
| 6498 | + | |
| 6499 | + | |
| 6500 | + | |
| 6501 | + | |
| 6502 | + | |
| 6503 | + | |
| 6504 | + | |
| 6505 | + | |
| 6506 | + | |
| 6507 | + | |
| 6508 | + | |
| 6509 | + | |
| 6510 | + | |
| 6511 | + | |
| 6512 | + | |
| 6513 | + | |
| 6514 | + | |
| 6515 | + | |
| 6516 | + | |
| 6517 | + | |
| 6518 | + | |
| 6519 | + | |
| 6520 | + | |
| 6521 | + | |
| 6522 | + | |
| 6523 | + | |
| 6524 | + | |
| 6525 | + | |
| 6526 | + | |
| 6527 | + | |
| 6528 | + | |
| 6529 | + | |
| 6530 | + | |
| 6531 | + | |
| 6532 | + | |
| 6533 | + | |
| 6534 | + | |
| 6535 | + | |
| 6536 | + | |
0 commit comments