Skip to content

fix(format): keep MODEL/AUDIT/METRIC header dialect-agnostic (#5773) - #5864

Merged
StuffbyYuki merged 1 commit into
SQLMesh:mainfrom
anxkhn:loop/sqlmesh__001
Jun 30, 2026
Merged

fix(format): keep MODEL/AUDIT/METRIC header dialect-agnostic (#5773)#5864
StuffbyYuki merged 1 commit into
SQLMesh:mainfrom
anxkhn:loop/sqlmesh__001

Conversation

@anxkhn

@anxkhn anxkhn commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Description

sqlmesh format corrupts boolean properties in the MODEL (...) header when the model dialect has no native TRUE keyword. On a T-SQL model, a property such as allow_partials true is rewritten to allow_partials (1 = 1), and SQLMesh can no longer re-parse that as a boolean, so the model file breaks. Fixes #5773.

Root cause: format_model_expressions (sqlmesh/core/dialect.py) rendered every parsed expression with the target dialect, including the SQLMesh-specific MODEL/AUDIT/METRIC header. That header is SQLMesh DDL, not standard SQL, so SQLGlot's tsql generator transpiles a boolean to (1 = 1).

This renders meta expressions (is_meta_expression -> Audit/Metric/Model) with dialect=None on both the single-meta early return and the multi-expression join path, so the header stays exactly as written, while the actual query/statement body still transpiles to the target dialect.

Test Plan

  • Added a tsql case to tests/core/test_dialect.py::test_format_model_expressions asserting the MODEL header keeps allow_partials TRUE (not (1 = 1)) while the query body still transpiles (SELECT TRUE -> 1, CAST(x AS INT) -> x::INTEGER).
  • Confirmed the new assertion fails on main and passes with this change.
  • pytest tests/core/test_dialect.py tests/core/test_format.py green; make style (ruff + mypy) clean on both files.

AUDIT/METRIC headers share the identical is_meta_expression code path and behave the same; happy to add an AUDIT assertion too if that would be useful.

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO

@StuffbyYuki
StuffbyYuki self-requested a review June 29, 2026 19:36

@StuffbyYuki StuffbyYuki left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anxkhn Thanks for this PR!

Optional: AUDIT/false boolean test cases; text_diff may deserve the same treatment in a follow-up.

format_model_expressions rendered every parsed expression with the target
dialect, including the SQLMesh-specific MODEL/AUDIT/METRIC header. That header
is SQLMesh DDL, not standard SQL, so transpiling it corrupts boolean properties:
under tsql (which has no TRUE keyword) a property like `allow_partials true`
was rewritten to `allow_partials (1 = 1)`.

Render meta expressions (is_meta_expression) with dialect=None so the header
stays as written, while the actual query/statement body still transpiles to the
target dialect.

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
@anxkhn
anxkhn force-pushed the loop/sqlmesh__001 branch from f6078cd to 5fc0cf8 Compare June 30, 2026 04:21
@StuffbyYuki
StuffbyYuki merged commit c3ec643 into SQLMesh:main Jun 30, 2026
32 checks passed
EhabEasee pushed a commit to EhabEasee/sqlmesh that referenced this pull request Jun 30, 2026
…#5773) (SQLMesh#5864)

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
Signed-off-by: EhabEasee <ehab.elbadrawi@easee.com>
fresioAS pushed a commit to fresioAS/sqlmesh that referenced this pull request Jul 13, 2026
…#5773) (SQLMesh#5864)

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
Signed-off-by: fresioAS <andreas.fredhoi@three60energy.com>
mday-io pushed a commit to mday-io/sqlmesh that referenced this pull request Aug 9, 2026
Cherry-picked from upstream sqlmesh/sqlmesh commit c3ec643
(fix(format): keep MODEL/AUDIT/METRIC header dialect-agnostic (SQLMesh#5773) (SQLMesh#5864)),
which this fork's main had not yet synced. Needed as a prerequisite for the
follow-up fixes to header property rendering.

Meta expressions (MODEL/AUDIT/METRIC) are SQLMesh DDL, not standard SQL, so
they must never be transpiled to the target dialect -- e.g. tsql would rewrite
a boolean property like `allow_partials TRUE` to `(1 = 1)`.

Signed-off-by: mday-io <mdaytn@gmail.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016cYEi6jnceseL6gRSPM5i9
mday-io added a commit to mday-io/sqlmesh that referenced this pull request Aug 9, 2026
…alect

SQLMesh#5864 stopped transpiling MODEL/AUDIT/METRIC headers so that SQLMesh's own
boolean properties would survive formatting -- on tsql, `allow_partials TRUE`
was being rewritten to `(1 = 1)`, which then fails to parse at all and leaves
the model file broken.

That fix rendered the entire header generically, including the properties whose
values are the user's warehouse SQL. Those lose their dialect: `columns
(ts DATETIME2(6))` becomes `TIMESTAMP(6)`, and an audit argument such as
`CAST('2024-01-01' AS DATETIME2)` is silently downgraded the same way.

Split the header per property instead of per expression. The split is derived
from the field declarations themselves: expression-typed fields (columns,
audits, signals, partitioned_by, physical_properties, ...) hold warehouse SQL
and render with the model dialect, while scalar-typed fields (allow_partials,
description, kind, ...) are SQLMesh's own semantics and stay dialect-agnostic.
Deriving it means the policy stays correct as properties are added, and a field
that is missed fails safe -- a keyword is not canonicalized, rather than a
user's SQL being corrupted.

Covers MODEL, AUDIT and METRIC headers, and the expression properties nested
inside `kind` such as `time_data_type` and `unique_key`.

Signed-off-by: mday-io <mdaytn@gmail.com>
mday-io pushed a commit to mday-io/sqlmesh that referenced this pull request Aug 9, 2026
Cherry-picked from upstream sqlmesh/sqlmesh commit c3ec643
(fix(format): keep MODEL/AUDIT/METRIC header dialect-agnostic (SQLMesh#5773) (SQLMesh#5864)),
which this fork's main had not yet synced. Needed as a prerequisite for the
follow-up fixes to header property rendering.

Meta expressions (MODEL/AUDIT/METRIC) are SQLMesh DDL, not standard SQL, so
they must never be transpiled to the target dialect -- e.g. tsql would rewrite
a boolean property like `allow_partials TRUE` to `(1 = 1)`.

Signed-off-by: mday-io <mdaytn@gmail.com>
mday-io added a commit to mday-io/sqlmesh that referenced this pull request Aug 9, 2026
…alect

SQLMesh#5864 stopped transpiling MODEL/AUDIT/METRIC headers so that SQLMesh's own
boolean properties would survive formatting -- on tsql, `allow_partials TRUE`
was being rewritten to `(1 = 1)`, which then fails to parse at all and leaves
the model file broken.

That fix rendered the entire header generically, including the properties whose
values are the user's warehouse SQL. Those lose their dialect: `columns
(ts DATETIME2(6))` becomes `TIMESTAMP(6)`, and an audit argument such as
`CAST('2024-01-01' AS DATETIME2)` is silently downgraded the same way.

Split the header per property instead of per expression. The split is derived
from the field declarations themselves: expression-typed fields (columns,
audits, signals, partitioned_by, physical_properties, ...) hold warehouse SQL
and render with the model dialect, while scalar-typed fields (allow_partials,
description, kind, ...) are SQLMesh's own semantics and stay dialect-agnostic.
Deriving it means the policy stays correct as properties are added, and a field
that is missed fails safe -- a keyword is not canonicalized, rather than a
user's SQL being corrupted.

Covers MODEL, AUDIT and METRIC headers, and the expression properties nested
inside `kind` such as `time_data_type` and `unique_key`.

Signed-off-by: mday-io <mdaytn@gmail.com>
mday-io added a commit to mday-io/sqlmesh that referenced this pull request Aug 9, 2026
…alect

SQLMesh#5864 stopped transpiling MODEL/AUDIT/METRIC headers so that SQLMesh's own
boolean properties would survive formatting -- on tsql, `allow_partials TRUE`
was being rewritten to `(1 = 1)`, which then fails to parse at all and leaves
the model file broken.

That fix rendered the entire header generically, including the properties whose
values are the user's warehouse SQL. Those lose their dialect: `columns
(ts DATETIME2(6))` becomes `TIMESTAMP(6)`, and an audit argument such as
`CAST('2024-01-01' AS DATETIME2)` is silently downgraded the same way.

Split the header per property instead of per expression. The split is derived
from the field declarations themselves: expression-typed fields (columns,
audits, signals, partitioned_by, physical_properties, ...) hold warehouse SQL
and render with the model dialect, while scalar-typed fields (allow_partials,
description, kind, ...) are SQLMesh's own semantics and stay dialect-agnostic.
Deriving it means the policy stays correct as properties are added, and a field
that is missed fails safe -- a keyword is not canonicalized, rather than a
user's SQL being corrupted.

Covers MODEL, AUDIT and METRIC headers, and the expression properties nested
inside `kind` such as `time_data_type` and `unique_key`.

Signed-off-by: mday-io <mdaytn@gmail.com>
mday-io added a commit to mday-io/sqlmesh that referenced this pull request Aug 9, 2026
…alect

SQLMesh#5864 stopped transpiling MODEL/AUDIT/METRIC headers so that SQLMesh's own
boolean properties would survive formatting -- on tsql, `allow_partials TRUE`
was being rewritten to `(1 = 1)`, which then fails to parse at all and leaves
the model file broken.

That fix rendered the entire header generically, including the properties whose
values are the user's warehouse SQL. Those lose their dialect: `columns
(ts DATETIME2(6))` becomes `TIMESTAMP(6)`, and an audit argument such as
`CAST('2024-01-01' AS DATETIME2)` is silently downgraded the same way.

Split the header per property instead of per expression. The split is derived
from the field declarations themselves: expression-typed fields (columns,
audits, signals, partitioned_by, physical_properties, ...) hold warehouse SQL
and render with the model dialect, while scalar-typed fields (allow_partials,
description, kind, ...) are SQLMesh's own semantics and stay dialect-agnostic.
Deriving it means the policy stays correct as properties are added, and a field
that is missed fails safe -- a keyword is not canonicalized, rather than a
user's SQL being corrupted.

Covers MODEL, AUDIT and METRIC headers, and the expression properties nested
inside `kind` such as `time_data_type` and `unique_key`.

Signed-off-by: mday-io <mdaytn@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TSQL dialect formatting corrupts boolean true in MODEL header to (1=1)

2 participants