Add SQL Server, Spanner and DuckDB to goldeneye: generate their dialects and check the analyze cases against live databases - #4619
Merged
Conversation
Add two engines to goldeneye, each generating relations.jsonl from a live database and checking the engine's analyze cases against it. mssql reads a SQL Server named by MSSQL_SERVER_URI: relations.jsonl is every view of sys and INFORMATION_SCHEMA as the server describes a SELECT * from it, listed from a scratch user database. The analyze check describes each query without running it: result columns from sys.dm_exec_describe_first_result_set, parameter types from sp_describe_undeclared_parameters, and the column a parameter is compared with or assigned to from the estimated showplan. spanner reads a Spanner Omni server named by SPANNER_SERVER_URI, the gRPC endpoint of the spanner-omni container image, and writes into the googlesql dialect: relations.jsonl is every view of INFORMATION_SCHEMA and SPANNER_SYS. The analyze check creates a database per case and compiles each query in PLAN mode, taking columns and parameter types from the result metadata and provenance from the query plan. Both engines get a gen workflow job, a docker-compose service and a README section. Generating the dialects and running the checks turned up a few things in sqlc and its cases: - A rowversion or sysname column is NOT NULL unless declared nullable, which the SQL Server converter now does. - SQL Server rejects CAST to an alias type and a comparison of vector values, and Spanner requires a length on STRING, rejects NUMERIC(p,s), BIGNUMERIC, GEOGRAPHY, INTERVAL and STRUCT columns and cannot return a STRUCT as a column, so the analyze cases are rewritten to what each database accepts; the BigQuery-flavoured types case is kept as analyze_types/bigquery, which the Spanner check does not read. - The GoogleSQL dialect gains a proto type, which SPANNER_SYS has a column of. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F7cPsawATXMfiYqWg8nBVb
DuckDB 2.0 has no release to download yet, so `install duckdb` fetches the current build of DuckDB's v2.0 preview channel, a rolling tarball per platform with no per-build download and no checksum to pin, into the user cache directory, where Locate finds it after DUCKDB and before PATH. GeneratedFrom records the build the dialect was generated from, and the dialect is regenerated from the channel's current build, which adds a few functions and lists each type alias once per schema it is visible in, which the generator now folds. The duckdb engine gains an analysis check over the DuckDB analyze cases. The CLI reports a parameter's type through the unoptimized logical plan of the prepared query explained with a string sentinel bound to each parameter, and a result column's name and type through DESCRIBE with a typed NULL in each parameter's place; it prints every column by its bare name, so which table a result column is read from and which column a parameter stands in for come from the query text resolved against the catalog, and since it tracks no nullability of expressions, the query is run over the fixture and over no rows to see which columns come back NULL. A spelling the generated types.jsonl lists as an alias, such as json for varchar, is reported by the dialect's name for it. The gen workflow gets a duckdb job that installs and generates from the channel, and the README and CLAUDE.md describe the engine. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F7cPsawATXMfiYqWg8nBVb
SQL Server: a schema is split into statements outside BEGIN ... END and CASE ... END blocks, so a trigger or procedure body keeps its semicolons, and a GO on the first line is honoured; each ? becomes a parameter of its own rather than every ? the same one; the catalog's columns are matched to the described ones by name rather than position; Analyze refuses a server of another major release the way Generate does; the views are ordered by the bytes of their names rather than the server's collation; the plan walk's cycle guard is scoped to the path; and the generator's comment no longer claims a canonicalization it does not do. Spanner: Close no longer dereferences a nil client when opening fails half way; a DML statement is recognised past a leading comment; a DML plan's outputs are read as the THEN RETURN columns followed by the written values, which is the order Omni prints, and a returned expression is no longer taken for the column it is named after; an UPDATE's SET list is read with quotes and parentheses honoured up to the last top-level WHERE; a case's database is named with a hash of the case, so two cases with the same head stay apart; NotFound is checked by status code; a failed fixture write is rolled back; @@variables are not parameters; and PROTO and ENUM types are spelled one way. DuckDB: sentinel types are read from EXPLAIN (FORMAT json), whose expressions are strings, rather than from the box the CLI draws, and a sentinel the plan prints bare is a VARCHAR; a conversion error that names a sentinel in any form, or none, rebinds a parameter to NULL; the type after a :: or CAST stops at anything but a multi-word type's words; a CTE that shares a table's name is not the table; a parameter inside a subquery takes the binder's type rather than a partner from the outer scope; parameters are replaced token-wise, leaving string literals alone; the value a parameter is bound to is chosen by its type as spelled, so a JSON parameter gets a JSON value; and two enums with the same labels resolve to the same name every run. The MSSQL converter also treats a schema-qualified sys.sysname as NOT NULL by default. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F7cPsawATXMfiYqWg8nBVb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three more engines get goldeneye's two checks: their generated dialect files are verified byte for byte against a live database, and their
analyze_*cases are verified against what the database itself reports.mssql,MSSQL_SERVER_URI, themcr.microsoft.com/mssql/server:2025-latestimage).relations.jsonlis every view ofsysandINFORMATION_SCHEMA, each described bysys.dm_exec_describe_first_result_setfrom a scratch user database, sincemasterlists internal views no query can name. The analyze check describes each query without running it: result columns from the same DMV, parameter types fromsp_describe_undeclared_parameters, and the column a parameter is compared with or assigned to from the estimated XML showplan compiled with the parameters declared as those types. Spellingstypes.jsonllists as aliases (numeric,timestamp) are reported by the dialect's name (decimal,rowversion).spanner,SPANNER_SERVER_URI, theus-docker.pkg.dev/spanner-omni/images/spanner-omni:2026.r2.1-betaimage, writing into thegooglesqldialect).relations.jsonlis every view ofINFORMATION_SCHEMAandSPANNER_SYS. The analyze check creates a database per case with the schema as DDL, writes the fixture, compiles each query inPLANmode (DML in a rolled-back read-write transaction), and reads columns and parameter types from the result metadata and provenance from the query plan:Serialize Resultlists the outputs,Scannodes name table columns, comparisons show up as($col = @param), and a DML plan lists its THEN RETURN outputs before the written values.duckdb). 2.0 has no release yet, soinstall duckdbdownloads the current build of DuckDB's v2.0 preview channel (a rolling tarball; nothing can be pinned,duckdb.GeneratedFromrecords the build the dialect came from) into the user cache directory, whereLocatefinds it afterDUCKDBand beforePATH. The dialect is regenerated from that build, which adds 21 functions. The new analyze check drives the CLI: parameter types fromEXPLAIN (FORMAT json) EXECUTEwith a string sentinel bound to each parameter, result types fromDESCRIBEwith a typed NULL in each parameter's place, provenance from the query text resolved againstduckdb_columns()(DuckDB prints plans with bare column names and describes no DML), and expression nullability by running the query with a value of each parameter's type over the fixture and over no rows.Each engine gets a
gen.ymljob, adocker-compose.ymlservice where a server is needed, tests that skip without a database, and a README section describing what the database says and what it keeps to itself.What the checks found
rowversionorsysnamecolumn is NOT NULL unless declared nullable; the SQL Server converter now does that.CASTto an alias type and=onvectorvalues, and its describe function calls every computed column nullable.analyze_types/mssqlnow casts nullable columns and moves the parameter casts into the WHERE clause; its golden is regenerated.STRING(MAX)and rejectsNUMERIC(p,s),BIGNUMERIC,GEOGRAPHY,INTERVALandSTRUCTcolumns, and cannot return aSTRUCTas a column. The three simple GoogleSQL cases getSTRING(MAX); the BigQuery-flavoured types case is kept intact asanalyze_types/bigquery, which the Spanner check does not read, beside a Spanner-validanalyze_types/googlesql.SPANNER_SYShas aPROTOcolumn, so the GoogleSQLtypes.jsonlgains aprototype.The last commit fixes what an adversarial review of the three packages found (Spanner's DML output order and a nil
Close, DuckDB's conversion-error matching and cast-type reading, SQL Server's?numbering and block-aware statement splitting, among others).Testing
go test ./...ininternal/goldeneyeagainst a live SQL Server 2025, Spanner Omni 2026.r2.1 and DuckDB v2.0.0-alpha41396: the three dialects match, and all 17 analyze cases (8 MSSQL, 4 GoogleSQL, 5 DuckDB) match byte for byte.TestReplay/basefor every case in the main module passes.gen.ymljobs themselves were not exercised.Known limits
genjob goes red whenever the preview channel moves past the recorded build, which is the intended signal until 2.0 is released and can be pinned with a checksum.🤖 Generated with Claude Code
https://claude.ai/code/session_01F7cPsawATXMfiYqWg8nBVb
Generated by Claude Code