Skip to content

Fix bugs found by the pre-release sweep, update docs - #46

Merged
doron050 merged 3 commits into
mainfrom
devin/1793712974-sweep-fixes
Nov 3, 2026
Merged

doron050 merged 3 commits into
mainfrom
devin/1793712974-sweep-fixes

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes the bugs found by a pre-release sweep of main (real PostgreSQL 16 / MySQL 8.3 / SQLite apps on Ruby 3.1–3.4), and brings the docs up to date.

Fixes

  • pg :batchexec / :batchone / :batchmany now encode parameters like every other command (Driver#encode_param), so bytea, json and arrays work. Before: string contains null byte / type errors.
  • pg transactions: a block that rescues a failed statement and returns normally used to "commit" (the server answered ROLLBACK, silently). It now raises PG::InFailedSqlTransaction.
  • mysql2 :copyfrom: binary-safe (CHARACTER SET binary), all-or-nothing (transaction, or savepoint inside one), and LOAD DATA LOCAL warnings (skipped duplicate keys, coerced invalid values) raise Mysql2::Error.
  • sqlite3 :copyfrom runs in a savepoint: a failing row no longer leaves earlier batches committed.
  • sqlite3 transactions: threads / fibers sharing one SQLite3::Database take turns (one reentrant lock per database, in the new connection.rb) instead of breaking each other's savepoints. Unknown mode: values raise ArgumentError instead of producing invalid SQL.
  • Types: SQLite JSON / JSONB / UUID columns are String (were Float); a db_type override now types the elements of arrays of that type (uuid[] → Array[String]) instead of replacing the array type.
  • RuboCop: schema names that break a naming cop ("UserId", two-letter or non-ASCII parameters) get a rubocop:disable / enable pair around only the class or method using them.
  • WASM: an empty syntax_suggest/core_ext.rb stub, so plugin errors are no longer prefixed by "`syntax_suggest' was not loaded".
  • Options: unknown keys in an override are rejected (with a "did you mean" hint); clearer rubyVersion range errors.

Behaviour changes worth a release note: pg code that swallowed a failed statement inside transaction now raises; MySQL loads that skipped duplicates now fail; users running Steep on generated SQLite code need library "monitor".

Docs: README reorganised (quick start, one options table, transactions / types / names / limitations sections) without dropping content; Design / Architecture / Testing / Releasing, CLAUDE.md and the PR template updated to the current code (new commands and hooks, stale limitations removed, labels matching the repo).

Tests: unit tests for each generator change; new E2E tests for batch encoding, the rescued pg statement, MySQL binary / duplicate / invalid / nested :copyfrom, SQLite copy atomicity, thread-shared transactions and invalid modes. examples/, golden files, request fixtures and Steep expectations regenerated. Locally (Ruby 3.3): rake test 85 runs, RuboCop and Steep clean, sqlc diff clean with the rebuilt WASM, E2E SQLite 35 / PostgreSQL 49 / MySQL 41 runs green.

Checklist

  • examples/ is regenerated (rake sqlc_generate_process) and committed
  • Tests added or updated (rake test, and rake end2end_* when the generated code changes)
  • The generated code stays RuboCop-clean (rake rubocop) and Steep-clean (rake steep)
  • Labelled (bug, enhancement, documentation, or breaking) so the release notes are grouped correctly

devin-ai-integration Bot and others added 2 commits November 3, 2026 13:48
- pg: :batch* commands encode bytea / json / string-array parameters like the other commands
- pg: a transaction whose failed statement was rescued raises PG::InFailedSqlTransaction instead of returning
- mysql2: :copyfrom is binary-safe (CHARACTER SET binary), atomic, and raises on duplicate keys / invalid values
- sqlite3: :copyfrom runs in a savepoint (all rows or none); one lock per database for threads / fibers;
  unknown transaction modes raise ArgumentError; JSON / JSONB / UUID declarations are Strings
- db_type overrides also type array elements; unknown override keys are rejected
- schema names breaking Naming cops get a rubocop:disable / enable pair where they occur
- WASM: stub syntax_suggest/core_ext so errors are not prefixed by a load warning
- clearer rubyVersion bound errors

Co-Authored-By: Doron Eli Rachman <doron050@gmail.com>
- README: quick start, one options table, sections for the generated code, transactions, types, names and limitations
- docs: current hooks, commands and files; :copyfrom atomicity, encoded batch params, SQLite locking, WASM stub
- labels match the repository (enhancement, not feature)

Co-Authored-By: Doron Eli Rachman <doron050@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration devin-ai-integration Bot added bug Something isn't working documentation Improvements or additions to documentation labels Nov 3, 2026
Co-Authored-By: Doron Eli Rachman <doron050@gmail.com>
@doron050
doron050 merged commit 7e813b7 into main Nov 3, 2026
14 checks passed
doron050 added a commit that referenced this pull request Nov 3, 2026
## Summary
Fixes the bugs found by the final pre-release sweep of 7e813b7, with an
end-to-end test for each.

- **MySQL `:copyfrom` into JSON columns** (regression from #46): `LOAD
DATA ... CHARACTER SET binary` can't create JSON values. JSON columns
now load into `@sqlc_vN` with `SET col = CONVERT(@sqlc_vN USING
utf8mb4)`.
- **PostgreSQL `:copyfrom` with booleans**: the default binary boolean
encoder sent invalid COPY text. Booleans now use
`PG::TextEncoder::Boolean`.
- **PostgreSQL nested `transaction` rescuing a failed statement**: this
used to poison the outer transaction. Now only the savepoint is rolled
back, `PG::InFailedSqlTransaction` is raised, and the outer block can
rescue it and continue.
- **PostgreSQL array parameters**: covers `bytea[]` (binary/NUL),
`jsonb[]` (Hash elements), `[]` and leading `nil`, and
`preparedStatements: false`. These are now sent as array literals
(`encode_array`). Floats are sent as exact text (`Float#to_s`).
- **`:copyfrom` conflict clauses**: SQLite keeps `INSERT OR REPLACE` /
`OR IGNORE` / `ON CONFLICT ... DO ...`. MySQL maps `INSERT IGNORE` and
`REPLACE` to `LOAD DATA ... IGNORE|REPLACE INTO TABLE`. Clauses COPY /
LOAD DATA can't express are generation errors: `ON CONFLICT` /
`RETURNING` on PostgreSQL, `ON DUPLICATE KEY UPDATE` on MySQL.
- **Names**: `snake_case` keeps acronyms together (`MIN_ID` → `min_id`,
not `m_i_n_i_d`). RuboCop exemptions are emitted only where a name
offends, including `Naming/VariableNumber` for repeated parameters
(`id_2`).
- **mysql2 batches** prepare a statement per row, because reusing one
statement broke binary/unsigned values.
- **Dead connections**: pg resets a `CONNECTION_BAD` connection on
checkout. mysql2 discards a closed connection from the pool.
- **Nested `transaction(isolation:)` / `transaction(mode:)`** now raise
`ArgumentError` instead of being silently ignored.

Drivers get the parsed verb/tail through `copy_from_lines(...,
query_name:, sources:, verb:, tail:)` and a `prepares_batch_rows?` hook.

New example tables back the e2e tests: `batch_types` (pg), `copy_types`
(mysql), `tags` (sqlite). README documents the new behaviour and
limitations.

### Behaviour changes (release notes)
- A nested `transaction` with `isolation:` / `mode:` raises
`ArgumentError`.
- A `:copyfrom` query with `ON CONFLICT` / `RETURNING` (PostgreSQL) or
`ON DUPLICATE KEY UPDATE` (MySQL) is rejected at generation time.

### Verified locally
- `rake test`: Ruby 3.1–3.4.
- RuboCop and Steep: clean. Steep expectations were refreshed; only line
shifts and new `PG`/`Mysql2` constant references changed.
- `sqlc diff`: clean through both the newly built `dist/plugin.wasm` and
`exe/plugin.rb`.
- E2E on Ruby 3.2–3.4, against PostgreSQL 16, MySQL 8.3 and SQLite.
- Randomized round-trip tests (insert/select, batch, copyfrom) on Ruby
3.1–3.4. They include binary `bytea[]`, `jsonb[]`, `Float::MAX` and
`5e-324`.

## Checklist
- [x] `examples/` is regenerated (`rake sqlc_generate_process`) and
committed
- [x] Tests added or updated (`rake test`, and `rake end2end_*` when the
generated code changes)
- [x] The generated code stays RuboCop-clean (`rake rubocop`) and
Steep-clean (`rake steep`)
- [x] Labelled (`bug`, `enhancement`, `documentation`, or `breaking`) so
the release notes are grouped correctly

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Doron Eli Rachman <doron050@gmail.com>
@doron050
doron050 deleted the devin/1793712974-sweep-fixes branch November 12, 2026 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant