Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Generated code, golden files and sqlc configs are compared byte for byte (sqlc diff, golden tests):
# keep LF on every OS, also on Windows checkouts with core.autocrlf.
* text=auto eol=lf
*.bin binary
*.wasm binary
*.cmd text eol=crlf
9 changes: 9 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ jobs:
build-wasm:
name: Build (WASM)
runs-on: ubuntu-latest
timeout-minutes: 10

steps:
- uses: actions/checkout@v7
Expand All @@ -46,6 +47,7 @@ jobs:
lint:
name: Lint
runs-on: ubuntu-latest
timeout-minutes: 10

steps:
- uses: actions/checkout@v7
Expand All @@ -70,6 +72,7 @@ jobs:
name: Codegen Tests
needs: [build-wasm]
runs-on: ubuntu-latest
timeout-minutes: 10

steps:
- uses: actions/checkout@v7
Expand Down Expand Up @@ -108,6 +111,7 @@ jobs:
name: Codegen Options (ruby ${{ matrix.ruby-version }}, types=${{ matrix.generate-types }}, gemfile=${{ matrix.generate-gemfile }})
needs: [build-wasm]
runs-on: ubuntu-latest
timeout-minutes: 10

strategy:
fail-fast: false
Expand Down Expand Up @@ -146,3 +150,8 @@ jobs:
run: >-
./scripts/tests/run_codegen.rb sqlc.ci.yaml
"${{ matrix.ruby-version }}" ${{ matrix.generate-types }} ${{ matrix.generate-gemfile }}
cross-platform:
name: Cross-platform
needs: [build-wasm]
uses: ./.github/workflows/cross-platform.yml
215 changes: 215 additions & 0 deletions .github/workflows/cross-platform.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,215 @@
name: Cross-platform

# Called by build.yml after build-wasm: macOS and Windows run the plugin.wasm built on Linux (the artifact a release
# publishes) through sqlc, and the generated code against natively installed PostgreSQL / MySQL and SQLite.
# The full Ruby / option matrix stays on Linux (tests.yml); here the oldest and newest supported Ruby.
on:
workflow_call:

defaults:
run:
shell: bash

permissions:
contents: read

env:
DB_NAME: tests
DB_USER: root
DB_PASS: pass # ignored by the servers below (trust / empty-password root), required by the pg tests' env
POSTGRES_HOST: 127.0.0.1
MYSQL_HOST: 127.0.0.1

jobs:
codegen:
name: Codegen (${{ matrix.os-name }}, Ruby ${{ matrix.ruby-version }})
runs-on: ${{ matrix.os }}
timeout-minutes: 10

strategy:
fail-fast: false
matrix:
os: [macos-latest, windows-latest]
ruby-version: ['3.1', '4.0']
include:
- { os: macos-latest, os-name: macOS }
- { os: windows-latest, os-name: Windows }

steps:
- uses: actions/checkout@v7

- name: Load tool versions
run: grep -E '^[A-Z_]+=' scripts/tool-versions.env >> "$GITHUB_ENV"

- uses: sqlc-dev/setup-sqlc@bac53b7fb28c039a6c7f5736fd1e89744021bdd6 # v5.0.0
with:
sqlc-version: '${{ env.SQLC_VERSION }}'

- uses: ruby/setup-ruby@v1
with:
ruby-version: ${{ matrix.ruby-version }}

- uses: actions/download-artifact@v8
with:
name: wasm-file
path: dist

- name: Updating plugin sha
run: ./scripts/wasm/update_sha.sh sqlc.ci.yaml

# the process plugin on this OS's Ruby: no bundler, only the standard library + minitest
- name: Unit & golden tests
run: ruby -Ilib -Itest -e 'Dir["test/*_test.rb"].sort.each { |file| require File.expand_path(file) }'

# the committed examples/ (LF, byte for byte) must come out of sqlc + the Linux-built WASM on this OS too
- name: Verify generated code is synced (WASM plugin)
run: sqlc -f sqlc.ci.yaml diff

- name: Verify generated code is synced (process plugin)
run: |
if [ "${RUNNER_OS}" = Windows ]; then
# Windows cannot execute plugin.rb directly: sqlc runs the exe\plugin.cmd wrapper (through cmd.exe, so backslashes)
sed 's#cmd: ./exe/plugin.rb#cmd: exe\\plugin.cmd#' sqlc.local.yaml > .sqlc-windows.yaml
trap 'rm -f .sqlc-windows.yaml' EXIT
sqlc -f .sqlc-windows.yaml diff
else
sqlc -f sqlc.local.yaml diff
fi
- name: Generate with the WASM for Ruby ${{ matrix.ruby-version }} and check the output
run: ruby scripts/tests/run_codegen.rb sqlc.ci.yaml "${{ matrix.ruby-version }}" true true

end2end:
name: End-to-End (${{ matrix.os-name }}, Ruby ${{ matrix.ruby-version }})
runs-on: ${{ matrix.os }}
timeout-minutes: 10

strategy:
fail-fast: false
matrix:
os: [macos-latest, windows-latest]
ruby-version: ['3.1', '4.0']
include:
- { os: macos-latest, os-name: macOS, postgres-version: '16' }
- { os: windows-latest, os-name: Windows, postgres-version: '17' } # the only version on windows-2025

env:
EXAMPLES_DIR: ${{ github.workspace }}/.examples-ruby-${{ matrix.ruby-version }}
BUNDLE_GEMFILE: ${{ github.workspace }}/Gemfile.e2e
BUNDLE_PATH: ${{ github.workspace }}/.e2e-gems

steps:
- uses: actions/checkout@v7

- name: Load tool versions
run: grep -E '^[A-Z_]+=' scripts/tool-versions.env >> "$GITHUB_ENV"

- uses: sqlc-dev/setup-sqlc@bac53b7fb28c039a6c7f5736fd1e89744021bdd6 # v5.0.0
with:
sqlc-version: '${{ env.SQLC_VERSION }}'

- uses: ankane/setup-postgres@b1a598d92f86cb2ed364411898d150d4eac6d5ca # v1
with:
postgres-version: ${{ matrix.postgres-version }}
database: ${{ env.DB_NAME }}
user: ${{ env.DB_USER }}

- uses: ankane/setup-mysql@4aa1b08412d4c8bf537fea5bc412e4e831d839d7 # v1
if: runner.os != 'Windows'
with:
mysql-version: '8.4'
database: ${{ env.DB_NAME }}
user: ${{ env.DB_USER }}

# windows-2025 ships MySQL 8.4 (on PATH); setup-mysql only treats 8.0 as preinstalled there and fails to
# reinstall 8.4 over it, so start the image's server the same way the action does
- name: Start the preinstalled MySQL 8.4 (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'
$PSNativeCommandUseErrorActionPreference = $true
$bin = 'C:\Program Files\MySQL\MySQL Server 8.4\bin'
& "$bin\mysqld" --initialize-insecure
& "$bin\mysqld" --install
net start MySQL
& "$bin\mysqladmin" -u $env:DB_USER create $env:DB_NAME
# after the databases: their bin dirs (PostgreSQL ships its own zlib/iconv DLLs) must come after Ruby's MSYS2
# toolchain on PATH, or gcc fails silently on Windows and no native gem builds
- uses: ruby/setup-ruby@v1
with:
ruby-version: ${{ matrix.ruby-version }}

- uses: actions/download-artifact@v8
with:
name: wasm-file
path: dist

- name: Updating plugin sha
run: ./scripts/wasm/update_sha.sh sqlc.ci.yaml

# the E2E suites below run this WASM output, with the Gemfile it generates
- name: Generate the E2E examples with the WASM
run: ruby scripts/tests/generate_examples.rb "${{ matrix.ruby-version }}" sqlc.ci.yaml --generate-gemfile

- name: Create the schemas (and allow LOAD DATA LOCAL INFILE for :copyfrom)
run: |
psql -v ON_ERROR_STOP=1 -h "${POSTGRES_HOST}" -U "${DB_USER}" -d "${DB_NAME}" -f examples/authors/postgresql/schema.sql
mysql -h "${MYSQL_HOST}" -u "${DB_USER}" -e 'SET GLOBAL local_infile = 1'
mysql -h "${MYSQL_HOST}" -u "${DB_USER}" --default-character-set=utf8mb4 "${DB_NAME}" < examples/authors/mysql/schema.sql
- name: Native build settings for mysql2
run: |
if [ "${RUNNER_OS}" = Windows ]; then
ridk.cmd exec pacman -S --noconfirm --needed mingw-w64-ucrt-x86_64-libmariadbclient
# mysql2's extconf can't glob for mariadb_config.exe through Windows PATH entries, so point at it
mariadb_config=$(cygpath -m "${RI_DEVKIT}/ucrt64/bin/mariadb_config.exe")
test -x "${mariadb_config}"
echo "BUNDLE_BUILD__MYSQL2=--with-mysql-config=${mariadb_config}" >> "$GITHUB_ENV"
else
echo "BUNDLE_BUILD__MYSQL2=--with-opt-dir=$(brew --prefix openssl@3):$(brew --prefix zstd)" >> "$GITHUB_ENV"
fi
# prism (minitest 6), mysql2 and date compile from source on Windows (~2 min): keep the installed E2E gems, per runner
# image (the native extensions link against its libraries)
- name: Runner image
id: image
run: echo "version=${ImageOS}-${ImageVersion}" >> "$GITHUB_OUTPUT"

- uses: actions/cache@v6
with:
path: ${{ env.BUNDLE_PATH }}
key: e2e-gems-${{ steps.image.outputs.version }}-ruby${{ matrix.ruby-version }}-${{ hashFiles('Gemfile.e2e') }}

- name: End-to-End tests (SQLite)
if: ${{ !cancelled() }}
env:
EXAMPLES_DRIVER: sqlite3
run: bundle install && bundle exec rake end2end_sqlite3

- name: End-to-End tests (PostgreSQL ${{ matrix.postgres-version }})
if: ${{ !cancelled() }}
env:
EXAMPLES_DRIVER: pg
run: bundle install && bundle exec rake end2end_pg

- name: End-to-End tests (MySQL 8.4)
if: ${{ !cancelled() }}
env:
EXAMPLES_DRIVER: mysql2
run: bundle install && bundle exec rake end2end_mysql2

- name: Generator options E2E
if: ${{ !cancelled() }}
env:
BUNDLE_GEMFILE: ${{ github.workspace }}/Gemfile.options.e2e
# default gem path (only prebuilt gems): with BUNDLE_PATH, the test's nested `bundle install` on Windows Ruby
# 3.1 trips over setup-ruby's newer Bundler next to the default 2.3 one
run: |
unset BUNDLE_PATH
bundle install && bundle exec ruby tests/end2end_options.rb
- name: Native extension build logs
if: failure()
run: find "$(ruby -e 'print Gem.dir')" -name mkmf.log -exec sh -c 'echo "== $1"; cat "$1"' _ {} \;
1 change: 1 addition & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ on:
jobs:
generate-docs:
runs-on: ubuntu-latest
timeout-minutes: 10

steps:
- name: Checkout repository
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ jobs:
plan:
name: Plan
runs-on: ubuntu-latest
timeout-minutes: 10
if: github.ref == 'refs/heads/main' # releases are cut from main only
outputs:
version: ${{ steps.next.outputs.version }}
Expand Down Expand Up @@ -74,6 +75,7 @@ jobs:
name: Release ${{ needs.plan.outputs.version }}
needs: [plan, build, tests]
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: write # tag + release
env:
Expand Down Expand Up @@ -140,6 +142,7 @@ jobs:
needs: [plan, release]
if: ${{ !inputs.dry_run && !inputs.draft }}
runs-on: ubuntu-latest
timeout-minutes: 10
env:
VERSION: ${{ needs.plan.outputs.version }}
SHA256: ${{ needs.release.outputs.sha256 }}
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ jobs:
unit-tests:
name: Unit Tests
runs-on: ubuntu-latest
timeout-minutes: 10

steps:
- uses: actions/checkout@v7
Expand All @@ -45,6 +46,7 @@ jobs:
compatibility:
name: Plugin on Ruby ${{ matrix.ruby-version }}
runs-on: ubuntu-latest
timeout-minutes: 10

strategy:
fail-fast: false
Expand All @@ -65,6 +67,7 @@ jobs:
end2end-pg:
name: End-to-End (PostgreSQL, Ruby ${{ matrix.ruby-version }}, ${{ matrix.variant }})
runs-on: ubuntu-latest
timeout-minutes: 10

strategy:
fail-fast: false
Expand Down Expand Up @@ -141,6 +144,7 @@ jobs:
end2end-mysql:
name: End-to-End (MySQL, Ruby ${{ matrix.ruby-version }}, ${{ matrix.variant }})
runs-on: ubuntu-latest
timeout-minutes: 10

strategy:
fail-fast: false
Expand Down Expand Up @@ -215,6 +219,7 @@ jobs:
end2end-sqlite:
name: End-to-End (SQLite, Ruby ${{ matrix.ruby-version }}, ${{ matrix.variant }})
runs-on: ubuntu-latest
timeout-minutes: 10

strategy:
fail-fast: false
Expand Down Expand Up @@ -269,6 +274,7 @@ jobs:
end2end-options:
name: Generator options E2E (Ruby ${{ matrix.ruby-version }})
runs-on: ubuntu-latest
timeout-minutes: 10

strategy:
fail-fast: false
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -145,3 +145,4 @@ plugins
# scratch output of scripts/tests/run_codegen.rb
.codegen-test*
.examples-ruby-*
.e2e-gems
4 changes: 3 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,9 @@ UPDATE_GOLDEN=1 bundle exec ruby -Itest test/extended_golden_test.rb # rewrite
* Develop on a feature branch, small commits; PRs target `main`. Label PRs (`bug`, `enhancement`, `documentation`; `breaking` for incompatible changes, see `.github/release.yml`).
* CI: `build.yml` (WASM, lint, `sqlc diff`, Steep, option matrix) and `tests.yml` (unit, compatibility Ruby
3.1/3.2/3.4/4.0, full E2E Ruby 3.1–4.0, Ruby 4.0 module-name variants and PostgreSQL without prepared statements,
plus generator-options E2E on Ruby 3.1/4.0 against PostgreSQL 16, MySQL 8.3 and SQLite). Both must be green.
plus generator-options E2E on Ruby 3.1/4.0 against PostgreSQL 16, MySQL 8.3 and SQLite). `build.yml` also runs
`cross-platform.yml`: the Linux-built WASM, the process plugin and all E2E suites on macOS and Windows (Ruby 3.1/4.0,
native databases). All must be green; every job has `timeout-minutes: 10`.
* **Never create tags or releases by hand and never add release markers to commit messages.** Releases are started by a
maintainer with the *Release* workflow (`docs/Releasing.md`); do not trigger it unless explicitly asked.
* Tool versions (sqlc, ruby.wasm) are pinned in `scripts/tool-versions.env`; bump them there only.
2 changes: 2 additions & 0 deletions Gemfile.e2e
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ examples_dir = ENV.fetch('EXAMPLES_DIR') { raise 'EXAMPLES_DIR must point at gen
driver = ENV.fetch('EXAMPLES_DRIVER') { raise 'EXAMPLES_DRIVER must be set for E2E bundling' }
eval_gemfile File.join(examples_dir, driver, 'Gemfile')

gem 'bigdecimal' # tests/; bundled (not default) gems since Ruby 3.4 / 4.0
gem 'logger'
gem 'async', RUBY_VERSION.start_with?('3.1') ? '< 2.7' : '>= 2.7'
gem 'minitest'
gem 'rake'
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ per table, one module of constants per enum), `connection.rb` (shared helpers) a
* **Logging:** `connect(params, logger: Rails.logger)` / `new(pool, logger:)` logs `QuerySql#get_author (0.4ms)` at
debug level for each query and transaction. Parameter values are never logged.
* **Async / fibers:** it is plain synchronous Ruby and works with the Fiber scheduler ([`async`](https://github.com/socketry/async)):
inside `Sync { ... }`, pooled `pg` and `mysql2` queries run concurrently. MySQL parameters are quoted client-side
inside `Sync { ... }`, pooled `pg` and `mysql2` queries run concurrently (`mysql2` blocks the thread on Windows). MySQL parameters are quoted client-side
with `Mysql2::Client#escape` and sent with `Client#query`, not server-side prepared statements. SQLite queries run
one at a time.
* **Style:** SQL lives in frozen heredoc constants (`GET_AUTHOR_SQL`), comments above a query become its doc comment,
Expand Down
1 change: 1 addition & 0 deletions docs/Testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ They `require` the generated code from `examples/` and run it against a real dat
|---|---|---|
| `build.yml` | push to `main`, pull requests (not for Markdown-only changes), release | build `plugin.wasm`; lint; `sqlc diff` with the WASM and the process plugin; Steep; generate with option combinations (Ruby 3.1/3.2/3.3/4.0 × types/Gemfile) and check the output with the matching Ruby |
| `tests.yml` | same | unit tests; plugin compatibility on Ruby 3.1 / 3.2 / 3.4 / 4.0; full E2E matrix on Ruby 3.1 / 3.2 / 3.3 / 3.4 / 4.0; Ruby 4.0 module-name variants for MySQL and SQLite and module-name + no-prepared-statements variant for PostgreSQL; PostgreSQL 16.2, MySQL 8.3.0, and SQLite; generator-options E2E on Ruby 3.1 / 4.0 |
| `cross-platform.yml` | called by `build.yml` after `build-wasm` | macOS and Windows × Ruby 3.1 / 4.0 (oldest and newest supported), as two parallel jobs each. *Codegen*: unit + golden tests; `sqlc diff` with the Linux-built `plugin.wasm` and with the process plugin (`exe\plugin.cmd` on Windows); WASM codegen check. *End-to-End*: full E2E for SQLite, PostgreSQL (16 on macOS, 17 on Windows) and MySQL 8.4, installed natively (`ankane/setup-postgres`, `ankane/setup-mysql`, the image's MySQL on Windows), using the generated Gemfiles (installed gems cached per runner image); generator-options E2E |
| `docs.yml` | `examples/` change on `main`, manual | regenerates `docs/Examples.md` and opens a PR |
| `release.yml` | manual only | see [Releasing](Releasing.md) |

Expand Down
2 changes: 1 addition & 1 deletion examples/mysql2/query_sql.rb
Original file line number Diff line number Diff line change
Expand Up @@ -1330,7 +1330,7 @@ def csv_line(values)
when true then '1'
when false then '0'
when Time then value.strftime('%Y-%m-%d %H:%M:%S.%6N')
else %("#{value.to_s.b.gsub('\\') { '\\\\' }.gsub('"', '""')}")
else %("#{value.to_s.b.gsub('\\') { '\\\\' }.gsub('"', '""').gsub("\x1A") { '\Z' }}")
end
end
"#{fields.join(',')}\n"
Expand Down
1 change: 1 addition & 0 deletions exe/plugin.cmd
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@ruby "%~dp0plugin.rb" %*
Loading
Loading