A Zig sidecar spike (TASK-355) evaluating whether mt's core engine could move out of the
Rust mt-tauri crate. This directory currently contains a standalone binary that opens mt.db
read-only, serves one library-read HTTP endpoint on loopback, and reproduces the
strip_sort_prefix SQL function used to compute library.artist_sort_key, proven byte-identical
to the Rust implementation via a shared golden-fixture test.
task zig:build # debug build
task zig:fixture # generate tests/fixtures/mt_fixture.db from the real Rust schema
task zig:test # unit tests, incl. std.testing.allocator leak checks
task zig:lint # zig fmt --check
task zig:format # zig fmt
task zig:cross-compile TARGET=aarch64-macos
task zig:cross-compile TARGET=x86_64-macos
task zig:cross-compile TARGET=x86_64-linux
task zig:cross-compile TARGET=x86_64-windowsThe sidecar ships inside the Tauri bundle via externalBin
(crates/mt-tauri/tauri.conf.json: bundle.externalBin: ["binaries/mt-zig-core"]).
Tauri resolves the on-disk binary by appending the Rust host triple to that
name, e.g. binaries/mt-zig-core-aarch64-apple-darwin β not a Zig target
triple, and the two naming schemes disagree (Zig's x86_64-linux is static
musl, filed under the Rust triple x86_64-unknown-linux-gnu).
task zig:stage TARGET=aarch64-apple-darwin
task zig:stage TARGET=x86_64-apple-darwin
task zig:stage TARGET=x86_64-unknown-linux-gnu
task zig:stage TARGET=x86_64-pc-windows-msvczig:stage builds with -Doptimize=ReleaseSafe, maps TARGET to a Zig
target through an explicit hardcoded table (taskfiles/zig.yml), and copies
(never strips) the result into crates/mt-tauri/binaries/. Stripping would
invalidate Zig's ad-hoc code signature on Apple Silicon. Because Tauri's
build script resolves externalBin paths eagerly, even a plain cargo check on mt-tauri requires this binary to already be staged for the
host's Rust triple β tauri:build, tauri:dev, and ci:build all carry a
zig:stage dependency for exactly this reason.
crates/mt-tauri/src/sidecar.rs owns the runtime side: it spawns the
sidecar via tauri-plugin-shell in .setup(), forwards its stdout/stderr
into tracing under target: "sidecar", polls for sidecar.json and
issues one authenticated health-check request, and kills the child on
RunEvent::Exit. A missing or unspawnable binary is logged and does not
prevent the rest of the app from starting.
On macOS, task ci:verify-signing TARGET=<target> (taskfiles/ci.yml) runs
after bundling and before notarizing: it asserts the sidecar's code
signature is valid, that it carries the hardened runtime flag, and that the
.app's nested code signatures verify β so a signing problem fails in
seconds rather than after a notarytool round trip.
mt-zig-core --db /path/to/mt.db --runtime-dir /path/to/app-data-dir--db is opened read-only; --runtime-dir is where the port/token file (see below) is written β
this sidecar does not resolve platform app-data directories itself (see "HTTP server" below for
why). Missing or unreadable --db/--runtime-dir exits non-zero with a message on stderr rather
than binding a port that cannot serve.
-
No system frameworks may be linked, on any platform. Linking a system framework (e.g. a macOS
.framework) makes the build require that platform's SDK on the build host, which defeats cross-compiling all four targets (aarch64-macos,x86_64-macos,x86_64-linux,x86_64-windows) from a single Linux CI runner viazig ccβ the core assumption behind TASK-355.The concrete mechanism this guards against:
SQLITE_ENABLE_LOCKING_STYLEdefaults to1on Apple platforms (vendor/sqlite3/sqlite3.c), which pulls in Darwin-specific VFS code that needsCoreFoundation.build.zigdefines it to0to keep SQLite on its portable POSIX VFS everywhere.
- Source: sqlite-amalgamation-3510100.zip
- Version: 3.51.1 β matches what
rusqlite0.38 /libsqlite3-sys0.36 bundle in the Rust crate (Cargo.lock), satisfying the "at least matching" requirement. - SHA3-256 (of the downloaded zip, computed at vendoring time):
856b52ffe7383d779bb86a0ed1ddc19c41b0e5751fa14ce6312f27534e629b64 - Only
sqlite3.candsqlite3.hare vendored, invendor/sqlite3/.shell.c(the CLI shell) andsqlite3ext.h(runtime extension loading, which is disabled) are omitted. - The repo is Unlicense-licensed and SQLite is public domain, so there is no license conflict.
Carried from libsqlite3-sys' build.rs, minus extensions unused anywhere in
crates/mt-tauri/src/db/:
-DSQLITE_CORE
-DSQLITE_DEFAULT_FOREIGN_KEYS=1
-DSQLITE_ENABLE_API_ARMOR
-DSQLITE_ENABLE_COLUMN_METADATA
-DSQLITE_ENABLE_STAT4
-DSQLITE_THREADSAFE=1
-DSQLITE_USE_URI
-DSQLITE_ENABLE_LOCKING_STYLE=0
-DSQLITE_OMIT_LOAD_EXTENSION
-DHAVE_USLEEP=1
-DHAVE_ISNAN
-D_POSIX_THREAD_SAFE_FUNCTIONS
Dropped relative to rusqlite's build, each unused by any query in crates/mt-tauri/src/db/ and
each real compile time saved: FTS3/FTS4/FTS5, RTREE, JSON1 (a builtin since 3.38 that
isn't queried), SOUNDEX, DBSTAT_VTAB, MEMORY_MANAGEMENT, and ENABLE_LOAD_EXTENSION (kept
explicitly off via SQLITE_OMIT_LOAD_EXTENSION).
SQLITE_ENABLE_STAT4 is deliberately kept β it changes SQLite's query planner, and
TASK-355.5's shadow-diff harness compares query plans/results against the Rust build.
- Download
sqlite-amalgamation-<version>.zipfrom https://sqlite.org/download.html. - Confirm
#define SQLITE_VERSIONin the zip'ssqlite3.hmatches the intended version. - Copy
sqlite3.candsqlite3.hintovendor/sqlite3/, overwriting the existing files. - Update the version, source URL, and SHA3-256 above.
- Run
task zig:testandtask zig:cross-compile TARGET=...for all four targets.
src/strip_sort_prefix.zig reimplements crates/mt-tauri/src/db/models.rs::strip_sort_prefix
(the function the SQL UDF registered in crates/mt-tauri/src/db/mod.rs delegates to): the
first ignore-word that prefixes value, followed by whitespace, is stripped, along with any
further leading whitespace. Therapy? is not stripped by the ignore-word the, since ? is not
whitespace.
tests/fixtures/strip_sort_prefix.json (repo root) is the parity contract: one JSON fixture read
by both a Rust test (db::tests::test_strip_sort_prefix_golden_fixture and
models::tests::test_strip_sort_prefix_matches_golden_fixture) and a Zig test
(strip_sort_prefix.zig's fixture test, plus sqlite.zig's test exercising the real
sqlite3_create_function_v2-registered UDF through SQL). Because both sides read the same file,
parity stays proven as either implementation changes, rather than being a one-time check made at
port time.
Known divergence: case-insensitive matching in the Zig port is ASCII-only (std.ascii),
while the Rust side uses Unicode-aware to_lowercase(). The shipped ignore-words list
(the, a, an β crates/mt-tauri/src/db/schema.rs, mirrored in
app/frontend/js/constants.js) is entirely ASCII, so this is bounded: it can only produce a
different result if a user sets a custom ignore word containing a non-ASCII uppercase letter
(e.g. a Turkish dotted/dotless I, or a German Γ/αΊ case pair).
sqlite.Db.openReadOnly applies the same PRAGMA sequence, in the same order, as the Rust
connection pool's with_init (crates/mt-tauri/src/db/mod.rs):
- Open with
SQLITE_OPEN_READONLY. sqlite3_busy_timeout(db, 5000)PRAGMA synchronous = NORMALPRAGMA foreign_keys = ONPRAGMA cache_size = -64000- Register
strip_sort_prefix.
PRAGMA journal_mode = WAL, table creation, and migrations are write-path / once-per-database
concerns in the Rust crate and are deliberately not ported β this sidecar is read-only.
src/server.zig binds 127.0.0.1 on an OS-assigned port (never hardcoded) and serves one route:
GET /api/library?search=&artist=&album=&source_filter=&limit=&offset=&sort_by=&sort_order=&ignore_words=
HTTP on loopback, rather than stdio, is deliberate: a stdio-only transport would make every
command that could later move to the sidecar a permanent Rust proxy, which defeats the point of
the migration. app/frontend/js/api/shared.js already carries a dormant HTTP client
(API_BASE/request()/ApiError) left over from a removed Python sidecar β this is the server
that client was shaped for. Wiring the frontend to actually call it is TASK-355.6, not this
task β nothing under app/frontend/ changes here.
Loopback is not a security boundary β any local process can connect β so every request must carry
Authorization: Bearer <token>. The port and a fresh 256-bit token are written as JSON to
<runtime-dir>/sidecar.json at 0600 on startup:
{"port": 54321, "token": "..."}--runtime-dir is a caller-supplied path rather than something this sidecar resolves itself:
Tauri's app_data_dir() (crates/mt-tauri/src/lib.rs) is the real production value, and
reimplementing that platform resolution logic in Zig would duplicate it and risk drift. TASK-355.4
(spawning) is what passes the real directory.
The token is regenerated on every startup β a leaked token stops working as soon as the sidecar
restarts β and compared with std.crypto.timing_safe.eql to avoid a timing side-channel on the
comparison itself (the length check ahead of it is unavoidably non-constant-time, but only leaks
the token's length). Missing or wrong token β 401 with {"detail":"unauthorized"}, the shape
shared.js already expects.
Windows note: 0600 is a POSIX file mode and is a no-op there β CreateFlags.mode is ignored
on Windows. ACL-based hardening of the runtime file on Windows is out of scope for this POC.
The response body is written directly to the connection via std.json.Stringify as rows are
fetched from SQLite β nothing is materialized as an in-memory tree first. This makes the response
chunked-transfer-encoded rather than Content-Length-framed (streaming means the total body size
isn't known up front), so a client reading the raw socket must dechunk before parsing JSON β curl
and fetch() both do this transparently.
One consequence of streaming: a SQLite failure partway through row-writing happens after the
200 status line is already sent. There is no way to downgrade to a 500 mid-stream; the client
sees a truncated body. This is an accepted cost of streaming, not a bug to work around.
The query builder (src/library.zig) is a byte-faithful port of
crates/mt-tauri/src/db/library.rs / db/models.rs β WHERE-condition order, ORDER BY
construction (including that ignore_words is string-interpolated into the SQL with ' doubled,
not bound, matching the Rust side exactly), and JSON field order/typing all mirror the Rust source
rather than being "improved."
Two known parity hazards, deliberately not fixed here because fixing them would mean diverging from the Rust behavior TASK-355.5 diffs against:
- No
idtiebreaker. Neither implementation'sORDER BYbreaks ties onid, so rows tied on every sort key have SQLite-defined ordering. Two independent builds (or two runs) can legitimately disagree on tie order. - Per-field row dropping. Rust's
FromSqlis strict per column (db/library.rs'srow_to_track): a row with a type-mismatched column silently drops out of the result (while still counting towardtotal) rather than erroring the whole query.rowIsMappableinlibrary.zigreproduces this per-field, including which columns are lenient (file_ctime_ns/source/remote_iduse Rust's.unwrap_or(...)fallback instead of dropping the row).
duration (SQLite REAL) is written with a .0 suffix when whole-valued (250 β "250.0"), matching
what Rust's JSON serializer does for an f64 that has no fractional part β std.json.Stringify's
default float formatting omits it, so library.zig reformats after the fact rather than trusting
the default writer.
Not handled in this task. The real caller (the Tauri webview, once TASK-355.6 flips the frontend) will need it, but pinning down the exact allowed origin without a real running webview to test against risks silently shipping a wrong value β nothing here would catch it until 355.6. None of this task's Acceptance Criteria require it, so it's deferred rather than guessed at.