Skip to content

fix(desktop): rewrite the tokenizers prebuild's install name before signing - #153

Merged
Brooooooklyn merged 1 commit into
mainfrom
fix/desktop-tokenizers-install-name
Sep 15, 2026
Merged

Brooooooklyn merged 1 commit into
mainfrom
fix/desktop-tokenizers-install-name

Conversation

@Brooooooklyn

@Brooooooklyn Brooooooklyn commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

What failed

The v0.0.14 desktop release (run 34937845040) failed in Sign, notarize, publishVerify bundle (pre-notarization):

[2/5] every Mach-O carries a Team ID
  scanned  18 Mach-O files
[3/5] no build paths baked into load commands
  FAIL     leaks a build path: Contents/Resources/app/node_modules/tokenizers/tokenizers.darwin-arm64.node
           /Users/runner/work/tokenizers/tokenizers/bindings/node/target/aarch64-apple-darwin/release/deps/libnode.dylib
BUNDLE GATE: FAIL

Why, and why it is our fixup to make

That path is the binary's LC_ID_DYLIB — its own install name, written by the publisher's CI:

LC_ID_DYLIB  /Users/runner/work/tokenizers/tokenizers/bindings/node/target/aarch64-apple-darwin/release/deps/libnode.dylib
LC_LOAD_DYLIB /usr/lib/libc++.1.dylib
LC_LOAD_DYLIB /usr/lib/libiconv.2.dylib
LC_LOAD_DYLIB /usr/lib/libSystem.B.dylib

tokenizers arrived with the delegate token counter (#150) and is loaded lazily by packages/dashboard/src/ingest/tokenizer.ts. This is the same defect mlx-core and @napi-rs/keyring ship with, and package.ts already rewrites both with install_name_tool -id @rpath/<file> on the staged copy, before packager/codesign. Rewriting the install name is therefore the established remedy here — not excluding the package (the token counter is a live feature) and not carving it out of the gate (the gate is what caught this).

stage-app.ts already ships exactly one slice of the package — its copy filter keeps only tokenizers.darwin-arm64.node — so a single rewrite covers the bundle.

The change (one file)

// The dashboard's delegate token counter imports `tokenizers` lazily, and
// stage-app ships exactly one slice of it. Its upstream LC_ID_DYLIB carries the
// publisher's CI path …, which is the same defect mlx-core and keyring arrive
// with, and step [3/5] of verify-bundle refuses it. Rewrite the staged copy.
const tokenizersBinary = 'tokenizers.darwin-arm64.node';
run('install_name_tool', ['-id', `@rpath/${tokenizersBinary}`, join(STAGE_APP, 'node_modules', 'tokenizers', tokenizersBinary)]);

Verification

Check Result
Step [3/5]'s own logic on the real binary, before the rewrite ["/Users/runner/work/tokenizers/tokenizers/bindings/node/target/aarch64-apple-darwin/release/deps/libnode.dylib"]
Same check after the rewrite []
process.dlopen of the rewritten copy loads
yarn typecheck clean
verify-bundle.test.ts + stage-app.test.ts 64/64

I discarded an earlier attempt that carved third-party prebuilds out of step [3/5]; the real defect turned out to be fixable in the bundle, which keeps the gate absolute.

Re-releasing needs your call

The workflow checks out inputs.dry_run ? github.ref : refs/tags/<release_tag>, so:

  • Validate this fix — dispatch with dry_run: true on main (or this branch): it builds and signs, and step [3/5] should pass.
  • Publish v0.0.14 — not possible from the current tag: v0.0.14's tree does not contain this fix. Either move the tag onto the fixed commit, or cut 0.0.15 from main.

My recommendation is 0.0.15: the number 0.0.14 already exists (npm may hold it), and re-pointing a tag after a publish makes one version mean two different trees.


Note

Low Risk
Packaging-only Mach-O install-name rewrite on the staged app tree; mirrors existing mlx-core/keyring fixups with no auth or data-path changes.

Overview
Fixes desktop release verify-bundle step [3/5] failing on tokenizers.darwin-arm64.node, which ships with the publisher’s CI path in LC_ID_DYLIB (same issue already handled for mlx-core and keyring).

package.ts now runs install_name_tool -id @rpath/tokenizers.darwin-arm64.node on the staged copy under node_modules/tokenizers before packager/codesign, so the delegate token counter’s lazy tokenizers import can stay in the bundle without weakening the gate.

Reviewed by Cursor Bugbot for commit 6b24c95. Bugbot is set up for automated code reviews on this repo. Configure here.

…igning

The v0.0.14 desktop release failed the bundle gate at step [3/5]:

  FAIL  leaks a build path: Contents/Resources/app/node_modules/tokenizers/tokenizers.darwin-arm64.node
        /Users/runner/work/tokenizers/tokenizers/bindings/node/target/aarch64-apple-darwin/release/deps/libnode.dylib

`tokenizers` arrived with the delegate token counter (#150), and that path is
its upstream `LC_ID_DYLIB` — the same defect `mlx-core` and `@napi-rs/keyring`
ship with, and `package.ts` already rewrites both of those. This adds the third
fixup, on the staged copy and before packager/codesign, beside the keyring one.
`stage-app.ts` already ships exactly one slice of the package
(`tokenizers.darwin-arm64.node`), so one rewrite covers the bundle.

Measured on the real artifact: before the rewrite, the gate's own
`buildPathLeaks(parseLoadCommandPaths(otool -l …))` reports that path; after it,
nothing — and `process.dlopen` still loads the rewritten addon.

Verified: `yarn typecheck` clean; `verify-bundle.test.ts` + `stage-app.test.ts`
64/64.
@coderabbitai

coderabbitai Bot commented Sep 15, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: c2beacfd-87e9-4449-b7c5-a008f437e10f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cursor

cursor Bot commented Sep 15, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_a971f2e9-4f4f-44b0-b358-fb6a07b0f042)

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 15, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-15T07:15:13.552335Z 6b24c95 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@Brooooooklyn
Brooooooklyn merged commit 19b796c into main Sep 15, 2026
4 checks passed
@Brooooooklyn
Brooooooklyn deleted the fix/desktop-tokenizers-install-name branch September 15, 2026 07:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant