Skip to content

refactor(codemode): run programs on one job queue instead of a fiber per async call - #57976

Draft
rekram1-node wants to merge 4 commits into
devfrom
job-queue-runner
Draft

rekram1-node wants to merge 4 commits into
devfrom
job-queue-runner

Conversation

@rekram1-node

@rekram1-node rekram1-node commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #54340.

Problem

Code Mode ran every async function call, promise, and generator step as its own Effect fiber, and a turn token (#54276) forced those fibers to take turns running program code. That worked, but:

  • Order differed from JavaScript. 10 of 29 microtask-ordering snippets logged in a different order than V8 (finally, promise adoption, empty combinators, async generators, for await over arrays).
  • Stuck programs hung. await new Promise(() => {}) waited for the timeout, and core passes no timeout, so it waited forever (codemode: execute hangs until eviction when a program awaits a promise nothing can settle #54340).
  • It was slow and heavy. Effect bookkeeping on every evaluation step dominated CPU time, and each paused async call held a fiber.

Change

The evaluator is now plain generator functions driven by one job queue per execution. Effect stays at the boundary: execute(), the timeout, interruption, tools, hooks, and extensions.

 const r = await tools.search({ q })
-  fork a fiber for the tool promise
-  suspend this fiber; the turn token passes to the next waiting fiber
-  Fiber.await, retake the turn, resume
+  start the tool as an Effect fiber (host work, as before)
+  yield Await: split the stack at the async function's boundary, park its frames on the promise
+  run the next job from the queue
+  tool settles: queue a job that resumes the parked frames
src/interpreter/
├── runner.ts       # new: job queue, explicit-stack driver, promise abstract operations, host work
├── interpreter.ts  # evaluator as generators; a program call is a Call signal, so recursion lives on the explicit stack
├── promises.ts     # Promise global on the spec's job structure (no fiber per promise)
├── generators.ts   # where a generator body splits off the stack; async generator request queue
└── execute.ts      # Effect boundary: timeout, interruption, result
  • Exact stall detection. No queued job and no running host work while the program body is unfinished means it is stuck, so it fails at once with Execution stalled: ....
  • The host stays responsive. A long job pauses every 1 ms to yield to Effect, then continues before any other job runs. Run-to-completion holds by construction.
  • Spec promise semantics. Resolving functions, thenable jobs, then, finally, combinators, async generators, and async-from-sync iteration follow the spec's job structure.
  • Sync fast paths. Leaves, primitive operators, i++, assignments, member reads, and built-in calls that return a value directly skip generator allocation.
  • Explicit call sites. Built-ins receive their call site, so rejections are located without the CallSite context reference.
  • A guard for the boundary. An oxlint rule forbids effect value imports in src/interpreter/** and src/stdlib/** except in runner.ts, execute.ts, and extensions.ts.

How to review: read runner.ts first. Most of the interpreter.ts and stdlib diff is mechanical (Effect.gen → function*, Effect.ensuring → try/finally, Effect.fail → throw); git diff -w shows +2,720/−2,124 instead of +3,593/−2,997.

Results (Apple M5, best of 18 runs, dev → this branch)

Workload dev job queue
fib(22) 148 ms 21 ms
1M-iteration loop 1,227 ms 126 ms
100k-element map/filter/reduce 414 ms 73 ms
10k sequential awaits 40 ms 4.5 ms
9k .then chain 91 ms 16 ms
100k generator steps 397 ms 45 ms
1k parallel tool calls 8.8 ms 5.5 ms
Memory per paused async call with a pending tool call 17.0 KB 9.1 KB

Behavior differences

  • Ordering. Microtask order now matches V8 on all 29 recorded snippets (19 before).
  • Never-settling programs. These fail at once instead of hanging.
  • Tool results. These settle after already-queued jobs, like I/O, instead of possibly one job earlier.
  • Deep nesting. Nesting inside one expression is bounded by the host stack, about where V8 stops: a 5,000-link .then() chain now fails with "Execution exceeded the maximum nesting depth." (V8 fails it too). Program recursion is unchanged at 10,000 calls.
  • Self-interrupting host work. This keeps its previous rule: it propagates through promises like an uncatchable rejection, is silent when unobserved, and interrupts the execution when it reaches the program body.

Isolation audit

Three independent reviews compared this branch with dev for host-reference leaks, mutation outside the execution, and confinement:

  • References and mutation: nothing new. No driver object, completion, request, AST node, or raw host error reaches program code. Built-ins, prototypes, tool catalog, and shared attribute objects stay per execution. The loop-binding reuse matched dev in 26 cases.
  • Confinement: six regressions, fixed in the last commit, each with a test:
Regression Fix
A loop made only of built-in calls (Iterator.from({ next: Object }).drop(Infinity).next()) never reached a slice, so the timeout could not fire native calls count as steps
Copying out or serializing values with shared references (2^n visits) did not yield the value walk counts each visit
A job that ran a task without signals skipped the slice check every finished job is counted
Unhandled rejections were all formatted in one pass after the program ended, past the timeout each is described when it happens
await Promise.race([pending, 0]) in a loop left one reaction per race on the pending promise combinators detach the reactions they no longer need
Generator.prototype.next.call(asyncGenerator) hit an internal error sync generator methods reject async generators with a TypeError

Pre-existing findings, unchanged by this PR: program regexes update the host's legacy RegExp.$1/lastMatch, and extension results with overridden map/slice/getTime/message/name can bypass copying.

Out of scope

  • Saving or resuming a paused program; paused frames are in-memory generators, as fibers were.
  • A per-execution memory cap.
  • Core passing a timeout to CodeMode.make; CPU-bound loops still need one.

Verification

  • packages/codemode:
    • bun typecheck: clean.
    • bun test: 14,786 pass, 0 fail (+25 test262 files removed from skipped.txt, +29 V8-ordering cases, +12 new tests for stalls, host-result order, interruption, host responsiveness, deep nesting, timeouts through built-in loops, value walks, and rejection reporting, and generator protocol misuse).
  • test262: a full run of all 15,008 vendored files passes 13,330 (13,305 on dev), with no file regressing.
  • Repo: bun run check: 34/34 tasks.
  • Core: packages/core: bun test test/codemode.test.ts test/codemode test/tool-execute.test.ts test/tool-registry.test.ts test/mcp.test.ts test/session-runner.test.ts gives 369 pass, 0 fail.
  • Probes:
    • Timeouts stop while (true) {} and endless promise-job streams within about 10 ms of the limit.
    • External interruption runs tool cleanup.
    • 20 concurrent executions on one runtime stay separate.
    • Tools see the caller's services and trace span.
    • The host event-loop gap during a CPU-bound run is 1.0 ms p50 (same as dev).

@argszero

Copy link
Copy Markdown
Contributor

Context: I have an open PR (#54976) that adds one path inside this same wrap — a host function may return an Effect, forked like a returned Promise. It exists so Code Mode's fetch global can resolve the per-execution Run service and consult the webfetch permission before the request leaves the process (the runtime is cached per inventory, so the guard cannot be closed over it).

Two things from reading job-queue-runner that may be useful to you, since wrap here has no Effect.isEffect branch:

An Effect global now falls into the blocking path. With the branch absent, result is not a Promise, so it takes blocking(ctx.runner.blocking(settle(Effect.succeed(result)))) and fromHost rejects it as "produced an Effect, which the program cannot hold". That may well be intended — I am not asking you to carry the feature. But that is the line where it would be removed, so it should be a decision rather than a side effect.

If that path is wanted later, blocking is the wrong home for it. A fetch guard waits on a host permission prompt, and blocking is a synchronous host call (yield host), so the program would stop for the whole prompt and Promise.all([fetch(a), fetch(b)]) would serialize. The shape that matches the Promise path is runner.newPromise() plus runner.host(settle(result), …, site): it starts the work now, settles through a program promise, and — the part I measured — registers the fiber in fibers, so interrupt() runs its finalizers when the program ends.

That last point is not theoretical. On current dev after #54276, sending an Effect global through the turn/job path instead of the host path meant the fiber never started, so interrupting it never reached Effect.onInterrupt; extensions.test.ts "an Effect still waiting when the program ends is interrupted with it" timed out until the call used the host fork. Your interrupt() over tracked host fibers preserves exactly that.

So: do you want the Effect-global capability in the new runner at all? If yes I will rebase #54976 onto runner.host once this lands and send it as a separate small diff rather than holding this one up. If you would rather fetch stay a Promise-returning global, tell me and I will rework the guard that way — that needs no interpreter change, and I would rather delete my interpreter diff than have yours pay for it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

codemode: execute hangs until eviction when a program awaits a promise nothing can settle

2 participants