Skip to content

[CLI] Boot faster by deferring additional worker creation#3396

Draft
brandonpayton wants to merge 9 commits intotrunkfrom
delayed-worker-boot
Draft

[CLI] Boot faster by deferring additional worker creation#3396
brandonpayton wants to merge 9 commits intotrunkfrom
delayed-worker-boot

Conversation

@brandonpayton
Copy link
Copy Markdown
Member

@brandonpayton brandonpayton commented Mar 15, 2026

Summary

  • Boots only worker 0 initially, deferring remaining workers so the server can start accepting requests sooner (saves ~16s on Windows)
  • Spawns background workers concurrently during WordPress installation so their WASM compilation overlaps with the single-threaded WP install on worker 0
  • Background workers join the pool as soon as they finish booting; post-install mounts are applied at the earliest moment (batch callback for workers ready before WP install, individually for workers ready after)
  • Adds poolAddInstance symbol to object pool proxy for dynamic pool growth without colliding with pooled object interfaces

Test plan

  • Verify CLI boots and serves requests correctly with default worker count
  • Verify post-install mounts (e.g., --mount) work correctly
  • Verify blueprint execution works after boot
  • Test on Windows to confirm startup time improvement

🤖 Generated with Claude Code

brandonpayton and others added 4 commits March 15, 2026 00:55
Add an addInstance() method to the pool proxy that allows adding new
instances after initial creation. The instance is immediately released
into the pool for use. This enables deferred worker boot where workers
join the pool progressively after the first one is ready.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three interleaved optimizations in the CLI boot path:

1. Deferred worker boot (~16s saved): Spawn only worker 0 first and
   boot WordPress on it. Report "Ready!" immediately, then spawn
   workers 1-5 in the background via addInstance().

2. Lazy module loading (~1.5s saved): Convert heavy static imports to
   dynamic import() — @wp-playground/blueprints, @wp-playground/tools,
   @php-wasm/xdebug-bridge, @php-wasm/cli-util, @zip.js/zip.js, and
   blueprint handlers. Handler loading runs in parallel with worker
   spawning using inline workerType instead of handler.getWorkerType().

3. SQLite pre-extraction (~0.9s saved): Extract the SQLite integration
   ZIP to the native filesystem in parallel with worker 0 spawning via
   Promise.all. The worker's preloadSqliteIntegration() finds files
   already in place and skips the expensive PHP-based unzip.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Start spawning background workers (1-N) concurrently with WordPress
installation instead of waiting until after "Ready!" is printed.
Workers join the pool as soon as they finish booting, and post-install
mounts are applied at the earliest possible moment:

- Workers ready before WP install: join pool immediately, receive
  mounts via applyPostInstallMountsToAllWorkers batch callback.
- Workers ready after WP install: apply mounts individually, then
  join pool.

The wordPressReady flag is set synchronously inside the batch callback
before iterating the worker map, closing the race window between the
map snapshot and later .then() callbacks. mountAfterWordPressInstall
is made idempotent in both v1 and v2 worker threads so overlapping
calls from the batch and individual paths are harmless.
Replace the named addInstance method with an exported
poolAddInstance symbol to avoid collisions with the pooled
object's own interface. Fix the playgroundPool type
declaration to include the symbol and cast firstWorkerApi
so TypeScript infers the correct generic parameter.
@brandonpayton brandonpayton changed the title CLI: deferred and concurrent worker boot [CLI] Boot faster by deferring additional worker creation Mar 15, 2026
Restore static imports and remove the preExtractSqliteIntegration
function while keeping the deferred worker boot intact.
bootWordPress() sets bootedWordPress=true before WordPress installs.
When applyPostInstallMountsToAllWorkers later calls
mountAfterWordPressInstall on worker 0, the idempotency guard saw
bootedWordPress=true and returned early — worker 0 never got its
post-install mounts (theme mounts, test directory mounts, etc.).

Use a separate postInstallMountsApplied flag (matching the V2 worker
thread implementation) so the guard doesn't conflict with the
WordPress boot flag.
@brandonpayton
Copy link
Copy Markdown
Member Author

This change doesn't seem to make much of an impact when I test in Windows.

I'd like to test some of the other optimizations from #3376 before doing this one, particularly:

For what @sejas was observing with Playground CLI startup taking about a minute, I wonder if this may have been the cause:

Prior to that change, which has now been merged, it appears the Windows tasklist command was being run for each stale temp directory that has not been cleaned up. If there were many such directories, it could be costly.

@adamziel
Copy link
Copy Markdown
Collaborator

Prior to that change, which has now been merged, it appears the Windows tasklist command was being run for each stale temp directory that has not been cleaned up. If there were many such directories, it could be costly.

If that was the case, wouldn't we observe one slow startup followed by many fast startups?

@brandonpayton
Copy link
Copy Markdown
Member Author

If that was the case, wouldn't we observe one slow startup followed by many fast startups?

🤔 @adamziel Yep, that's right, unless the cleanup is also failing somehow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants