Keep extracted auto-import packages across cancelled builds by hood · Pull Request #65412 · microsoft/TypeScript

Andrea Cappuccio (hood) · 2026-11-09T20:45:57Z

While typing, the auto-import warm-up gets cancelled on every keystroke and used to throw away every package it had already extracted, so it kept re-reading and re-parsing node_modules from scratch and never finished if you type faster than one full build. This keeps per-package extraction results on the registry, shared from one clone to the next, so a cancelled build leaves its finished packages for the next one:

a package extraction that has started runs to completion even if the build gets cancelled, and concurrent builds wait on the in-flight one instead of starting it again (single-flight per realpath + package name + directory-search flag). New packages still aren't started after cancellation.
the kept results get dropped as soon as anything under node_modules changes, any file is created or deleted, a package.json changes, bucket build preferences or project-reference outputs change, and once a build completes, so memory goes back to what it was once the index is ready.
other edits are logged per snapshot generation. Each kept extraction remembers the generation that produced it and every file it read (entrypoints, alias-resolved files, project-reference sources), and it's only reused if none of those files, nor anything under the package dir, changed between the two generations. That also covers a cancelled build finishing a package after an edit already landed.

recharts `src/cartesian/Line.tsx`, 120 keystrokes at 150 ms, diagnostics + semantic tokens per key	`main` (7 runs)	this PR (3 runs)
process CPU over 18 s of typing	42.6–44.8 s	13.5–14.0 s (−68%), 13.7–13.8 s after review fixes
first completion after typing stops	217–257 ms (index never got built)	11–13 ms (index ready)
`textDocument/diagnostic` median	53–55 ms	47.5–50 ms
completion-heavy typing (completion + diag + semantic tokens per key), CPU	~17.9 s	~17.5–17.9 s (no change)

Measured with this harness: https://gist.github.com/hood/506e87027c83bbec1b44c1f687273349

TYPING_DIR=<recharts> TYPING_FILE=src/cartesian/Line.tsx TYPING_LINE=408 \
  TYPING_ONLY=diagnostic,semanticTokens TYPING_INTERVAL=150ms TYPING_REPS=2 TYPING_POSTCOMPLETE=1 \
  ./lspbench -test.run TestTypingBench

Tests:

new TestCancelledBuildKeepsExtractedPackages: a build cancelled mid-extraction doesn't extract that package again on the next build (fails on main), and a node_modules change still re-extracts it.
new TestCancelledBuildTracksExtractionDependencies: a workspace package re-exporting ../shared gets re-extracted when shared changes after a cancelled build.
new TestPartialExtractionsInvalidation: deterministic interleavings (older build publishing after an edit, edits outside the package root, unrelated edits, sibling snapshots, replacement set after a prefs change).
completion lists are identical to main on the same session across 13 prefixes: 21,610 items, 6,335 of them auto-imports.
go test ./..., npx hereby lint and npx hereby check:format pass.

AI disclosure: I polished my solution, my breakdown of it and -most importantly- did most of the measurements with help from an AI agent. I've gone 2x through everything and I'll handle any follow-up myself ofc.

The auto-import warm-up is cancelled on every keystroke and used to drop every package it had already extracted, so while typing it re-read and re-parsed node_modules over and over without ever finishing. Started extractions now run to completion and are shared with later builds until node_modules, a kept package's directory or the build inputs change.

Copilot

🟡 Changes recommended

Active cache invalidation gaps can reuse stale exports after workspace edits, while cancelled warm-ups can repeatedly restart extraction because compatible cache replacements are not retained.

3 open issues

What changed in this PR

Addresses #65410 by retaining package extraction work across cancelled auto-import builds, reducing repeated parsing while typing in the language server.

Changes:

Shares completed and in-flight package extractions across registry clones.
Adds cache invalidation for file and build-input changes, plus cleanup after successful builds.
Adds regression tests for reuse after cancellation and invalidation after node_modules changes.

File	Description
`tsc/internal/ls/autoimport/registry.go`	Adds shared extraction caching, single-flight coordination, and invalidation.
`tsc/internal/ls/autoimport/registry_test.go`	Tests cancelled-build reuse and re-extraction after package changes.

_{Review effort: Balanced}

Copilot

🟡 Changes recommended

Critical invalidation gaps can reuse stale auto-import exports after filesystem changes, while cancellation handling and repeated validation also risk obsolete work and excessive locking.

4 open issues

3 resolved since last review

_{Review effort: Balanced}

Andrea Cappuccio (hood) · 2026-11-10T17:56:23Z

+	case builder.partialExtractions == nil || requiresNewPartialExtractions(change, r.caseSensitivity),
+		change.Changed.Len() > 0 && builder.partialExtractions.isEmpty():
+		builder.partialExtractions = newPartialExtractions(r.caseSensitivity)


good bot, yep. RegistryChange now has an InvalidateAll flag, Snapshot.Clone passes it through, and it drops the whole partial extraction cache. fwiw completed buckets have the same gap on main since the flag never reached the registry before, I kept that out of this PR to keep it scoped.

Fixed in 247459b.

Andrea Cappuccio (hood) · 2026-11-10T17:56:25Z

+	case builder.partialExtractions == nil || requiresNewPartialExtractions(change, r.caseSensitivity),
+		change.Changed.Len() > 0 && builder.partialExtractions.isEmpty():
+		builder.partialExtractions = newPartialExtractions(r.caseSensitivity)


nice catch clanker, that one's sneaky. When processFileChanges clears a bulk batch because nothing overlaps the snapshot cache, it now marks the summary, and the registry gets that as InvalidateAll, so the partial cache goes away. Simpler than matching every dependency against the batch, and bulk batches are rare enough that losing the reuse there kinda doesn't matter. Regression test: bulk node_modules changes discard extracted packages (1001 watch events, fails on the previous commit).

Fixed in 247459b.

Copilot

🔵 Needs a closer look

Cache invalidation gaps can reuse stale auto-import exports after manifest changes or directory deletions, while unbounded change history and detached cancellation can cause memory growth and delay session shutdown.

2 open issues

5 previously missed issues

In code that hasn't changed since last review

Add concurrent single-flight and waiter-cancellation tests

tsc/internal/ls/autoimport/registry.go:475

The new single-flight path has no concurrent test; the current cancellation subtest cancels before calling extract, so it never exercises a blocked waiter. Add deterministic testing/synctest coverage that blocks the owner, starts another caller for the same key, and verifies both receive the same result from one extraction callback. Also cancel a caller while it is actually waiting and assert that it returns nil without starting another extraction.

Manifest invalidation ignores case-insensitive filename matches

tsc/internal/ls/autoimport/registry.go:486

On a case-insensitive filesystem, a watch event for packages/shared/Package.json refers to the same manifest the resolver reads as package.json, but this comparison does not invalidate it. If cached workspace package a re-exports ../shared, changing that manifest's types field can therefore reuse stale exports after cancellation: the manifest is neither a GetSourceFile dependency nor inside a's directory. Compare the canonical basename using the supplied case sensitivity.

Partial-cache change history grows without bound

tsc/internal/ls/autoimport/registry.go:577

Once a cancelled build leaves partial results for project B, typing in an already-prepared project A can skip warm-up but still append a change record on every snapshot. These records are never pruned, so memory grows with the editing session and returning to B makes each retained package scan the accumulated history under p.mu. Bound the history by rotating the partial cache after a limit, or prune it while preserving generations still reachable by snapshots and pending extractions.

Warm-up tasks can block session shutdown

tsc/internal/ls/autoimport/registry.go:1644

context.WithoutCancel also removes session-shutdown cancellation, while Session.Close() waits in backgroundQueue.Close() for the warm-up task. On large workspaces, shutdown can therefore wait for all already-started package extractions to finish. Detach from build cancellation while retaining a session-lifetime cancellation signal, and remove aborted pending entries rather than caching incomplete results.

Directory deletions can leave stale partial-cache exports

tsc/internal/project/snapshot.go:682

After a cancelled extraction of packages/a re-exporting ../shared/index, a single deletion event for packages/shared is discarded by expandAndFilterWatchEvents because its auto-import-only files are not in the snapshot cache. DroppedWatchChanges is only set for bulk events, so the retry can reuse exports from the deleted dependency. Preserve partial-cache invalidation before ordinary directory-event filtering, and add a directory-deletion variant of TestCancelledBuildTracksExtractionDependencies.

2 resolved since last review

_{Review effort: Max}

Andrea Cappuccio (hood) · 2026-11-10T19:03:22Z

ok clanker, went through the 5 "previously missed" ones from the last review:

single-flight / waiter cancellation tests: added TestPartialExtractionsConcurrency on testing/synctest. The owner gets blocked, a second caller for the same key gets the same result from one extraction, and a caller cancelled while waiting returns nil without starting another one.
Package.json on case-insensitive fs: the basename now gets compared through the case sensitivity.
unbounded change history: capped at 256 records, after that the next clone with changes rotates the partial cache. Typing in a prepared project with a leftover partial cache can't grow it forever anymore.
shutdown blocked by WithoutCancel: the warm-up context now carries the session's background context as the extraction lifetime, so started extractions survive a keystroke cancel but stop on Session.Close. If an extraction gets aborted that way, its pending entry gets removed instead of caching a partial result, and any waiters retry or bail on their own ctx.
directory deletes dropped by expandAndFilterWatchEvents: dropping a delete there now sets DroppedWatchChanges too, same path as the bulk case. Test: deleted dependency directory in TestCancelledBuildTracksExtractionDependencies, fails without the fix.

Fixed in e78087c.

Copilot

🔵 Needs a closer look

Cache ownership and cancellation interactions require human assessment because they can retain obsolete snapshot state and allow duplicate package extraction across builds.

2 open issues

_{Review effort: Max}

Copilot AI balanced review requested due to automatic review settings November 9, 2026 20:45

github-project-automation Bot added this to PR Backlog Nov 9, 2026

github-project-automation Bot moved this to Not started in PR Backlog Nov 9, 2026

typescript-automation Bot added the For Uncommitted Bug PR for untriaged, rejected, closed or missing bug label Nov 9, 2026

Jake Bailey (jakebailey) added the Domain: Performance Performance related label Nov 9, 2026

Jake Bailey (jakebailey) requested a review from Andrew Branch (andrewbranch) November 9, 2026 20:46

Copilot started reviewing on behalf of Andrea Cappuccio (hood) November 9, 2026 20:47

View session

Copilot AI reviewed Nov 9, 2026

View reviewed changes

Comment thread tsc/internal/ls/autoimport/registry.go Resolved

Comment thread tsc/internal/ls/autoimport/registry.go Resolved

Comment thread tsc/internal/ls/autoimport/registry.go Resolved

Invalidate partial extractions by generation and deps

cde9454

typescript-automation Bot assigned Andrew Branch (andrewbranch) Nov 10, 2026

Andrew Branch (andrewbranch) requested a balanced review from Copilot November 10, 2026 16:02

Copilot started reviewing on behalf of Andrew Branch (andrewbranch) November 10, 2026 16:03

View session

Copilot AI reviewed Nov 10, 2026

View reviewed changes

Drop partial extractions on unseen bulk changes

247459b

Jake Bailey (jakebailey) requested a max review from Copilot November 10, 2026 18:10

Copilot started reviewing on behalf of Jake Bailey (jakebailey) November 10, 2026 18:11

View session

Copilot AI reviewed Nov 10, 2026

View reviewed changes

Bound partial extraction history and honor shutdown

e78087c

Andrew Branch (andrewbranch) requested a max review from Copilot November 10, 2026 20:15

Copilot started reviewing on behalf of Andrew Branch (andrewbranch) November 10, 2026 20:15

View session

Copilot AI reviewed Nov 10, 2026

View reviewed changes

Navigation Menu

Repository navigation

Keep extracted auto-import packages across cancelled builds - #65412

Keep extracted auto-import packages across cancelled builds#65412
Andrea Cappuccio (hood) wants to merge 4 commits into
microsoft:mainfrom
hood:perf/autoimport-partial-extraction

Andrea Cappuccio (hood) commented Nov 9, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Andrea Cappuccio (hood) Nov 10, 2026

Uh oh!

Andrea Cappuccio (hood) Nov 10, 2026

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Andrea Cappuccio (hood) commented Nov 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Andrea Cappuccio (hood) commented Nov 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Andrea Cappuccio (hood) Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

Andrea Cappuccio (hood) Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Andrea Cappuccio (hood) commented Nov 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Andrea Cappuccio (hood) commented Nov 9, 2026 •

edited

Loading