API client: concurrent Program.getSourceFile calls for one file stampede the cache, each fetching the file whole #65503

Acknowledgement

• I acknowledge that issues using this template may be closed without further explanation at the maintainer's discretion.

Comment

Description

In the JS API client (typescript/async), Program.getSourceFile checks the source file cache and, on a miss, sends
getSourceFile to the server. The result is cached only once its response is decoded. N calls for the same file that
start before the first response arrives each miss the cache (a cache stampede), so the server sends and the client
decodes the whole file N times. The cache's set then returns the existing object, so callers all get one SourceFile
and the work of the other N−1 fetches is thrown away.

The same Program already avoids this for its other per-file lookup: getSourceFileMetadata keeps the request's
promise in sourceFileMetadataCache, so concurrent calls for one file share one getSourceFileMetadata request.
getSourceFile has no such sharing.

The code is Program.getSourceFile in _packages/native-preview/src/api/async/api.ts (main, 2026-11-14; shipped as
getSourceFileWorker in dist/api/async/api.js of typescript@7.1.0-dev.20261114.1):

const retained = this.sourceFileCache.getRetained(path, this.snapshotId, this.project.id);
if (retained) {
    return retained;
}
// Fetch from server
const binaryData = await this.client.apiRequestBinary("getSourceFile", { snapshot: this.snapshotId, project: this.project.id, file });
...
return this.sourceFileCache.set(path, sourceFile, parseOptionsKey, contentHash, this.snapshotId, this.project.id);

This is common when a client resolves symbols in parallel (Promise.all over a file's declarations), because many
of them lead to the same declaration file, lib.dom.d.ts above all.

Repro

import { mkdtempSync, writeFileSync } from "node:fs";
import os from "node:os";
import path from "node:path";
import { API } from "typescript/async";

const dir = mkdtempSync(path.join(os.tmpdir(), "repro-"));
writeFileSync(path.join(dir, "a.ts"), "document.title = 'x';\n");
const api = new API({ cwd: dir });
const program = await api.createProgram([path.join(dir, "a.ts")], { lib: ["lib.dom.d.ts", "lib.es2022.d.ts"] });
const dom = (await program.getSourceFileNames()).find((name) => name.endsWith("lib.dom.d.ts"));

let start = performance.now();
await Promise.all(Array.from({ length: 10 }, () => program.getSourceFile(dom)));
console.log("10 concurrent:", Math.round(performance.now() - start), "ms");

api.clearSourceFileCache();
start = performance.now();
await program.getSourceFile(dom);
console.log("1 call:", Math.round(performance.now() - start), "ms");
await api.close();

Actual

On macOS arm64, Node 26, typescript@7.1.0-dev.20261111.1:

• 10 concurrent calls take 317 ms, sending 10 getSourceFile requests. The requests were counted by wrapping
  client.apiRequestBinary.
• 1 call after clearSourceFileCache() takes 24 ms with 1 request.
• A call after the concurrent batch sends no request, so the cache works once a response has landed.

In a larger client that resolves a file's declarations in parallel, the duplicate fetches of lib.dom.d.ts took a
request from about 100 ms to about 1.6 s.

Expected

Concurrent calls for one file share a single request: 10 calls cost about as much as one.

Possible fix

getSourceFile can do what getSourceFileMetadata does: a Program has one snapshot and project, so the path is the whole key, and the
entry goes once the request settles (failure included), leaving the cache as the one home of a fetched file.

private sourceFileRequests = new Map<Path, Promise<SourceFile | undefined>>();

async getSourceFile(file: DocumentIdentifier): Promise<SourceFile | undefined> {
    const path = this.toPath(resolveFileName(file));
    const retained = this.sourceFileCache.getRetained(path, this.snapshotId, this.project.id);
    if (retained) {
        return retained;
    }
    let request = this.sourceFileRequests.get(path);
    if (request === undefined) {
        request = this.fetchSourceFile(file, path).finally(() => this.sourceFileRequests.delete(path));
        this.sourceFileRequests.set(path, request);
    }
    return request;
}

// The current body from "Fetch from server" on, unchanged.
private async fetchSourceFile(file: DocumentIdentifier, path: Path): Promise<SourceFile | undefined> { ... }

The sync client (api/sync) cannot overlap calls, so it needs no change.