Servers have to guess the client's tool-result limit, and never learn when they guess wrong #3216

Before a possible SEP — looking for interest, prior art, and whether this is the right shape.

The problem

A client that caps tool-result size enforces that cap after the server has produced and returned the result. No error, no retry signal, and nothing else reaches the server. The user's call fails; the server logs a success.

So a server returning variable-size results has to guess a client-side constant it cannot observe, and a wrong guess is invisible to the only party able to correct it. The number can only be found by bisecting against production behaviour.

Two costs:

• Servers under-deliver by default. With an unknown ceiling, the safe budget is a small one, so responses get trimmed harder than the client would have required.
• The failure is silent and non-local, which is why it persists rather than being noticed and fixed.

What I measured

Driving one tool at increasing page sizes through one real client:

Threshold for that client: 42,280 < T < 54,262 characters — obtained by bisection, because nothing in the protocol exposes it.

Two details that shape the design question:

1. The enforced quantity is tokens, not bytes. The rejection reads "result (N characters) exceeds maximum allowed tokens". A server cannot compute the client's token count without the client's tokenizer, so any server-side budget is a proxy — correct for one client, confidently wrong for others.

2. It is not one endpoint's problem. Four unrelated surfaces in the same server exceeded the threshold independently. One was a mutating call whose server-side effect had already succeeded when the response was rejected — the user saw an error for work that had been done, which invites retrying a state-changing operation.

Why the open set isn't quite enough

ClientCapabilities is explicitly not a closed set and carries both experimental and extensions, so a client can advertise a limit today with no spec change. I've implemented exactly that and it works.

What it doesn't do is interoperate. If each server invents its own key, a client must advertise N vendor-specific keys to be understood by N servers — the situation capability negotiation exists to avoid. The gap is a shared name, not a mechanism.

For reference, current ClientCapabilities is experimental, roots, sampling, elicitation, extensions; no field describes a size or token limit. maxTokens appears only in CreateMessageRequestParams (sampling, and deprecated), which is a request parameter rather than a capability.

Open questions — where I'd most like input

1. Units. Tokens are what clients enforce but servers cannot compute. Bytes/characters are computable but approximate — JSON escaping alone can inflate raw bytes several-fold, and invalid UTF-8 is a distinct cost class. A documented bytes-equivalent proxy may be the practical answer; a tokens-only field would be honest but unusable by most servers.

2. Whose limit is it? Transport, host application, and model context window can each impose a ceiling. The useful value is the effective minimum, which argues for one resolved number rather than exposing the layers.

3. Is a static capability even the right shape? The limit can vary with remaining context. A per-request hint, or a structured "result too large" error the server can react to, addresses the same problem differently — and an error path has the decisive advantage of being observable, which is exactly what today's failure lacks. I'd be glad to be told (3) is the better direction.

Prototype

Already implemented and in use, if it's useful as evidence of feasibility: read a vendor-prefixed key from extensions, then experimental; validate it (reject wrong types, non-finite and non-integral values, non-positive values, and anything above a plausibility ceiling — each treated as absent rather than trusted); fall back to a measured default; and report on the response which source decided the budget, so an operator can see why a result was trimmed.

Happy to write this up as a SEP if there's interest and a sponsor, or to drop it if the error-path framing in (3) is preferred.