[feat] Introduce partial results as part of progress notifications #383
Conversation
| "chunk": { | ||
| "additionalProperties": {}, | ||
| "description": "The partial result data chunk.", | ||
| "type": "object" | ||
| }, |
There was a problem hiding this comment.
question:
- there is nothing technically wrong with modeling this as a
dict[str, Any], but this is not howCallToolResultmodels its content. Just curious if there is a specific reason not to keep same structure asCallToolResult? Intuitively i would expect them to be very similar. - Also a nitpick on naming but should we call this chunk? why not "content" or something (similar to CallToolResult)
There was a problem hiding this comment.
1/ It was from perspective that progress is not tool specific, even server to client request can have a progress
2/ Yeah also totally good option, only reason is chunk intuitively gives a sense of something incomplete.
There was a problem hiding this comment.
- I dont think the objects in content of CallToolResult are like tied to CallTool - they are generic content types. They're useful for consumer to be able to write logic against the various response modalities. If we return arbitrary JSON then we almost need a new schema for this. Also you could include DataContent from this PR: https://github.com/modelcontextprotocol/modelcontextprotocol/pull/356/files to achieve the JSON part too.
- IMO chunk is kind of specific to LLM nomenclature. I would suggest something more generic either borrowing form CallToolResult "content" or something like "partialResult"
There was a problem hiding this comment.
- Whole RPC result object can be in chunk/content, each result can have different structure. So intention of modeling it like dict is to model it like https://github.com/modelcontextprotocol/modelcontextprotocol/blob/main/schema/2025-03-26/schema.ts#L61
a. Headsup I think that PR got abandoned and RFC: addTool.outputSchemaandCallToolResult.structuredContent#371 is merged. - Notice key just outside of this one is also called partialResult. But not tied to chunk, content is also fine.
| "properties": { | ||
| "_meta": { | ||
| "properties": { | ||
| "partialResults": { |
There was a problem hiding this comment.
NIT: I feel that renaming this streamPartialResults or notifyPartialResults would be more apt than adding a boolean value under partialResults.
There was a problem hiding this comment.
yes that makes sense, I can update
|
How should this be handled by clients when not all results are delivered, or they aren't all received in the order they were sent? As I've noted in #430, notifications aren't guaranteed to be received in any particular order, as there's no way for the client to acknowledge receipt (also noted in the JSON-RPC spec). |
The progress number in the message increments monotonically, making it possible to recognize out of order delivery. However progress doesn’t need to have a predictable step, so recognizing missed result chunks is a problem. It’s probably best to introduce a partial result ID that must increment monotonically by 1 when present. If out of order chunk is received, the client spec could be to disconnect and reconnect with resumption token. Unidirectional SSE is constraining in this regard… however even sync l task results don’t have a delivery receipt guarantee. |
Would reconnection be sufficient here? The messages would have already been sent successfully, so the server would have no reason to buffer them anymore. It'd make sense for the client to hold an ordered buffer and reorder messages correctly according to the server-provided ID instead of forcing in-order receipt, and use a "task complete" notification with the final ID as a cutoff. However, that still wouldn't handle the case where receipt doesn't happen at all, though. Not having any form of client acknowledgement is a fundamental flaw if partial results are intended to work as an alternative to receiving a response all at once. It'd work if partial results were represented with server->client requests instead of notifications, but that then introduces much more synchronization into the process, since the server needs to wait for an ack before sending the next part (it's also unintuitive, but setting that aside). I almost wonder if it'd be better to do this by implementing something like ranges as an optional parameter on resources, and use progress as a way of telling the client where it can read from 🤔 That way this can be best-effort and tolerant of dropped messages (e.g. client holds its last successful read position, reads to the server-provided latest byte on progress notifications - if the operation completes, just read to the end). |
|
I might have missed it, but why make partial results part of progress notifications? Why not have separate notifications for each? One feature that I would like to see is mergeable partial results. For example, if you have a partial result with
Do we think
Since the server will send an empty JSON-RPC response to cap things off, is this flag necessary? |
|
I am wondering if we want a notification to represent partial results or a general streaming mechansim. In my mind, notifications would imply that they can be ignored on client side and would mean that a full result must be presented at the end, duplicating the data provided. I think that streaming of a results via SSE as returns of an RPC call, instead of a notifications, might be generally more fitting for our model, in particular if we consider streaming audio or other formats. Now the problem I see with not using notifications, but relying on streamed results, is that it might stretch JSON-RPC specification. But I think I am okay with that. Hence I rather see streaming within results and not as notifications, unless i am missing a point here. |
|
Good point @LucaButBoring there should be an id to track sequence of chunks, similar to @adamkaplan suggested. On the other suggestion, I think there is value in returning payload as part of streaming result as client doesn't need to do a second lookup to get the actual payload. Thanks, @jonathanhefner , great points. On the larger questions here:
These are both excellent questions, When raising this PR I referenced two options: 1/ https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#partialResults, 2/ https://google-a2a.github.io/A2A/tutorials/python/7-streaming-and-multiturn/#key-features-demonstrated. In terms of guarantees, both approaches are same in the sense they are going to be best effort(client can disconnect). MCP Streamable HTTP has more concrete semantics on this, but still best effort. Between Request/Response and Notifications, If I understand correctly current MCP spec has the expectation that a 1 request should have 1 response and notification doesn't require a response. So both fit well with option1/. Another dimension I was thinking about is that these partial results will be applicable not only for tools, but other requests also like elicitation/sampling. And liked the concept of client passing a token with which it can keep track of progress/results. Eventually progress can be requested out of band also, so instead of receiving progress notification client can also getProgress. PartialResults can very well be a separate notification, I was thinking progress and results are related, so combined them. |
|
This one: #776, is more current thought I also have after implementing more such servers. |
|
I am closing this. If you feel this enhancement is useful, please open a sep as per https://modelcontextprotocol.io/community/sep-guidelines |
7 participants
Added support for streaming partial results through progress notifications, allowing for incremental updates during long-running operations. This enables more responsive agent-based workflows where intermediate results can be shared while a top-level operation is still running. #111, #117, #314.
Motivation and Context
This enhancement addresses a key need identified in discussion #111 for "structured, formatted intermediate updates from server -> client, so a deep agent graph can provide information to the user even while a top-level tool is still being run."
Progress notifications were an ideal mechanism to extend for this purpose, as they already provide a communication channel during long-running operations. By adding support for partial results, we enable servers to stream incremental data to clients without waiting for operations to complete, creating a more responsive and interactive experience.
LSP's partial result progress has been referred for this implementation: https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#partialResults, specially the recommendation around if server send partial results over progressNotification then final result should be empty.
How Has This Been Tested?
The implementation has been validated through:
Breaking Changes
None. This is a non-breaking change that adds new optional fields while maintaining backward compatibility with existing implementations. Clients that don't support partial results can simply ignore the new fields.
Types of changes
[x] New feature (non-breaking change which adds functionality)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[x] Documentation update
Checklist
[x] I have read the MCP Documentation
[x] My code follows the repository's style guidelines
[x] New and existing tests pass locally
[x] I have added appropriate error handling
[x] I have added or updated documentation as needed
Additional context
The implementation adds three key components:
a. chunk: The partial result data (any JSON object)
b. append: Boolean flag indicating whether to append to previously received chunks
c. lastChunk: Boolean flag indicating the final chunk
Protocol
To request partial results, the client includes both a
progressTokenandpartialResults: truein the request metadata:{ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "_meta": { "progressToken": "abc123", "partialResults": true }, "name": "analyzeData", "arguments": { "datasetId": "financial-q1-2025" } } }The server can then include partial results in progress notifications:
{ "jsonrpc": "2.0", "method": "notifications/progress", "params": { "progressToken": "abc123", "progress": 30, "total": 100, "message": "Initial analysis complete", "partialResult": { "chunk": { "content": [ { "type": "text", "text": "## Preliminary Analysis\n\nData processing has revealed initial patterns..." } ] }, "append": false, "lastChunk": false } } }sequenceDiagram participant Sender participant Receiver Note over Sender,Receiver: Request with progress token and partialResults Sender->>Receiver: Method request with progressToken and partialResults=true Note over Sender,Receiver: Progress updates with partial results Receiver-->>Sender: Progress notification (30%, initial chunk) Receiver-->>Sender: Progress notification (60%, append=true) Receiver-->>Sender: Progress notification (100%, lastChunk=true) Note over Sender,Receiver: Operation complete Receiver->>Sender: Empty result response