Repository navigation
Conversation
This was referenced Sep 26, 2026
This was referenced Sep 26, 2026
mafredri
force-pushed
the
mafredri/codagt-757/03-agent-process-tool-calls
branch
2 times, most recently
from
September 28, 2026 11:11
f866e85 to
fe0677c
Compare
mafredri
force-pushed
the
mafredri/codagt-757/04-chatd-execute-tool-calls
branch
2 times, most recently
from
September 28, 2026 11:19
1c397cc to
ebecc64
Compare
mafredri
force-pushed
the
mafredri/codagt-757/03-agent-process-tool-calls
branch
from
September 28, 2026 11:21
fe0677c to
6854f3b
Compare
A task retry or ownership change runs unresolved tool calls again, and execute started a new process each time. Workspace agents with API 2.13 record tool calls by their tool call headers and run each at most once, so chatd now identifies execute requests to them. - The generation dispatcher checks once per batch whether the chat's agent is capable (api_version at least 2.13, from the agent row getWorkspaceConn uses) and only then attaches a ToolCallIdentity, which chatloop hands to each tool call. Other agents get today's requests. - execute sends the tool call headers and, in the foreground, the execute timeout, then waits on output until the agent reports exit, timed_out, or canceled, taking the wall time from duration_ms. - Coded refusals map to not run or unknown. A request without an answer is sent again until AgentAnswerTimeout (1 minute) ends, then the result is unknown with the tool call UUID. An HTTP error answer keeps today's text. - NewToolCallTools builds execute, edit_files, and write_file for generation, so the interrupt run can build the same requests.
mafredri
force-pushed
the
mafredri/codagt-757/04-chatd-execute-tool-calls
branch
from
September 28, 2026 11:24
ebecc64 to
62a44a8
Compare
Member
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A chatd task retry or ownership change runs every unresolved tool call again, and
executestarted a new process each time (the CODAGT-757 incident had one tool call start its command 36 times). This PR makes chatd follow the tool call contract forexecute.executeLocalToolsgives each tool call a context carrying its identity: chat ID, assistant message ID, provider tool call ID, and a tool call age computed from one database clock read per batch plus coderd's monotonic time since that read, so coderd's clock is never compared with the database's.executeattaches that identity to its start request. When the agent returns the process it already started for the tool call, a foregroundexecutewaits only for what remains of its timeout (timeoutminus the start response'sRunAge, the process's run age measured by the agent), looping over the agent's 5-minute wait cap; with no time left it reads the process once and reports the real exit or today's timed-out result withbackground_process_id; if that read fails too, the result says the outcome is unknown instead of claiming a timeout. Each start request waits at mostchattool.AgentAnswerTimeout(1 minute) for the agent's answer, so a connected agent that never answers gives unknown with the tool call UUID instead of a task retry every 15 minutes;chattool.AwaitAgentAnswertells that timeout apart from the task context ending, which keeps today's text because nothing commits. It serves US1, US4 and US5.chattool.ClassifyAgentErrorsorts an error from an agent request made for a tool call into a refusal (ToolCallErrorcode), an HTTP answer, no answer (the request may have arrived), or an answer that could not be read.chattool.AgentErrorTextmaps such an error to a result text with each tool's own words, soexecutehere and interrupt handling and the file tools in later PRs follow the same rules. Forexecute,agent_started_after_tool_call, no answer, and an unreadable answer report that the outcome is unknown and point atprocess_outputwith the tool call's process ID, instead of inviting the model to run the command again.input_mismatchreports that the request changed nothing because a process for the tool call already exists with a different input.stale_tool_call,tool_call_canceled, and HTTP error answers keep today's text; the refusals only reach a stale attempt whose commit fails the history version fence. With an identity, a failure to get the workspace connection is also unknown, since an earlier attempt may have started the command; a chat without a workspace, a deleted workspace, and a workspace with no running agent keep today's result, because no agent exists that could have run it.An agent without tool call support returns its own process ID, and
executethen behaves exactly as before (US8); without an identity, results are today's. No model-facing schema changes. Deliberately left out: interrupt handling (next PR), file tools, the attempt timer, andprocess_output. Today's recursive wait over the agent's cap is kept rather than porting the earlier stack's loop with a retry delay, so the old-agent and no-identity paths stay unchanged.ARCHITECTURE.mdgets a TODO at the "at least once" paragraph for the human author.Part 4 of 6 of the CODAGT-757 stack.
Depends on #29958.
Refs https://linear.app/codercom/issue/CODAGT-757
Design: chatd and workspace agent tool call contract
CODAGT-757: contract between chatd and the workspace agent for tool side effects
Status: implemented by the six draft PRs in section 9. File and line references in sections 2 to 7 describe
mainat9afdea4808, the base the design was written against.0. Terms
Every term below has one meaning in this document. The last column lists earlier names (in this draft or in the #27369 stack) that the term replaces.
chat_messages.id) of the assistant message that contains the tool call.unresolvedToolCallsFromHistory).runTaskWithRetry).call_ageCoder-Tool-Call-Message-Id,Coder-Tool-Call-Id,Coder-Tool-Call-Age-Ms, sent next to the existingCoder-Chat-Idon every request with a side effect.Idempotency-Key,op_idworkspacesdkfunction on both sides. Used as the process ID of a process the tool call starts, and as the ID in the cancel route.stale_tool_callagent_started_after_tool_callcreated_before_agent_start,unknowntool_call_canceledagenttoolcallthat applies the decision table to the start, edit, and write routes, runs the handler at most once per tool call, and records its response.Content-Type, and body of the one run of a tool call's request. Repeated requests get it back unchanged.executethe run age is the process age. Returned in theCoder-Tool-Call-Run-Age-Msresponse header and in cancel answers.age_msstop_if_run_age_below_msexecutestops waiting: process start plus thetimeoutargument.Spelling:
canceled, matching the codebase (514 uses versus 5 ofcancelledincoderd,agent,codersdk).1. User stories
Each story ends with the observable condition that proves it.
execute. The process is killed and the result says it was canceled by the user, with the partial output. Proof: the process is gone after the interrupt; the result is not "interrupted before it produced a result".edit_filesorwrite_fileis in flight, the result is the real edit result, "not applied", or "unknown" with the reason. (Design requirement: the contract must support it. Implemented in the final PR of the stack, afterexecute.)process_signalor agent shutdown ends it.tool_call_canceled.2. What breaks today
executestarts a new process each cycletasks.go:28,tasks.go:155-174; CODAGT-757 incident, 36 starts in 9h (Linear, 2026-08-27)CommitStepARCHITECTURE.md:884ARCHITECTURE.md:882("at least once")tasks.go:316,tasks.go:696-773,message_conversion.go:27;ARCHITECTURE.md:880agent/agentproc/process.go:120Root cause: the agent picks the ID of every side effect, so chatd cannot refer to it again without the original response. And chatd treats "I stopped waiting" as "the work stopped".
3. Who owns what
Rejected:
PUTto client-chosen URLs (earlier revision). It replaced existing routes; headers keep the existing API.4. The contract
API
POST /api/v0/processes/startCoder-Tool-Call-Run-Age-Ms.GET /api/v0/processes/{id}/outputPOST /api/v0/processes/{id}/signalprocess_signaltool.POST /api/v0/edit-filesPOST /api/v0/write-filePOST /api/v0/tool-calls/{id}/cancel{"stop_if_run_age_below_ms": N}.{id}is the tool call UUID. Returns whether the request ever ran, its recorded response, and the state of the process it started, if any.Every tool with a side effect uses the same headers, the same middleware, and the same cancel route; only the process stop rule is tool specific.
What the agent does with a request
For a start, edit, or cancel request in chat C, for a tool call in message M:
stale_tool_call, nothing happensstale_tool_callinput_mismatchif the request differs; 409tool_call_canceledif a cancel recorded itstop_if_run_age_below_ms, kill it; otherwise leave it running. Return the recorded response and the process stateagent_started_after_tool_call, nothing happensagent_started_after_tool_callWhen C's latest message ID rises, the agent drops the tool call records of older messages in C. Processes themselves are unaffected. A background process keeps running and stays addressable by its process ID for
process_outputandprocess_signal, and it is removed after exit as today.Why "no record" proves "never received"
unresolvedToolCallsFromHistoryreads the last assistant message only (generation.go:295-305). A new assistant message is generated only after every local tool call of the previous one has a result (generation.go:207-218).chat_messages_id_seq,dump.sql:2082). Every transition locks the chat row before inserting (ARCHITECTURE.md:98), so IDs increase within a chat.agent_started_after_tool_call.State per chat: one latest message ID, plus records for that message's tool calls. It grows with the number of chats, not the number of tool calls.
Agent guarantees
context.Background(),process.go:126-129; edits do not check the request context.)api.go:167-177,api.go:228-237) stays. Cancel needs no ownership lookup: its path ID must equal the UUID derived from the request's own chat ID and headers, otherwise 400. Requests with tool call headers but withoutCoder-Chat-Idare rejected with 400. The tool call UUID includes the chat ID, so two chats never derive the same process ID.Coder-Tool-Call-Age-Msand returns run ages (Coder-Tool-Call-Run-Age-Ms, and in cancel answers). Both are durations.agent_started_after_tool_callrather than a second run.chatd obligations
StartInterrupt) sends oneCancelToolCallfor every unresolved tool call whose tool sends tool call headers (execute;edit_filesandwrite_filein the final PR), withstop_if_run_age_below_msset to the effective timeout for a foregroundexecuteand 0 otherwise, and commits results built from the answers. It waits for the answers; its timeout exists only so an unreachable agent yields "unknown". A process past its execute deadline, or started in the background, is never stopped (US6).timeout- run age. Every start, edit, write, and cancel request waits at most the agent answer timeout for its answer; when that expires while the task still runs, the result is unknown with the tool call UUID. When the wait for output reaches the execute deadline and the output cannot be read either, the result says the outcome is unknown, not that the command timed out.executeresultedit_filesresultbackground_process_id(today's behavior); background: the background start resultagent_started_after_tool_callstale_tool_callortool_call_canceledOld agents
I confirmed with a throwaway chi test, mirroring today's process routes, that the new paths return 404. The test has been removed.
Inside the agent
The record store is one per agent, in
agenttoolcall: keyed by chat ID, tool call message ID, and provider tool call ID, holding each chat's latest message ID, the agent start, and each tool call's recorded response. The tool call middleware wraps the start, edit, and write routes, so those handlers stay unaware of repeats. The one cancel handler lives next to the store and asks process management to stop a tool call's process under the run-age rule; file tools need no stop step. Process management keeps owning output, exit state, and signals.5. Tool coverage
execute(foreground and background) first, because it has the worst failure today: duplicate processes that keep running.edit_filesandwrite_filein the final PR, using the same headers, decision table, and cancel route.write_filereplaces the whole file (agent/agentfiles/files.go:302,files.go:574-629), so repeating it is less harmful than repeatingedit_files, but it follows the same contract.read_file,ls,process_output,process_list. They have no side effect, so running them again is harmless.spawn_agent,attach_file(CODAGT-875), andcreate_workspace. The same idea (identity from the tool call, enforced by a database constraint) applies separately.6. Relation to the current stack
agent_started_after_tool_callinput_mismatchstart_pendingStartedAtcompared against coderd's clockedit_filesandwrite_fileuse the same shape (final PR)The message ID was questioned in #27370 review as possibly unnecessary. It is now required: it is how the agent knows which records it may drop.
7. Residual risks
agent_started_after_tool_calland report "unknown". Persisting records to disk would not bring the processes back.agent_started_after_tool_calleven if no request ever reached an earlier agent. That takes an agent restart between committing the tool call and its first request, or a request landing within the margin right after workspace start./write-filestill streams.edit_filescan partially apply on a disk error (agent/agentfiles/files.go:452-454). The recorded result is the handler's existing error response, which does not list the files written before the failure. Unchanged behavior; the recording only replays it.ARCHITECTURE.md:880-882("best-effort" interrupts, "at least once") would change. Per repo rules, the implementing PR leaves TODOs there for the human author.8. Decisions and open questions
Decided (operator, 2026-09-25):
executeonly. Aprocess_outputwait on a background process has no side effect and is not canceled. If CODAGT-1108's combined process view confuses users later, extending C5 is a small follow-up. (From the final PR on, unresolvededit_filesandwrite_filecalls also get a cancel, but only to learn their state; an edit in progress still finishes.)PUT(operator, 2026-09-25).executefirst;edit_filesandwrite_filein the final PR of the stack. They are in the design so the contract is general, not execute-shaped.write_filegets no special exception.Coder-Tool-Call-*headers, not one structured header that the agent would have to parse.Open: none.
9. Implementation stack (revision 2)
This section is the single source of truth for names, signatures, and PR boundaries. Anything not listed here is out of scope; a deviation needs a user story (section 1 style) that motivates it. Revision 2 (Q8) replaces the per-route cancel routes, the per-package record tables, and the
age_msbody fields of revision 1. All six PRs are published drafts; rework lands as new commits and children merge their parents (no force-push).PRs
mafredri/codagt-757/...)01-sdk-tool-callmaincodersdk/workspacesdk: tool call identity, headers (incl. run age), UUID, per-request headers, error codes,CancelToolCallclient.02-agent-tool-call-recordsagent/agenttoolcall: the record store, the tool call middleware, the cancel handler, the error table.03-agent-process-tool-callsagent/agentprocand wiring: start runs through the middleware with the tool call UUID as process ID; the process stop rule; reaping rule.04-chatd-execute-tool-callscoderd/x/chatd: dispatch identity, tool call age,executesends headers, attaches, deadline from run age, result mapping, the shared error classifier.05-chatd-interrupt-cancelcoderd/x/chatd: interrupt handling sendsCancelToolCallfor every unresolved tool call and commits results from the answers.06-file-tool-callsedit_filesandwrite_filethrough the same middleware and cancel route; SDK result rebuilding; chatd file tools.PR 1: SDK (
codersdk/workspacesdk)Unchanged from revision 1:
ToolCall,SetHeaders,ToolCallFromHeaders,ToolCallUUIDNamespace,ToolCallUUID(test vector: chat aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee, message 42, tool call "toolu_01/%x y" gives 1a9cb501-66b5-51bd-a7c6-1bf0e500b268),WithToolCall,ToolCallFromContext,ToolCallErrorCodeand its four codes,ToolCallError, and the coded-409 decoding helper.Changes:
Ageplus the time spent connecting, and send over that connection. Requests without a tool call are unchanged.AgentConn:CancelProcessandCancelProcessResponseare replaced byCancelToolCall(ctx context.Context, id string, req CancelToolCallRequest) (CancelToolCallResponse, error):POST /api/v0/tool-calls/{id}/cancel, headers fromWithToolCall, coded 409s as*ToolCallError.StartProcessfillsRunAgefrom the header. Regenerateagentconnmock.PR 2: record store, middleware, cancel handler (
agent/agenttoolcall)Revision 1's generic
Records[V]and PR 6'sChatsmerge into one store of recorded responses. The decision rules and their order are unchanged (stale; record exists;clock.Since(start) - ageMargin <= agemeans the agent started after the tool call; otherwise run once),ageMargin = 2 * time.Second, ages clamped at 0, per-chat state kept for the agent's lifetime, older messages' records dropped when a chat's latest message ID rises, waiters wake on success, error, and panic.type Store, one per agent,func NewStore(clock quartz.Clock) *Store(the agent start isclock.Now()).func (s *Store) Middleware(next http.Handler) http.Handler:nextunchanged.agentchat.FromContext) or with malformed headers. Input hash = SHA-256 over method, URL path, raw query, and the full request body (the body is buffered; a new body reader is handed tonext).next, runsnextonce with a response recorder, records status,Content-Type, and body, and writes them to the client. Repeated requests get the recorded response. Every answered response (new or replayed) carriesCoder-Tool-Call-Run-Age-Ms, the time since the record was created on the store's clock.ToolCallErrorbodies from one table of codes and sentence-case messages in this package.func FromContext(ctx context.Context) (ToolCall, bool)for handlers, whereToolCallholds theKeyand the tool call UUID.func (s *Store) Current(key Key) bool(the key is in its chat's latest message and has a record).POST /api/v0/tool-calls/{id}/cancel:{id}must equal the UUID derived from the request's own chat ID and headers (400). No record and provable: record canceled, answerstarted: false. Pending run: wait. Started: askstopper(nil allowed), answer the recorded response plusProcesswhen found. Sentinel errors: 409 codes. Request context ended: return without writing.PR 3: agent processes (
agent/agentproc,agent/agent.go,agent/api.go)store.Middleware(store passed toagentproc.NewAPIthrough an option). With a tool call in context,manager.startuses the tool call UUID as process ID and remembers the key; otherwise unchanged.agentprocimplementsProcessStopper: find the process by ID; running with run age below the threshold: SIGKILL the process group, mark it canceled by a cancel, wait for exit (bounded by ctx); running at or above: leave it; exited: final state. ReportCanceledfor a process a cancel killed, on every later cancel too. Run age from the process's own start time on the manager clock.store.Currentis not reaped; otherwise the 5-minute rule is unchanged.POST /processes/{id}/cancel,age_msin bodies,WithToolCallChats.agent/agent.gobuilds oneagenttoolcall.Storeon the agent clock;agent/api.gomounts the cancel route withprocessAPIas stopper.Closekills, one git watcher notification, chat isolation) and add the run-age rule: below threshold kills, at or above leaves running, 0 never stops.PR 4: chatd execute (
coderd/x/chatd)Unchanged from revision 1 except:
StartProcessResponse.RunAge(clamped at 0).chattoolholds the agent answer timeout (1 minute). Start requests wait at most that long; when it expires while the tool's context is still live, the result is unknown with the tool call UUID. The foreground start still also ends at the execute timeout, whichever comes first.processSnapshotResultremoved (waitForProcesswith an expired context gives the same result and texts; prove it with the existing tests);chatlooptakes one batch-levelToolCallIdentityinstead of three fields and thetoolCallBatchtype; small helpers for attaching headers, the connection-error check, age calculation, the default timeout, and message reload, each only where it removes duplication.executekeep today's result),input_mismatchwording, the regression tests, the oneARCHITECTURE.mdTODO line.PR 5: chatd interrupt (
coderd/x/chatd)CancelToolCallper call, in parallel under the agent answer timeout, outside the locked transaction.stop_if_run_age_below_ms: the effective timeout for a foregroundexecute, 0 for backgroundexecute(and for file tools in PR 6).chattool;tasks.goonly collects results and commits them. Foreground: not started gives "not run";Process.Canceledgives canceled by the user after the run age with partial output;Process.Runninggives the timed-out, still running result withbackground_process_id; exited gives the real result. Background: not started gives "not run" (US10); running or exited gives the background start result withbackground_process_id. A start recorded as successful whose process the agent no longer reports gives the background start result (background) or unknown with the tool call UUID (foreground). Errors go through PR 4's classifier; old agent (404 without a code), no workspace, deleted workspace, and no agent forexecutekeep today's result; unreachable and unreadable give unknown with the tool call UUID.ProcessOutputpre-checks,InterruptBackgroundExecute, the per-tool branching intasks.go.ARCHITECTURE.mdTODO line.PR 6: file tools
POST /edit-filesandPOST /write-fileare wrapped by the samestore.Middleware(store passed toagentfiles.NewAPIthrough an option). Removed:agentfiles/toolcall.go, thefileResultrefactor infiles.go, per-route input hashing, the file cancel routes,WithToolCallChats, theChatstype. The file handlers go back to today's code apart from what the middleware needs.EditFilesandWriteFilereturn*ToolCallErrorfor coded 409s;CancelToolCallResponsegets methods that rebuild exactly whatEditFilesorWriteFilewould have returned from the recorded response.CancelEditFilesandCancelWriteFileare removed.edit_filesandwrite_filekeep their identity, classifier, and wording from revision 1; interrupt handling uses PR 5's one path withstop_if_run_age_below_ms0 and PR 6's result rebuilding.chattool.InterruptedCalldecides per tool whether a call gets a cancel and maps the answer, sotasks.gohas no tool switch and one converter for every committed interrupt result. The twoARCHITECTURE.mdTODO lines cover file tools.chatloop.TruncateToolResult). Story: an interruptededit_filescall whose recorded result holds a multi-megabyte diff would otherwise be committed uncut and overflow the next prompt, while the same result arriving live is truncated.TestAgent_ToolCalls(realAgentConnthrough the agent router) covers the one cancel route for a process and an edit, a stale request, and the run-age rule.