Repository navigation
Conversation
mafredri
force-pushed
the
chatd-cache-workspace-home
branch
from
September 29, 2026 12:36
170abd4 to
b80cb3f
Compare
Every generation step of a root chat built the plan path by dialing the workspace agent and sending an LS for the home directory. Preparation waits for it, so each step paid a ping plus a request on a new TCP connection before the model or tool ran. One tool call and one reply ran four steps and four LS calls. The chat runner now keeps the home directory, keyed by agent ID, for as long as it owns the chat, the same way it keeps stop-nudge state across tasks. A new build gets a new agent ID, so a rebuilt workspace is read again. The dial was also the only per-step caller of trackWorkspaceUsage, so the plan path block now bumps workspace activity itself. Without that, steps that run no tools would stop extending the autostop deadline.
mafredri
force-pushed
the
chatd-cache-workspace-home
branch
from
September 29, 2026 12:37
b80cb3f to
0387af1
Compare
Each generation step builds a new workspace context and releases its agent connection when the step ends, so every step that touched the workspace acquired a new AgentConn. On coderd that means another reachability ping to the agent before the first request of the step. The chat runner's workspace state now also holds the connection while the runner owns the chat. Each step still resolves the agent from the database, then reuses the held connection only for the same workspace and agent, and only when the agent row it just read does not show the agent as disconnected. Error paths that clear the step's cached connection also drop the held one, so a bad connection is not reused. The runner releases the connection after its tasks finish.
mafredri
force-pushed
the
chatd-keep-agent-conn
branch
from
September 29, 2026 12:46
83265f1 to
3db3959
Compare
mafredri
force-pushed
the
chatd-cache-workspace-home
branch
from
September 30, 2026 08:04
0387af1 to
e404e13
Compare
Member
Author
|
Folded into #30051, which now reuses the whole workspace context across steps instead of adding a separate holder for the connection.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Each generation step builds a new workspace context and releases its agent connection when the step ends. A turn with one tool call runs several steps, so each step that touched the workspace acquired a new
AgentConn. On coderd that costs another reachability ping to the agent before the step's first request. In the new test, one tool call and one reply dialed the agent twice on top of #30051.The runner workspace state from #30051 now also holds the connection while the runner owns the chat. Each step still resolves the agent from the database, so the checks for deleted workspaces and changed agent bindings are unchanged. The step reuses the held connection only for the same workspace and agent, and only when the agent row it just read does not show the agent as disconnected. Error paths that clear the step's cached connection also drop the held one, and the runner releases it after its tasks finish.
The runner TODO in
ARCHITECTURE.mdfrom #30051 now mentions the connection too.Depends on #30051. This is PR 2 of 3 to reduce round trips per Coder Agents tool call. The next PR reuses TCP connections to the agent through the existing per-agent pool in
ServerTailnet.Implementation plan
Problem
A tool step in a root chat costs about 7 round trips between coderd and the agent for
read_fileand 10 forexecute. About 4 of those happen before the tool runs.Findings
turnWorkspaceContextand releases it at the end.StartGenerationreturns after each action, so a task lasts one step. The chat runner outlives tasks and already keeps per-chat state across them.LS(fixed in perf(coderd/x/chatd): reuse agent connection across steps #30051).ServerTailnetalready pools keep-alive connections per agent for workspace apps.IdleTimeoutis unset and falls back toReadTimeout.Steps
ServerTailnet, set an idle timeout on the agent's API server, and remove the double ping on dial.executein one request.Expected round trips per tool step