Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions coderd/x/chatd/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ I don't recommend reading the rest of section thoroughly if this is your first t

- `Create(initialMessages)` creates a new chat, initializes `snapshot_version` to 1, inserts its initial history, and lands in `running`. The inserted initial history sets `history_version` to 1. Since the queue has not changed, `queue_version` remains 0. This transition is a special case: since the chat does not exist at the time it's run, the chat row cannot be locked before the transition is applied.
- `SetArchived(archived)` sets or clears the archived marker for one chat.
- `SendMessage(m, busy_behavior)` inserts a user message directly when the chat is idle, or queues it when the chat is busy. `busy_behavior` must be either `queue` or `interrupt`. With `busy_behavior=interrupt`, it also requests interruption or cancels a pending dynamic-tool action as needed.
- `SendMessage(m, busy_behavior)` inserts a user message directly when the chat is idle, or queues it when the chat is busy. `busy_behavior` must be `queue`, `steer`, or `interrupt`. With `busy_behavior=interrupt`, it also requests interruption or cancels a pending dynamic-tool action as needed.
- `EditMessage(k, replacement)` clears queued messages, cancels or obsoletes active work, marks the truncated active-history suffix as deleted, inserts the replacement turn followed by any caller-provided suffix messages, and lands in `running`.
- `DeleteQueuedMessage(qid)` removes one queued message without changing the active history.
- `PromoteQueuedMessage(qid)` makes a queued message the next message to process. It reorders the queue, interrupts active work, cancels pending dynamic-tool action, or promotes into history immediately as required by the input state.
Expand All @@ -131,6 +131,7 @@ I don't recommend reading the rest of section thoroughly if this is your first t
- `EnterRequiresAction` records a pending-action episode by relying on the committed assistant tool-call messages as the durable call set, sets `requires_action_deadline_at`, which is a timestamp 5 minutes in the future, and lands in `requires_action`.
- `FinishInterruption(optionalPartialStep)` inserts one final interrupted assistant/tool suffix if present, or finalizes interruption without a suffix if none is available, clears the interrupting state, and lands in `waiting` if no queued message is promoted. If interrupt finalization also promotes the queue head, it inserts the promoted queued message into history and lands in `running`.
- `RecordGenerationAttempt` verifies the chat is still `running`, increments `generation_attempt`, and returns the updated chat snapshot.
- `PromoteQueuedBeforeStep` verifies the chat is still `running`, removes the due queued messages from the queue, inserts them into history in queue order, and stays in `running`. A queued message is due if it is a `steer` or `interrupt` message, or if a `steer` or `interrupt` message is behind it in the queue.
- `RecordRetryState(payload)` verifies the chat is still `running`, stores the retry payload sent to clients as `retry_state`, and returns the updated chat snapshot.
- `FinishTurn` completes the current generation turn atomically. If the queue is empty, it lands in `waiting`. If the queue is non-empty, it removes the queue head, inserts it into history as a user turn, and lands in `running`.
- `FinishError(err)` parks the chat in `error` and persists `last_error = err`, replacing any previously stored error. It is allowed when an unarchived chat is waiting or running.
Expand Down Expand Up @@ -183,6 +184,8 @@ stateDiagram-v2
R0 --> W: FinishTurn / queue empty
R0 --> E0: FinishError
R0 --> R1: SendMessage(queue)
R0 --> R1: SendMessage(steer)
R0 --> R0: PromoteQueuedBeforeStep

R1 --> R1: RecordGenerationAttempt
R1 --> R1: RecordRetryState
Expand All @@ -193,6 +196,9 @@ stateDiagram-v2
R1 --> R0: EditMessage
R1 --> E1: FinishError
R1 --> R1: SendMessage(queue)
R1 --> R1: SendMessage(steer)
R1 --> R0: PromoteQueuedBeforeStep / promoted last queued
R1 --> R1: PromoteQueuedBeforeStep / queue still non-empty
R1 --> R0: DeleteQueuedMessage / removed last queued
R1 --> R1: DeleteQueuedMessage / queue still non-empty
R1 --> I1: PromoteQueuedMessage
Expand All @@ -215,13 +221,15 @@ stateDiagram-v2
A0 --> R0: Interrupt
A0 --> R0: CancelRequiresAction
A0 --> A1: SendMessage(queue)
A0 --> A1: SendMessage(steer)
A0 --> R1: SendMessage(interrupt)
A0 --> R0: EditMessage

A1 --> R1: CompleteRequiresAction
A1 --> R1: Interrupt
A1 --> R1: CancelRequiresAction
A1 --> A1: SendMessage(queue)
A1 --> A1: SendMessage(steer)
A1 --> R1: SendMessage(interrupt)
A1 --> R0: EditMessage
A1 --> A0: DeleteQueuedMessage / removed last queued
Expand Down Expand Up @@ -493,7 +501,7 @@ For `busy_behavior=interrupt`, `SendMessage(m, interrupt)` supports:
- `A0 -> SendMessage(m, interrupt) -> R1`
- `A1 -> SendMessage(m, interrupt) -> R1`

When `SendMessage(m, interrupt)` lands in `I1`, the queued message is promoted later by `FinishInterruption(partial?)` after the interrupted suffix is finalized.
When `SendMessage(m, interrupt)` lands in `I1`, the queued message is promoted later by `FinishInterruption(partial?)` after the interrupted suffix is finalized. `FinishInterruption` promotes only the queue head, so if other queued messages were ahead of `m`, the chat worker promotes `m` with `PromoteQueuedBeforeStep` before its next LLM API call. When `SendMessage(m, interrupt)` lands in `R1` from `A0` or `A1`, `m` stays queued, and the chat worker promotes it the same way.

Other input states are not supported.

Expand Down
2 changes: 1 addition & 1 deletion coderd/x/chatd/chatd.go
Original file line number Diff line number Diff line change
Expand Up @@ -1439,7 +1439,7 @@ func (p *Server) SendMessage(
busyBehavior = database.ChatBusyBehaviorQueue
}
switch busyBehavior {
case database.ChatBusyBehaviorQueue, database.ChatBusyBehaviorInterrupt:
case database.ChatBusyBehaviorQueue, database.ChatBusyBehaviorSteer, database.ChatBusyBehaviorInterrupt:
Comment thread
ethanndickson marked this conversation as resolved.
Resolved
default:
return SendMessageResult{}, xerrors.Errorf("invalid busy behavior %q", opts.BusyBehavior)
}
Expand Down
171 changes: 171 additions & 0 deletions coderd/x/chatd/chatd_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2136,6 +2136,177 @@ func TestAutoPromoteQueuedMessagesPreservesPerTurnModelOrder(t *testing.T) {
require.Equal(t, []uuid.UUID{modelConfigA.ID, modelConfigB.ID, modelConfigC.ID}, userModelConfigIDs)
}

// TestActiveServer_SteerDelivery checks that steer and interrupt messages
// sent to a busy chat reach the next model call, after the queued
// messages that were sent before them.
func TestActiveServer_SteerDelivery(t *testing.T) {
t.Parallel()

t.Run("steer during a tool call", func(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitLong)
h := startSteerTestChat(ctx, t, "read_file")
h.send(ctx, t, "queued one", database.ChatBusyBehaviorQueue)
h.send(ctx, t, "steer message", database.ChatBusyBehaviorSteer)
h.send(ctx, t, "queued two", database.ChatBusyBehaviorQueue)
close(h.releaseTool)

calls := h.waitForCalls(ctx, t)
require.Len(t, calls, 3)
requireOpenAIMessageOrder(t, calls[1], "tool output", "queued one", "steer message")
require.False(t, openAIMessagesContain(calls[1], "queued two"))
require.True(t, openAIMessagesContain(calls[2], "queued two"))
})

t.Run("interrupt behind a queued message", func(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitLong)
h := startSteerTestChat(ctx, t, "read_file")
h.send(ctx, t, "queued one", database.ChatBusyBehaviorQueue)
h.send(ctx, t, "interrupt message", database.ChatBusyBehaviorInterrupt)

calls := h.waitForCalls(ctx, t)
require.Len(t, calls, 2)
requireOpenAIMessageOrder(t, calls[1], "queued one", "interrupt message")
})

t.Run("interrupt in requires action", func(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitLong)
h := startSteerTestChat(ctx, t, "my_dynamic_tool")
h.send(ctx, t, "interrupt message", database.ChatBusyBehaviorInterrupt)

calls := h.waitForCalls(ctx, t)
require.Len(t, calls, 2)
requireOpenAIMessageOrder(t, calls[1], "Tool execution interrupted by user message", "interrupt message")
})
}

type steerTestChat struct {
db database.Store
server *chatd.Server
chat database.Chat
modelID uuid.UUID
releaseTool chan struct{}

mu sync.Mutex
calls [][]chattest.OpenAIMessage
}

// startSteerTestChat starts a chat whose first model call requests
// firstTool and whose later model calls answer with text. It returns
// while the chat is busy: read_file blocks until releaseTool closes, and
// any other tool is a dynamic tool that leaves the chat in
// requires_action.
func startSteerTestChat(ctx context.Context, t *testing.T, firstTool string) *steerTestChat {
t.Helper()
db, ps := dbtestutil.NewDB(t)
h := &steerTestChat{db: db, releaseTool: make(chan struct{})}
dynamic := firstTool != "read_file"
firstCall := chattest.OpenAIToolCallChunk(firstTool, `{"path":"/tmp/steer.txt"}`)
openAIURL := chattest.NewOpenAI(t, func(req *chattest.OpenAIRequest) chattest.OpenAIResponse {
if !req.Stream {
return chattest.OpenAINonStreamingResponse("title")
}
h.mu.Lock()
h.calls = append(h.calls, slices.Clone(req.Messages))
first := len(h.calls) == 1
h.mu.Unlock()
if first {
return chattest.OpenAIStreamingResponse(firstCall)
}
return chattest.OpenAIStreamingResponse(chattest.OpenAITextChunks("done")...)
})

toolStarted := make(chan struct{})
mockConn := agentconnmock.NewMockAgentConn(gomock.NewController(t))
setupToolExecutionAgentConn(t, mockConn)
mockConn.EXPECT().ReadFileLines(gomock.Any(), "/tmp/steer.txt", int64(1), int64(0), gomock.Any()).
DoAndReturn(func(ctx context.Context, _ string, _, _ int64, _ workspacesdk.ReadFileLinesLimits) (workspacesdk.ReadFileLinesResponse, error) {
close(toolStarted)
select {
case <-h.releaseTool:
case <-ctx.Done():
return workspacesdk.ReadFileLinesResponse{}, ctx.Err()
}
return workspacesdk.ReadFileLinesResponse{Success: true, FileSize: 11, TotalLines: 1, LinesRead: 1, Content: "tool output"}, nil
}).MaxTimes(1)

user, org, model := seedChatDependenciesWithProvider(t, db, "openai-compat", openAIURL)
ws, dbAgent := seedWorkspaceWithAgent(t, db, user.ID)
h.server = newActiveTestServer(t, db, ps, func(cfg *chatd.Config) {
cfg.AIBridgeTransportFactory = chatAIGatewayTransportFactoryPointer(chattest.NewMockAIBridgeTransport(t, openAIURL))
cfg.AgentConn = func(context.Context, uuid.UUID) (workspacesdk.AgentConn, func(), error) {
return mockConn, func() {}, nil
}
})
opts := chatd.CreateOptions{
OrganizationID: org.ID,
OwnerID: user.ID,
WorkspaceID: uuid.NullUUID{UUID: ws.ID, Valid: true},
AgentID: uuid.NullUUID{UUID: dbAgent.ID, Valid: true},
Title: "steer",
ModelConfigID: model.ID,
InitialUserContent: []codersdk.ChatMessagePart{codersdk.ChatMessageText("start")},
}
if dynamic {
opts.DynamicTools = dynamicToolJSON(t, firstTool)
}
chat, err := h.server.CreateChat(ctx, opts)
require.NoError(t, err)
h.chat, h.modelID = chat, model.ID
if dynamic {
testutil.Eventually(ctx, t, func(ctx context.Context) bool {
got, err := db.GetChatByID(ctx, chat.ID)
return err == nil && got.Status == database.ChatStatusRequiresAction
}, testutil.IntervalFast)
} else {
testutil.TryReceive(ctx, t, toolStarted)
}
return h
}

func (h *steerTestChat) send(ctx context.Context, t *testing.T, text string, behavior database.ChatBusyBehavior) {
t.Helper()
result, err := h.server.SendMessage(ctx, chatd.SendMessageOptions{
ChatID: h.chat.ID,
CreatedBy: h.chat.OwnerID,
ModelConfigID: h.modelID,
Content: []codersdk.ChatMessagePart{codersdk.ChatMessageText(text)},
BusyBehavior: behavior,
})
require.NoError(t, err)
require.True(t, result.Queued)
require.Equal(t, behavior, result.QueuedMessage.BusyBehavior)
}

// waitForCalls waits until the chat is idle and returns the messages of
// every streamed model call.
func (h *steerTestChat) waitForCalls(ctx context.Context, t *testing.T) [][]chattest.OpenAIMessage {
t.Helper()
waitForChatStatus(ctx, t, h.db, h.chat.ID, database.ChatStatusWaiting)
h.mu.Lock()
defer h.mu.Unlock()
return slices.Clone(h.calls)
}

// requireOpenAIMessageOrder asserts that each text appears in messages
// after the previous text.
func requireOpenAIMessageOrder(t *testing.T, messages []chattest.OpenAIMessage, texts ...string) {
t.Helper()
contents := make([]string, 0, len(messages))
for _, msg := range messages {
contents = append(contents, msg.Content)
}
joined := strings.Join(contents, "\n")
rest := joined
for _, text := range texts {
idx := strings.Index(rest, text)
require.GreaterOrEqualf(t, idx, 0, "missing %q in order %q within %q", text, texts, joined)
rest = rest[idx+len(text):]
}
}

func TestEditMessageRejectsMissingMessage(t *testing.T) {
t.Parallel()

Expand Down
1 change: 1 addition & 0 deletions coderd/x/chatd/chatstate/export_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@ var AllExecutionTransitions = []Transition{
TransitionEnterRequiresAction,
TransitionFinishInterruption,
TransitionFinishTurn,
TransitionPromoteQueuedBeforeStep,
TransitionFinishError,
TransitionCancelRequiresAction,
TransitionReconcileInvalidState,
Expand Down
9 changes: 8 additions & 1 deletion coderd/x/chatd/chatstate/helpers_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,21 @@ func (r *recordingPubsub) ownershipPublishCount() int {
// ChatBusyBehaviorQueue. The chat must already be in a state that allows
// SendMessage (typically R0, R1, or I*).
func sendQueuedMessage(t *testing.T, f *testFixture, m *chatstate.ChatMachine, body string) chatstate.SendMessageResult {
t.Helper()
return sendMessageWithBehavior(t, f, m, body, database.ChatBusyBehaviorQueue)
}

// sendMessageWithBehavior sends one user message via SendMessage with
// the given busy behavior.
func sendMessageWithBehavior(t *testing.T, f *testFixture, m *chatstate.ChatMachine, body string, behavior database.ChatBusyBehavior) chatstate.SendMessageResult {
t.Helper()
ctx := testutil.Context(t, testutil.WaitShort)
var send chatstate.SendMessageResult
require.NoError(t, m.Update(ctx, func(tx *chatstate.Tx, store database.Store) error {
var err error
send, err = tx.SendMessage(chatstate.SendMessageInput{
Message: userTextMessage(body, f.User.ID, f.Model.ID),
BusyBehavior: database.ChatBusyBehaviorQueue,
BusyBehavior: behavior,
})
return err
}))
Expand Down
6 changes: 6 additions & 0 deletions coderd/x/chatd/chatstate/transition.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ const (
TransitionEnterRequiresAction Transition = "EnterRequiresAction"
TransitionFinishInterruption Transition = "FinishInterruption"
TransitionFinishTurn Transition = "FinishTurn"
TransitionPromoteQueuedBeforeStep Transition = "PromoteQueuedBeforeStep"
Comment thread
ethanndickson marked this conversation as resolved.
Resolved
TransitionFinishError Transition = "FinishError"
TransitionCancelRequiresAction Transition = "CancelRequiresAction"
TransitionReconcileInvalidState Transition = "ReconcileInvalidState"
Expand Down Expand Up @@ -81,6 +82,10 @@ var transitionMatrix = map[ExecutionState]map[Transition][]ExecutionState{
TransitionEnterRequiresAction: {StateA0},
TransitionFinishTurn: {StateW},
TransitionFinishError: {StateE0},
// R0 is allowed because a user can delete the queued rows
// after the generation loop decides to promote. The
// transition then writes nothing.
TransitionPromoteQueuedBeforeStep: {StateR0},
},
StateR1: {
TransitionSendMessage: {StateR1, StateI1},
Expand All @@ -94,6 +99,7 @@ var transitionMatrix = map[ExecutionState]map[Transition][]ExecutionState{
TransitionEnterRequiresAction: {StateA1},
TransitionFinishTurn: {StateR0, StateR1},
TransitionFinishError: {StateE1},
TransitionPromoteQueuedBeforeStep: {StateR0, StateR1},
},
StateI0: {
TransitionSendMessage: {StateI1},
Expand Down
Loading
Loading