Skip to content

feat(aibridge/x/proxy/responsesws): add Responses WebSocket session core - #30198

Closed
ThomasK33 wants to merge 8 commits into
tk/ai-gateway-extract-responsesfrom
tk/ai-gateway-responses-ws-core
Closed

ThomasK33 wants to merge 8 commits into
tk/ai-gateway-extract-responsesfrom
tk/ai-gateway-responses-ws-core

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

Adds aibridge/x/proxy/responsesws, a transport-agnostic session core for OpenAI Responses WebSocket mode. It is the first gateway piece of Responses WebSocket support (context: AIGOV-410). Nothing imports it yet; the WebSocket endpoint and per-user gating come in later PRs in this stack.

The session relays frames between a client and an upstream connection and records them like the HTTP path does:

  • Every frame is forwarded byte for byte in both directions, including unknown client event types. Gateway MCP tools are never injected.
  • Each client response.create runs an admission callback (budget and rate checks go here) before it is forwarded. A refusal returns a Responses error event with the request's stream_id, nothing is sent upstream, and the session stays open.
  • Each admitted response.create gets one interception, with the model and, for tool-result rounds, the function_call_output call ID so threads link like on the HTTP path.
  • Every response the provider announces or finishes is recorded on exactly one interception of the session. A response.created binds to the oldest create on its lane whose upstream write succeeded, else it opens its own interception; a terminal event for an unknown response also opens one, so terminal usage is never dropped.
  • response.steer input is recorded as a prompt on the interception that owns previous_response_id. Automatic steer continuations are not promised to share that interception; they bind like any other response and are grouped by client session.
  • Token usage uses the same math as the HTTP Responses interceptor, including ProviderModel. It is recorded from response.completed, response.incomplete, and response.failed when usage is present; the HTTP streaming path only reads response.completed.
  • Every blocking step in Send ends with either the caller or the session, and session shutdown records all open interceptions under one shared cleanup deadline.
  • The Go API (NewSession, Send, Recv, Close) is what the WebSocket endpoint and Coder Agents will use. Upstream is a small MessageConn interface.

Follow-ups, tracked in this stack: the next layer replaces the built-in observer with the shared aibridge/extract Responses extractor and gives the interception end record its own bounded context before custom observers are supported.

Must land before the WebSocket endpoint (G2) merges, tracked in AIGOV-697 as a separate layer on top of this PR:

  • The reader records usage synchronously before relaying each frame, so a slow recorder can delay frames by up to 5s and stall every stream on the connection. The extractor swap makes recording asynchronous or relays first.
  • An upstream error that arrives while a create's write is still in flight is dropped, and that interception is recorded as successful. G2 keeps the error and ends the interception with it, with a test that fires the error during a blocked write.

Validation: go test -race ./aibridge/x/proxy/... (including a 300-scenario seeded invariant test that every terminal response's usage is recorded exactly once), golangci-lint (0 issues), and make pre-commit pass. The local pre-push test run failed only in packages this PR does not touch (load timeouts on a saturated host and a known TestGetModulesArchive file-mode issue), so pushes skipped the local hook and CI is the test gate.


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-30T19:27:34.696763Z d8b65a0 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 03e1aae625

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread aibridge/x/proxy/responsesws/session.go Resolved
Comment thread aibridge/x/proxy/responsesws/session.go Resolved
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 395ec44277

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread aibridge/x/proxy/responsesws/session.go Resolved
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bc46ec6165

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread aibridge/x/proxy/responsesws/session.go Resolved
Comment thread aibridge/x/proxy/responsesws/session.go Resolved
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7ce05fb703

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread aibridge/x/proxy/responsesws/session.go Resolved
Comment thread aibridge/x/proxy/responsesws/session.go Resolved
Comment thread aibridge/x/proxy/responsesws/session.go Resolved
Comment thread aibridge/x/proxy/responsesws/session.go Resolved
Comment thread aibridge/x/proxy/responsesws/session.go Resolved
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d8b65a09b3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread aibridge/x/proxy/responsesws/session.go Resolved
Comment thread aibridge/x/proxy/responsesws/session.go Resolved
@ThomasK33
ThomasK33 changed the base branch from main to tk/ai-gateway-extract-responses September 30, 2026 21:50
@ThomasK33
ThomasK33 added this pull request to stack #30212 September 30, 2026 21:51
@ThomasK33

Copy link
Copy Markdown
Member Author

This PR now stacks on #30211 (E1), which stacks on #30210 (X1): stack #30212. Head 9de05e5010 merges E1 head a83a8aafc9, and the diff against E1 is this PR's 5 files only.

No pull_request CI run started for this head, so I dispatched ci manually: https://github.com/coder/coder/actions/runs/36782199420 (headSha 9de05e5010e7f7c2f029c96cafce9d954ae20f76). It will not show in this PR's check list.


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

@linear-code

linear-code Bot commented Sep 30, 2026

Copy link
Copy Markdown

AIGOV-697

@ThomasK33

Copy link
Copy Markdown
Member Author

Head 8503c189b3 merges E1's final head 6fd7d1957c (Codex clean, CI green on #30211). The diff against E1 is still this PR's 5 files.

No pull_request CI run started for this head, so I dispatched ci manually: https://github.com/coder/coder/actions/runs/36787039602 (headSha 8503c189b36f53e352bff0d24675275d01c8791e). It will not show in this PR's check list.


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

Add a transport-agnostic session core for OpenAI Responses WebSocket mode.
A Session relays frames between a client and an upstream MessageConn byte
for byte, and records AI Gateway interceptions from the event stream. It has
no WebSocket framing, dialing, or HTTP handler; the upgrade handler and
chatd can drive it through Send, Recv, and Close.

Recording model:
- Each admitted client response.create starts one interception, recorded
  synchronously before the frame is forwarded. An injected admission
  callback can refuse a create; refused or unrecordable creates are not
  forwarded and the client gets a synthesized error event instead.
- Server responses bind to interceptions per stream_id lane in FIFO order.
  A response.created that no create or steer explains opens its own
  interception.
- A response.steer records its input as a prompt on the interception that
  owns the steered response. A response that ends incomplete with reason
  "steered" keeps its interception open for the automatic continuation.
- Interceptions end on completed, failed, or incomplete responses, on
  lane-scoped error events, and on session end.
- A small Observer interface receives bound frames. The default observer
  records prompts and token usage with the HTTP Responses interceptor math.

This stacks on #29727 and only adds files under aibridge/x/proxy/responsesws.


---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
Address review round 1 on the Responses WebSocket session core:

- A pending client response.create now wins over a steered continuation
  when binding response.created. response.steer.pending and
  response.steer.failed drop the waiting continuation, and so does a client
  create whose previous_response_id targets that response.
- error events are request-scoped: they end only the oldest pending create
  on their lane and never the in-flight response.
- A response that completes normally with an accepted, unapplied steer keeps
  its interception open for the automatic continuation.
- Usage is recorded from every terminal response that carries it, including
  incomplete and failed responses; this is documented and tested.
- Recorder calls use a context detached from cancellation with the async
  timeout. The steer prompt is recorded only after the frame is written.
  The reader copies frames before buffering. Terminal events for unknown
  response IDs are logged and ignored. Synthesized errors omit param.
- Add tests for each fix and a concurrent Send, Recv, and Close test that
  fills every buffer.


---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
…on and credential types

Port the session core onto main's aibridge APIs with no behavior change:

- Provider embeds interceptionerror.ErrorCategorizer, the interface name
  on main.
- Options.CredentialKind is a credential.Kind, the type main's recorder
  uses for InterceptionRecord.CredentialKind.
- recordEnded calls Categorize(provider, err). The old code passed a zero
  fallback status, so no status information is lost: upstream errors are
  still *intercept.ResponseError values carrying their HTTP status, and
  the OpenAI provider's CategorizeError maps that status through
  recorder.ErrorTypeFromStatus.
- Tests pass the email argument that aibcontext.AsActor takes on main and
  use credential.KindCentralized.


---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
- Set CreatedAt on prompt and token usage records. Production recorders
  are wrapped by recorder.WithValidation, which refuses records with an
  unset timestamp, so usage and prompts were silently dropped. Tests now
  record through a ValidatingRecorder whose refusals fail the test.
- Record the upstream response model as ProviderModel on token usage, as
  the HTTP Responses interceptor does.
- Run admission and the interception start on a context canceled by the
  caller or a session close, so Close no longer leaves Send blocked in
  Admit. A create admitted or refused after the session closed returns
  ErrClosed without recording or forwarding anything.
- Delete per-lane queue entries when they become empty, so a client using
  a fresh stream_id per request does not grow the session's maps.


---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
… exits

When upstream Read ended with EOF or a network error, the reader marked
the session closed but never canceled the session context. Work bound to
it did not stop: a Send waiting in admission or in the interception start
stayed blocked. The reader now cancels the session with the read error as
cause before ending open interceptions, so Close, cancellation of the
session's parent context, and upstream EOF or errors all cancel it. After
Close the reader's cancel is a no-op, so Close's cause is kept. Recv still
reports the read error, or ErrClosed after Close.


---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
…and session

Every blocking step of Send now runs under one operation context that ends
when the caller's context or the session ends: admission, recording the
interception start, queueing a synthesized error event, and the upstream
write on both the plain-frame and response.create paths. Previously the
plain write and the create write watched only the caller's context, so a
Send blocked in Write hung after the upstream reader exited.

Send documents how a failed or cut-short step is classified: a session end
returns ErrClosed and records or synthesizes nothing more; a caller
cancellation returns the caller's error and synthesizes nothing; only a
step's own failure queues an error event. Admission returning the operation
context's error after the caller canceled is therefore no longer relayed as
a 403 refusal. An interception Send started is always ended, with the
session's cause, the caller's error, or the write error. Error events are
queued without racing an ended operation, so an event is queued exactly
when Send returns nil after a failed step.


---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
… continuation handling

Make usage loss impossible by construction and drop the speculative steer
continuation grouping:

- A create becomes bindable only after its upstream write succeeded. The
  transition happens under the session lock, so a response.created handled
  while the write is in flight never binds to it and opens its own
  interception. A failed write therefore never removes a mapping a
  response relied on. A create whose write raced such an unattributed
  response.created or error event on its lane ends without a response,
  since that event may have been its answer; binding it later would shift
  every following response on the lane.
- A terminal event for a response no interception owns records its usage
  on a new interception instead of being dropped. A response mapping is
  removed only after its terminal usage was recorded or the session ended.
- Automatic continuations are no longer grouped with the steered
  interception. Steer frames are still relayed and steer input is still
  recorded as a prompt on the interception owning the steered response;
  continuations bind like any other response.
- Record CorrelatingToolCallID from a trailing function_call_output input
  item, as the HTTP Responses interceptor does.
- Session shutdown ends all open interceptions within one shared cleanup
  deadline, so Close is bounded even when the recorder blocks.
- Note that the observer shares the end record's context, to revisit before
  custom observers are supported.

A seeded randomized test drives interleavings of creates, steers, write
outcomes, server events, and session end, and checks that every terminal
response's usage is recorded exactly once, every interception is ended, and
no response is recorded on a create not yet written when it was announced.


---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
…ng ended interceptions

When upstream ends during a create's write, Send can return ErrClosed
before the reader records the session's end, so the assertion raced.
Close waits for the reader, which makes the check deterministic.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33
ThomasK33 force-pushed the tk/ai-gateway-responses-ws-core branch from 8503c18 to 2f0c01f Compare October 2, 2026 08:16
@github-actions github-actions Bot added the stale This issue is like stale bread. label Nov 7, 2026
@github-actions github-actions Bot closed this Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale This issue is like stale bread.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant