Skip to content

fix(aibridge/intercept): close upstream response bodies in Responses interceptors - #30287

Closed
ibetitsmike wants to merge 3 commits into
mike/golangci-lint-v2.14from
mike/aibridge-close-upstream-body
Closed

ibetitsmike wants to merge 3 commits into
mike/golangci-lint-v2.14from
mike/aibridge-close-upstream-body

Conversation

@ibetitsmike

@ibetitsmike ibetitsmike commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

responseCopier.copyMiddleware hands the OpenAI SDK an io.NopCloser over a TeeReader of the upstream body, so readAll can drain the body after the SDK closes its view. Nothing closed the real body, so every Responses API interception kept its upstream connection until the request context was canceled.

The copier now keeps the real body and closes it when the interception returns and before an agentic or failover iteration replaces the copier. SDK retries and key failover reuse one copier, so each new attempt first closes the previous body and clears what the copier recorded about it. Only the final attempt's response is forwarded, and a final attempt without a response gets the same 500 the interceptor already sends when the first attempt gets no response.

Clearing the buffer also fixes a bug already on main: after blocking key failover, the client received the failed key's error body followed by the successful response.

Stack context: second layer of the Go 1.27 upgrade. Context cancellation hides this leak on Go 1.26. Go 1.27.1 has net/http regressions that stop hiding it (golang/go#81411 and three related fixes, all backported to release-branch.go1.27 for Go 1.27.2), and aibridge/internal/integrationtest fails its goleak check. The fix is correct on any Go version, so it lands before the version bump.

Testing

  • go test ./aibridge/... and go test -race ./aibridge/intercept/... pass on Go 1.26.5, and on Go 1.27.1 with chore: bump Go to 1.27.1 #30288 on top.
  • On Go 1.27.1, go test ./aibridge/internal/integrationtest fails goleak without this change and passes with it.
  • TestResponseCopierReleasesUpstreamBeforeRetry, TestResponseCopierDiscardsPreviousAttempt, and the body check added to TestInterception_KeyFailover each fail without the part of the change they cover.

Review

  • Codex round 1: a retry kept the previous upstream connection until the retry finished. Fixed in e1b46fa.
  • Codex round 2: a retry that got no response made the interceptor read the closed previous body and send an empty 200. Fixed in 35a4171.
  • Codex round 3: clean on 35a4171.
  • Coder Agents UAT on chore: bump Go to 1.27.1 #30288's head (5aaeb6f, Go 1.27.1): through the real server, 500 then 200, 500 then dropped connections, key failover after a 429, and key failover into a dropped connection each ran 20 times blocking and 20 times streaming. Every success was the final attempt's body alone, every failure was a non-empty 500, and goroutines and upstream connections returned to baseline.

Xum opened this PR on behalf of Mike.

copyMiddleware hands the OpenAI SDK an io.NopCloser over a TeeReader
of the upstream body, so readAll can still drain the body after the
SDK closes its view. Nothing closed the real body, so each Responses
API interception kept its upstream connection until the request
context was canceled.

Keep the real body on the copier and close it when the interception
returns, before an agentic or failover iteration replaces the copier,
and when an SDK retry passes a new response through the middleware.

Context cancellation hides the leak on Go 1.26. Go 1.27.1 net/http
regressions, fixed on release-branch.go1.27 for Go 1.27.2, stop
hiding it, and the aibridge integration tests fail their goleak check.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Requested by Xum on behalf of @ibetitsmike.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-02T16:54:51.255225Z 35a4171 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 33bf54e203

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread aibridge/intercept/responses/base.go Resolved
…re a retry

SDK retries and centralized key failover run copyMiddleware again on
the same copier, but it closed the previous upstream body only after
the next attempt returned. The SDK cannot close that body itself
through the NopCloser, so each retry held the earlier connection for
the whole backoff and attempt.

Close the previous body before calling next. The new test sends a
retryable 500 through the real SDK and fails when the first upstream
connection is still held while the retry is in flight.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Requested by Xum on behalf of @ibetitsmike.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e1b46fadea

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread aibridge/intercept/responses/base.go Resolved
SDK retries and centralized-key failover reuse one responseCopier.
Since the previous commit closes the earlier attempt's body before the
next attempt, a retry that got no response left the copier pointing at
that closed body. The interceptors then failed to read it, wrote
nothing, and the client got an empty 200.

copyMiddleware now clears the received flag and the buffer when an
attempt starts, so a final attempt without a response takes the
interceptors' custom-error path.

Clearing the buffer also fixes blocking key failover, which forwarded
the failed key's error body concatenated with the successful response.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Requested by Xum on behalf of @ibetitsmike.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: 35a417163d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ibetitsmike ibetitsmike changed the title fix(aibridge/intercept/responses): close upstream response bodies fix(aibridge/intercept): close upstream response bodies in Responses interceptors Oct 2, 2026
@linear-code

linear-code Bot commented Oct 8, 2026

Copy link
Copy Markdown

CODAGT-1116

@github-actions github-actions Bot added the stale This issue is like stale bread. label Nov 11, 2026
@github-actions github-actions Bot closed this Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale This issue is like stale bread.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant