Skip to content

refactor(aibridge): share request admission and shutdown middleware - #29348

Open
pawbana wants to merge 2 commits into
pawel/aigov-614-add-ai-gateway-reverse-proxy-experiment-and-stateless-proxy-1from
pawel/aigov-614-add-ai-gateway-reverse-proxy-experiment-and-stateless-proxy-2
Open

pawbana wants to merge 2 commits into
pawel/aigov-614-add-ai-gateway-reverse-proxy-experiment-and-stateless-proxy-1from
pawel/aigov-614-add-ai-gateway-reverse-proxy-experiment-and-stateless-proxy-2

Conversation

@pawbana

@pawbana pawbana commented Sep 15, 2026

Copy link
Copy Markdown
Contributor
  • Share request admission, context cancellation, and drain accounting through InflightGate middleware.
  • Migrate interception to the shared middleware while preserving request body limits.
  • Reject new requests during shutdown with 503 and cancel on deadline without waiting indefinitely for handlers that ignore cancellation.
  • Add admission and shutdown regression coverage.

Generated by Coder Agents on behalf of @pawbana.

@linear-code

linear-code Bot commented Sep 15, 2026

Copy link
Copy Markdown

AIGOV-614

@pawbana
pawbana added this pull request to stack #29350 September 15, 2026 13:02
@pawbana
pawbana force-pushed the pawel/aigov-614-add-ai-gateway-reverse-proxy-experiment-and-stateless-proxy-2 branch from 7ebb218 to 9b67d8e Compare September 15, 2026 13:36
@pawbana
pawbana force-pushed the pawel/aigov-614-add-ai-gateway-reverse-proxy-experiment-and-stateless-proxy-2 branch from 9b67d8e to 83f2261 Compare September 15, 2026 14:41
@pawbana
pawbana force-pushed the pawel/aigov-614-add-ai-gateway-reverse-proxy-experiment-and-stateless-proxy-2 branch from 83f2261 to 7bb3d13 Compare September 15, 2026 14:57
@pawbana
pawbana marked this pull request as ready for review September 15, 2026 15:58
@pawbana
pawbana requested a review from johnstcn September 15, 2026 15:58
Comment thread aibridge/bridge.go Outdated
opt(b)
}
b.handler = b.inflight.Middleware(func() {
_ = b.clock.Now("serve_admission") // Trap point for deterministic race tests.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can tell, this onAdmit function is only ever used for this purpose? If so, should we just inline this behaviour into the inflight middleware?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Removed onAdmit.

Comment thread aibridge/inflight.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants