Repository navigation
Conversation
f070c19 to
116c8ff
Compare
1cdb6f2 to
a9b1a1a
Compare
9c5c959 to
98ccbfd
Compare
b62dd7e to
09ec334
Compare
98ccbfd to
7dc2cf5
Compare
09ec334 to
5341ac5
Compare
e8d590a to
afb06f1
Compare
5521e7b to
208c1c7
Compare
ae07413 to
0d07390
Compare
| cancelFn context.CancelCauseFunc | ||
|
|
||
| // poolOptions configures the interception pool created at startup. | ||
| // poolOptions configures the interception pool and recording in both modes. |
There was a problem hiding this comment.
nit: "both" sounds vague, which modes exactly?
There was a problem hiding this comment.
Thanks, updated comment.
| _ = pool.Shutdown(context.Background()) | ||
| return s.Err() | ||
| } | ||
| s.backend.Store(&backend{pool: pool, keyPools: pool.KeyPools}) |
| require.NotNil(t, rec, "proxy mode initializes the recorder before providers load") | ||
|
|
||
| providers := []aibridge.Provider{aibridge.NewOpenAIProvider(config.OpenAI{})} | ||
| for range 2 { |
There was a problem hiding this comment.
nit: worth adding a comment on why 2?
There was a problem hiding this comment.
To be honest it is not necessarily needed. Pass 1 checks nil to router, and pass 2 checks router to a new router. It is more defensive check that even router is definitely changed:
require.NotSame(t, previous, server.backend.Load().proxyRouter)
regardless of what was store there before but recorder stays the same.
| Passthrough: []string{"/v1/models"}, | ||
| URL: upstream.URL, | ||
| Bridged: []string{"/v1/chat/completions", "/v1/models/bridged"}, | ||
| Passthrough: []string{"/v1/models", "/v1/models/", "/v1/tokens"}, |
There was a problem hiding this comment.
nit: do we need both /v1/models and /v1/models/ (with slash)?
There was a problem hiding this comment.
/models/ covers whole /models/* sub tree while/models cover exact /models path.
Changes path names to better reflect their meaning, made test cases even more extensive.
|
|
||
| func (*forwardingHandler) forward(w http.ResponseWriter, r *http.Request) error { | ||
| // TODO: forward only when lifecycle recording is connected. | ||
| http.NotFound(w, r) |
| } | ||
|
|
||
| func (*forwardingHandler) finishForwarding(context.Context, *recorder.InterceptionRecord, error) { | ||
| // TODO: finalize the outcome and lifecycle records, including breaker rejection. |
There was a problem hiding this comment.
this is just cleanup doesn't return anything.
| return nil, err | ||
| } | ||
|
|
||
| snapshot := slices.Clone(providers) |
There was a problem hiding this comment.
do need to clone all providers not just ones we need?
There was a problem hiding this comment.
All providers are needed. Router holds all providers, mounts newForwardingHandler per provider.
Clone in general is a bit unnecessary, it is shallow copy so it only protects from slice manipulation, which never happens, it doesn't protect from modifying provider instances directly that is in slice, which never happens even more.
In theory if some code would change KeyPool for the same provider instance hold in that slice then it would change the Router.KeyPools() return value but this is very unlikely scenario.
The providers argument slice is created by initialization / reloader and consumed only here in aibridged.Server.
| } | ||
|
|
||
| bridged := newForwardingHandler(prov, logger, m, tracer, inflight, rec) | ||
| for _, path := range prov.BridgedRoutes() { |
There was a problem hiding this comment.
nit: would it be useful debug log all routes? (same for passthrough)
1a3d015 to
df88311
Compare
…bug logs for registered paths
89bd18b to
74bc42d
Compare
74bc42d to
60b2e22
Compare
Adds
forwardingHandlerfor "bridged" routes in proxy mode. Wires provider snapshots to a shared mux and recorder, tracing, circuit breakers, and the server-owned inflight gate.Construction lifecycle
Server, created byaibridged.Newduring embedded or standalone gateway startup:inflightonce in New. The same gate tracks requests across all router snapshots.recorderonce ininitializeBackendwhen proxy mode is selected. All proxy handlers reuse it. Interception mode is unchanged:CachedBridgePool.Acquirecreates a recorder on a cache miss, cache hits reuse the bridge and its recorder.Serverholds theRouterthroughbackend.proxyRouter:Server.ReplaceProvidersthroughproxy.NewRouter.ReplaceProviderson the initial provider load and each subsequent reload.For each enabled provider,
NewRoutercreates:aibridge.NewPassthroughHandler, mounted on the provider’s passthrough routes with provider-prefix stripping and shared inflight middleware.forwardingHandlermounted on every entry in that provider’sBridgedRoutes(). EachforwardingHandlerholds:Router.ServeHTTPlimits request-body reads before dispatching through the shared mux.Request processing
forwardingHandler.ServeHTTPruns these stages:inflight.Admitandinflight.RequestContext: reject new requests with 503 after shutdown begins and link admitted requests to forced shutdown. The handler also starts aProxytrace span.checkRequest: rejects WebSocket upgrades and declared oversized bodies, validates Agent Firewall headers. Actor, recorder, and credential validation and interception record metadata population remain unimplemented.breaker.Execute: apply the provider's circuit breaker for the matched route, rejecting an open circuit before outbound preparation or forwarding.prepareForwarding: clone the request. Provider-header preparation and a replayable request body will be added here.forward: returns the 404 stub for now. This stage will perform forwarding to upstream once lifecycle recording is connected.finishForwarding: run through a defer after the breaker returns, including when it rejects the request. Currently a no-op, it will finalize the outcome and record interception ended.Deferred cleanup ends the trace span, cancels the derived request context, and releases inflight admission.
Generated by Coder Agents on behalf of @pawbana.