Skip to content

aibridge: key failover transport cannot report the key used or the key pool error #31964

Closed as not planned

Description

@pawbana

Problem

keypool.NewKeyFailoverTransport picks keys from the pool internally. It
exposes neither the key that served the request nor the key pool error when
every key fails. It turns pool exhaustion into the provider's error
response (BuildKeyPoolResponse). Callers cannot get either from the transport
alone.

The transport cannot track key use through credential.CentralizedPool records the key in use (NextKey), but
credential imports keypool, so the transport in keypool cannot use it
without an import cycle.

Bridged interceptors avoid this by running their own key loop with
CentralizedPool.NextKey: cred.Hint() names the key in use and the
*keypool.Error is returned. The AI Gateway proxy uses the transport, so it
captures the key and the error by wrapping the provider's InjectAuthKey and
BuildKeyPoolResponse hooks in newProxy (aibridge/x/proxy/handler.go,
#30226).

Suggested responsibility split

  • credential: the kind of credential a request uses, the key in use, and the
    final key pool error.
  • Key failover transport: failover over HTTP for the request's credential.
  • keypool: internals (keys, states, walker, marking, metrics), used only by
    credential and the transport.

After the split, provider, intercept, and x/proxy would use the
credential or transport instead of keypool types. Today they use
keypool.KeyFailoverConfig, Pool, Key, Walker, and Error directly.
Whether keypool can become fully internal needs checking: pool construction
(cli/aibridged.go, aibridge/config), pool state metrics
(coderd/aibridged, the x/proxy router), and error categorization and
rendering (aibridge/interceptionerror, aibridge/intercept) also use
keypool types.


Generated by Coder Agents on behalf of @pawbana.

Activity

  1. linear-code commented on Nov 10, 2026

    @linear-code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions