Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions coderd/x/chatd/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -1247,7 +1247,7 @@ Request preparation reads the transport from the model instead of recomputing it

- Provider option conversion chooses between the Responses and Chat Completions option structs. The SDK type-asserts the concrete struct, so a mismatch discards every OpenAI provider option rather than failing.
- Reasoning effort injection creates those option structs when a config has no OpenAI options of its own.
- File part conversion (`Model.AcceptsFilePartMediaType`) gates attachments, because the Responses API natively accepts only images and PDFs. A mismatch here drops text attachments.
- File part conversion (`Model.AcceptsFilePartMediaType`) gates user attachments and tool-result media, because the Responses API natively accepts only images and PDFs. A mismatch here loses audio, the only media type that the two transports treat differently.

The first two happen together in `chatprovider.ProviderOptionsForCall`, the only entry point in `chatprovider` that builds provider options for a call; it delegates transport-aware OpenAI conversion to `chatopenai.ProviderOptionsFromChatConfig`. Config conversion and effort injection cannot pick different option types because one function owns both.

Expand Down Expand Up @@ -1305,7 +1305,7 @@ Details that follow from the override:
Failure to read the override row or load provider credentials stops preparation.
A usable override that fails at use (route or client construction, provider call failure) fails the generation visibly through the normal error path; there is no silent fallback.
The override model client is constructed inside the compact generation action, not at prepare time, so a broken override cannot fail turns that finish without compacting (including turns over the threshold whose last assistant step already completed).
- Prompt safety: the prompt is built and sanitized for the chat model, so when the override points at a different provider the compaction copy of the prompt is re-sanitized: provider-executed tool history is flattened into plain text parts (keeping its content while dropping the provider-specific wire shape), file parts the compaction model rejects are replaced with text placeholders, and Anthropic provider-tool sanitization is re-run for the compaction provider. The assistant generation prompt is never mutated.
- Prompt safety: the prompt is built and sanitized for the chat model, so when the override points at a different provider the compaction copy of the prompt is re-sanitized: provider-executed tool history is flattened into plain text parts (keeping its content while dropping the provider-specific wire shape), file parts and tool-result media the compaction model rejects are replaced with text placeholders, and Anthropic provider-tool sanitization is re-run for the compaction provider. The assistant generation prompt is never mutated.
- Observability: compaction metrics and chat debug runs record the provider and model that actually generated the summary. This includes the "still over limit" terminal error, which is recorded before the override client is built: prepare-time resolution keeps the override's provider/model identity so that error lands on the same metric series as the compact action's own events.

#### Interrupt goroutine
Expand Down
6 changes: 3 additions & 3 deletions coderd/x/chatd/chatd.go
Original file line number Diff line number Diff line change
Expand Up @@ -1624,14 +1624,14 @@ func (p *Server) PublishDiffStatusChange(ctx context.Context, chatID uuid.UUID,
// the prompt forward). A future change should skip the file with a
// user-facing warning, but that requires altering the FileResolver
// contract.
func (p *Server) chatFileResolver(provider string) chatprompt.FileResolver {
func (p *Server) chatFileResolver(model chatprovider.Model) chatprompt.FileResolver {
imageCap, hasImageCap := model.InlineImageCapBytes()
normalizedProvider := model.RouteProvider()
return func(ctx context.Context, ids []uuid.UUID) (map[uuid.UUID]chatprompt.FileData, error) {
files, err := p.db.GetChatFilesByIDs(ctx, ids)
if err != nil {
return nil, err
}
imageCap, hasImageCap := chatprovider.InlineImageCapBytes(provider)
normalizedProvider := chatprovider.NormalizeProvider(provider)
result := make(map[uuid.UUID]chatprompt.FileData, len(files))
for _, f := range files {
if hasImageCap &&
Expand Down
61 changes: 47 additions & 14 deletions coderd/x/chatd/chatfile_internal_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,23 @@ import (
"github.com/coder/coder/v2/coderd/database/dbmock"
"github.com/coder/coder/v2/coderd/x/chatd/chaterror"
"github.com/coder/coder/v2/coderd/x/chatd/chatprovider"
"github.com/coder/coder/v2/coderd/x/chatd/chattest"
"github.com/coder/coder/v2/codersdk"
)

// inlineImageCapFor returns the provider's inline image cap. Fails
// the test if the provider has no documented cap.
func inlineImageCapFor(t *testing.T, provider string) int {
// routedModel returns a model whose requests reach the configured
// provider through the transport client.
func routedModel(transport, provider string) chatprovider.Model {
return chatprovider.NewModel(&chattest.FakeModel{ProviderName: transport, ModelName: "model"}, nil).
WithConfiguredProvider(provider)
}

// inlineImageCapFor returns the route's inline image cap. Fails the
// test if the route has no documented cap.
func inlineImageCapFor(t *testing.T, model chatprovider.Model) int {
t.Helper()
imageCap, ok := chatprovider.InlineImageCapBytes(provider)
require.Truef(t, ok, "expected provider %q to have an inline image cap", provider)
imageCap, ok := model.InlineImageCapBytes()
require.Truef(t, ok, "expected route %q/%q to have an inline image cap", model.RouteProvider(), model.Provider())
return imageCap
}

Expand All @@ -34,10 +42,11 @@ func TestChatFileResolver_RejectsOversizedImages(t *testing.T) {
t.Parallel()

// Computed so the table tracks any future cap retune.
anthropicCap := inlineImageCapFor(t, "anthropic")
anthropicCap := inlineImageCapFor(t, routedModel("anthropic", "anthropic"))

tests := []struct {
name string
transport string
provider string
mimetype string
size int
Expand All @@ -46,6 +55,7 @@ func TestChatFileResolver_RejectsOversizedImages(t *testing.T) {
}{
{
name: "OversizedAnthropicPNG_Rejected",
transport: "anthropic",
provider: "anthropic",
mimetype: "image/png",
size: anthropicCap + 1,
Expand All @@ -54,6 +64,7 @@ func TestChatFileResolver_RejectsOversizedImages(t *testing.T) {
},
{
name: "OversizedAnthropicJPEG_Rejected",
transport: "anthropic",
provider: "anthropic",
mimetype: "image/jpeg",
size: anthropicCap + 1024,
Expand All @@ -65,6 +76,7 @@ func TestChatFileResolver_RejectsOversizedImages(t *testing.T) {
// Anthropic's docs say "5 MB maximum" without
// specifying inclusivity, so reject strictly.
name: "AtLimitAnthropicImage_Rejected",
transport: "anthropic",
provider: "anthropic",
mimetype: "image/png",
size: anthropicCap,
Expand All @@ -73,6 +85,7 @@ func TestChatFileResolver_RejectsOversizedImages(t *testing.T) {
},
{
name: "JustUnderLimitAnthropicImage_Accepted",
transport: "anthropic",
provider: "anthropic",
mimetype: "image/png",
size: anthropicCap - 1,
Expand All @@ -81,23 +94,37 @@ func TestChatFileResolver_RejectsOversizedImages(t *testing.T) {
},
{
name: "UndersizedAnthropicImage_Accepted",
transport: "anthropic",
provider: "anthropic",
mimetype: "image/png",
size: 1024,
expectReject: false,
expectProviderID: "anthropic",
},
{
// Bedrock reuses Anthropic's cap.
// Bedrock Claude models use the Anthropic transport and its cap.
name: "OversizedBedrockPNG_Rejected",
transport: "anthropic",
provider: "bedrock",
mimetype: "image/png",
size: anthropicCap + 1,
expectReject: true,
expectProviderID: "bedrock",
},
{
// Other Bedrock models use the OpenAI transport, which has
// no inline image cap.
name: "OversizedBedrockOpenAIPNG_Accepted",
transport: "openai",
provider: "bedrock",
mimetype: "image/png",
size: anthropicCap + 1,
expectReject: false,
expectProviderID: "bedrock",
},
{
name: "OversizedOpenAIImage_Accepted",
transport: "openai",
provider: "openai",
mimetype: "image/png",
size: anthropicCap + 1,
Expand All @@ -106,6 +133,7 @@ func TestChatFileResolver_RejectsOversizedImages(t *testing.T) {
},
{
name: "OversizedAnthropicText_Accepted",
transport: "anthropic",
provider: "anthropic",
mimetype: "text/plain",
size: anthropicCap + 1,
Expand All @@ -114,6 +142,7 @@ func TestChatFileResolver_RejectsOversizedImages(t *testing.T) {
},
{
name: "ProviderMixedCase_Rejected",
transport: "anthropic",
provider: "Anthropic",
mimetype: "image/png",
size: anthropicCap + 1,
Expand All @@ -122,6 +151,7 @@ func TestChatFileResolver_RejectsOversizedImages(t *testing.T) {
},
{
name: "ProviderAllCaps_Rejected",
transport: "anthropic",
provider: "ANTHROPIC",
mimetype: "image/png",
size: anthropicCap + 1,
Expand All @@ -130,6 +160,7 @@ func TestChatFileResolver_RejectsOversizedImages(t *testing.T) {
},
{
name: "ProviderPaddedWhitespace_Rejected",
transport: "anthropic",
provider: " anthropic ",
mimetype: "image/png",
size: anthropicCap + 1,
Expand Down Expand Up @@ -170,7 +201,8 @@ func TestChatFileResolver_RejectsOversizedImages(t *testing.T) {
Return([]database.ChatFile{row}, nil).
Times(1)

resolver := server.chatFileResolver(tc.provider)
model := routedModel(tc.transport, tc.provider)
resolver := server.chatFileResolver(model)
got, err := resolver(ctx, []uuid.UUID{fileID})

if tc.expectReject {
Expand All @@ -187,7 +219,7 @@ func TestChatFileResolver_RejectsOversizedImages(t *testing.T) {
// the wrapped developer error.
displayName := chatprovider.ProviderDisplayName(tc.expectProviderID)
require.Contains(t, classified.Message, displayName)
imageCap := inlineImageCapFor(t, tc.expectProviderID)
imageCap := inlineImageCapFor(t, model)
//nolint:gosec // imageCap is a small positive constant defined in chatprovider.
require.Contains(t, classified.Message, humanize.IBytes(uint64(imageCap)))
require.NotContains(
Expand Down Expand Up @@ -218,7 +250,8 @@ func TestChatFileResolver_MultiFileFailsFastOnFirstOversized(t *testing.T) {
db := dbmock.NewMockStore(ctrl)
server := &Server{db: db}

anthropicCap := inlineImageCapFor(t, "anthropic")
anthropicModel := routedModel("anthropic", "anthropic")
anthropicCap := inlineImageCapFor(t, anthropicModel)
// Shared buffer; ok files take small prefixes.
buf := make([]byte, anthropicCap+1)
okFileA := database.ChatFile{
Expand Down Expand Up @@ -246,7 +279,7 @@ func TestChatFileResolver_MultiFileFailsFastOnFirstOversized(t *testing.T) {
Return([]database.ChatFile{okFileA, oversized, okFileB}, nil).
Times(1)

resolver := server.chatFileResolver("anthropic")
resolver := server.chatFileResolver(anthropicModel)
got, err := resolver(ctx, ids)
require.Error(t, err)
require.Nil(t, got)
Expand Down Expand Up @@ -274,7 +307,7 @@ func TestChatFileResolver_PropagatesDBError(t *testing.T) {
Return(nil, sentinel).
Times(1)

resolver := server.chatFileResolver("anthropic")
resolver := server.chatFileResolver(routedModel("anthropic", "anthropic"))
got, err := resolver(ctx, []uuid.UUID{fileID})
require.ErrorIs(t, err, sentinel)
require.Nil(t, got)
Expand All @@ -294,7 +327,7 @@ func TestChatFileResolver_UnknownProviderSkipsCapCheck(t *testing.T) {
// Exactly 1 byte above the Anthropic cap is enough to prove
// the backstop is skipped for uncapped providers; no need to
// allocate tens of MiB in CI.
overAnyCap := inlineImageCapFor(t, "anthropic") + 1
overAnyCap := inlineImageCapFor(t, routedModel("anthropic", "anthropic")) + 1
row := database.ChatFile{
ID: fileID,
Name: "huge.png",
Expand All @@ -306,7 +339,7 @@ func TestChatFileResolver_UnknownProviderSkipsCapCheck(t *testing.T) {
Return([]database.ChatFile{row}, nil).
Times(1)

resolver := server.chatFileResolver("openrouter")
resolver := server.chatFileResolver(routedModel("openai-compat", "openrouter"))
got, err := resolver(ctx, []uuid.UUID{fileID})
require.NoError(t, err)
require.Contains(t, got, fileID)
Expand Down
Loading
Loading