Skip to content

feat: model-level provider priority lists with runtime fallback#23649

Closed
ibetitsmike wants to merge 25 commits into
mike/multi-provider-configs-stage1from
mike/multi-provider-configs-stage2
Closed

feat: model-level provider priority lists with runtime fallback#23649
ibetitsmike wants to merge 25 commits into
mike/multi-provider-configs-stage1from
mike/multi-provider-configs-stage2

Conversation

@ibetitsmike

@ibetitsmike ibetitsmike commented Mar 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

Builds on #23647 (Stage 1). Adds model-level provider config priority lists so each model can reference multiple provider configs from the same provider family in fallback order. Disabled/unusable configs are skipped at selection time.

Enforces mandatory explicit attachments (Design A): every chat model must be bound to at least one provider config. Models without attachments are hidden from users and rejected at runtime.

Mux working on behalf of Mike.

Database

  • Migration 000467: New chat_model_provider_configs join table with model_config_id, provider_config_id, priority, timestamps, and proper constraints/indexes.
  • Backfill: Existing models get a priority-0 attachment for their family's currently enabled provider (using ROW_NUMBER() for safe multi-row handling).

API

  • ChatModelConfig response now includes provider_configs (ordered attachments with provider metadata, enabled/has_api_key status).
  • CreateChatModelConfigRequest requires provider_config_ids (ordered list). Requests without at least one ID are rejected with HTTP 400.
  • UpdateChatModelConfigRequest accepts provider_config_ids (null = unchanged, non-null = full replacement).
  • Cross-family validation rejects attachments from the wrong provider family.
  • Duplicate provider config IDs are rejected.
  • Deleting a provider triggers reconcileChatModelProviderConfigsAfterProviderDelete, which re-indexes remaining attachments or soft-deletes models that lose all attachments.

Runtime

  • resolveChatModel walks ordered provider attachments instead of merging by provider family name.
  • Skips disabled providers and providers without API keys.
  • First usable attachment wins. If none usable, model is unavailable.
  • Models with zero attachments return an explicit "no explicit provider attachments configured" error.

Cache

  • ModelAttachments(ctx, modelID) provides cached attachment lookups via tlru and singleflight.
  • Epoch-based invalidation (modelAttachmentEpoch, modelTopologyEpoch) prevents stale singleflight fetches from overwriting newer user updates.

Frontend

  • ModelForm: Provider config priority editor with drag-and-drop reorder, status badges, and minimum-one-attachment Yup validation.
  • ModelsSection: Provider config count indicator in model list rows.
  • ProvidersSection: Shows bound model count per provider.
  • modelProviderOptions: Utility for extracting provider option labels and descriptions.
  • storyFixtures: Shared test fixtures for Storybook stories.

Tests

  • API tests for create with/without configs, cross-family rejection, duplicate IDs, reorder, nil update, shared ProviderConfigIDValidation table-driven cases.
  • Runtime tests for attachment-based provider resolution with table-driven TestResolveChatModel.
  • Cache tests for attachment lookups, epoch-based stale-write prevention, cascading invalidation.
  • Frontend unit tests for form value initialization, ordering, and provider option extraction.

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@ibetitsmike ibetitsmike changed the title feat(chatd): model-level provider priority lists with runtime fallback feat: model-level provider priority lists with runtime fallback Mar 26, 2026
@coder-tasks

coder-tasks Bot commented Mar 26, 2026

Copy link
Copy Markdown
Contributor

Documentation Check

Updates Needed

  • docs/ai-coder/agents/models.md - The Add a model section describes selecting a single provider per model, but this PR introduces model-level provider config priority lists with fallback ordering. The section should document: (1) how to attach multiple provider configs to a model in priority order, (2) using the reorder (up/down) buttons in the new priority editor UI, (3) status badges (disabled / no API key) shown per provider config attachment, (4) fallback runtime behavior — the model walks through ordered attachments, skips disabled providers or those without API keys, and becomes unavailable if none are usable, and (5) the provider config count indicator now visible per model row in the models list.

Automated review via Coder Tasks

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 08f847d066

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/exp_chats.go Outdated
Comment thread coderd/database/migrations/000454_model_provider_priority.down.sql Outdated
@ibetitsmike ibetitsmike force-pushed the mike/multi-provider-configs-stage1 branch from 2821e52 to a475e54 Compare March 26, 2026 09:18
@ibetitsmike ibetitsmike force-pushed the mike/multi-provider-configs-stage2 branch 5 times, most recently from f3ff651 to 185fe71 Compare March 26, 2026 13:44
@ibetitsmike ibetitsmike force-pushed the mike/multi-provider-configs-stage1 branch 3 times, most recently from e44bf71 to ceb2d46 Compare March 27, 2026 00:11
@ibetitsmike ibetitsmike force-pushed the mike/multi-provider-configs-stage1 branch 4 times, most recently from 1628633 to c4adf90 Compare April 3, 2026 00:57
@ibetitsmike ibetitsmike force-pushed the mike/multi-provider-configs-stage1 branch 9 times, most recently from 5912186 to dcbf8bc Compare April 8, 2026 19:00
@ibetitsmike ibetitsmike force-pushed the mike/multi-provider-configs-stage2 branch from 185fe71 to a492f04 Compare April 9, 2026 01:54
@ibetitsmike ibetitsmike force-pushed the mike/multi-provider-configs-stage2 branch from 0aa113b to bee4901 Compare April 9, 2026 10:48
@ibetitsmike ibetitsmike force-pushed the mike/multi-provider-configs-stage1 branch from cc30959 to a3e75e4 Compare April 9, 2026 15:14
@ibetitsmike ibetitsmike force-pushed the mike/multi-provider-configs-stage2 branch from bee4901 to 31b937e Compare April 9, 2026 15:21
@github-actions github-actions Bot added the stale This issue is like stale bread. label Apr 30, 2026
@github-actions github-actions Bot closed this May 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale This issue is like stale bread.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants