Skip to content

fix(executor): resolve saved model credentials inside each fallback attempt in Router and Evaluator - #9968

Merged
waleedlatif1 merged 1 commit into
stagingfrom
fix/model-fallback-credentials
Nov 13, 2026
Merged

waleedlatif1 merged 1 commit into
stagingfrom
fix/model-fallback-credentials

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Router (v1 and v2) and Evaluator now resolve saved credentials inside each model attempt. Before, they resolved the saved Vertex or Bedrock credential before their fallback loop, so a credential that couldn't be resolved failed the block before any configured fallback model ran. This covers revoked or unauthorized credentials and a transient AWS STS failure. The Agent block already resolves credentials per attempt.
  • New shared helper, resolveBlockModelCredentials in executor/utils/model-credentials.ts, used by Agent, Router and Evaluator. It replaces three inline copies of the Vertex lookup and the Bedrock-only helper.
  • Fallback behavior in executeModelRequestWithFallbacks:
    • The credential is resolved once per call, only for candidates on the primary's provider, and merged after the fallback API-key choice. A same-provider fallback row never carries its own key, so its key and token are unchanged.
    • A failed resolution fails the remaining same-provider candidates without calling again, then the loop moves to other providers.
    • Other-provider candidates still get no session and no family credentials.
  • Agent's cross-provider omit now uses the shared BLOCK_MODEL_CREDENTIAL_FIELDS list.
  • Docs fix: the v2 knowledge connector credentialId description no longer says it is null for every API-key source. A saved-credential apiKey source sets it.

Type of Change

  • Bug fix

Testing

  • New tests:
    • Router and Evaluator answer from a cross-provider fallback when the saved AWS credential is unavailable on the final try, and record the primary as failed.
    • A same-provider fallback reuses the resolved session without resolving again.
    • A failed resolution is not retried on the next same-provider candidate, and no session reaches the other provider.
  • Red proofs:
    • The handler tests fail on the old handlers.
    • The reuse test fails without the cache.
    • The failed-resolution test fails when only successful results are cached.
  • Existing tests: the existing Bedrock signing, blank-credential, fail-closed and fallback-omit tests are unchanged and pass.
  • Runs: 1,693 executor and LLM tests pass. So do bun run lint and check-block-registry.ts origin/staging.
  • Audits: check:openapi, check:cli-api, check:cli-docs, check:mcp-operations, check:api-validation:strict, check:test-patterns, check:comment-hygiene, check:explicit-any, check:client-boundary, check:boundaries, check:utils, check:file-names, check:docs and check:tool-registry-boundary pass.
  • Type-check and the full suite run in CI.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Nov 13, 2026 1:00am UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 11 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge.

Summary

Router and Evaluator now resolve saved model credentials inside the fallback loop, allowing another provider to answer when credential lookup fails.

  • Router and Evaluator keep trying fallbacks after credential lookup fails.
  • Agent model calls use the shared saved-credential resolver.
  • Knowledge connector docs distinguish saved credentials from inline API keys.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Start model attempt] --> B{Primary provider?}
  B -->|Yes| C[Resolve or reuse saved credential]
  B -->|No| D[Drop primary provider credentials]
  C -->|Success| E[Call provider]
  C -->|Failure| F[Record failed model]
  D --> E
  E -->|Failure| F
  F --> G{More candidates and retry allowed?}
  G -->|Yes| A
  G -->|No| H[Return error]
  E -->|Success| I[Return answer]
Loading

Reviews (1) · Last reviewed commit: "fix(executor): resolve saved model crede..." · Reviewed by Greptile

@waleedlatif1
waleedlatif1 merged commit 9c1857c into staging Nov 13, 2026
83 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/model-fallback-credentials branch November 13, 2026 01:30

This branch was successfully deployed

1 active deployment
Preview — a4720928 Deployed Nov 13, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant