Provider System Architecture

Relevant source files

• apps/docs/content/docs/en/execution/basics.mdx
• apps/sim/app/api/mcp/tools/execute/route.ts
• apps/sim/app/api/providers/route.ts
• apps/sim/blocks/blocks/agent.ts
• apps/sim/blocks/blocks/evaluator.ts
• apps/sim/blocks/blocks/guardrails.ts
• apps/sim/blocks/blocks/router.ts
• apps/sim/blocks/blocks/translate.ts
• apps/sim/blocks/utils.ts
• apps/sim/executor/handlers/agent/agent-handler.test.ts
• apps/sim/executor/handlers/agent/agent-handler.ts
• apps/sim/executor/handlers/agent/types.ts
• apps/sim/executor/handlers/evaluator/evaluator-handler.test.ts
• apps/sim/executor/handlers/evaluator/evaluator-handler.ts
• apps/sim/executor/handlers/router/router-handler.test.ts
• apps/sim/executor/handlers/router/router-handler.ts
• apps/sim/executor/handlers/wait/wait-handler.test.ts
• apps/sim/executor/handlers/wait/wait-handler.ts
• apps/sim/lib/execution/__tests__/call-chain.test.ts
• apps/sim/lib/execution/call-chain.ts
• apps/sim/providers/anthropic/core.ts
• apps/sim/providers/anthropic/index.ts
• apps/sim/providers/bedrock/index.ts
• apps/sim/providers/index.ts
• apps/sim/providers/models.ts
• apps/sim/providers/openai/core.ts
• apps/sim/providers/types.ts
• apps/sim/providers/utils.test.ts
• apps/sim/providers/utils.ts
• apps/sim/providers/vertex/index.ts
• apps/sim/tools/llm/chat.ts
• packages/testing/src/mocks/executor.mock.ts

Overview

The provider system implements a unified abstraction layer for 30+ LLM providers including OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure, and self-hosted options like Ollama and vLLM. All providers implement the ProviderConfig interface and expose an executeRequest() method. The executeProviderRequest() dispatcher function routes requests to the appropriate provider implementation based on ProviderId.

Core Components:

• ProviderConfig interface - Standard provider contract apps/sim/providers/types.ts40-51
• executeProviderRequest() - Central dispatcher apps/sim/providers/index.ts34
• PROVIDER_DEFINITIONS - Model metadata registry apps/sim/providers/models.ts73-450
• getProviderFromModel() - Model-to-provider resolution apps/sim/providers/utils.ts214-248

Related Pages:

• 5.2 Supported Providers - Provider-specific implementations
• 5.3 Model Registry & Capabilities - Model metadata and capability detection
• 5.4 Cost Calculation & Billing - Token pricing and cost tracking
• 5.5 Streaming & Response Handling - SSE stream processing
• 5.6 Tool Execution in Agents - Tool/function calling patterns

Sources: apps/sim/providers/index.ts apps/sim/providers/types.ts40-51 apps/sim/providers/models.ts73-450 apps/sim/providers/utils.ts214-248

---

Core Architecture

Request Routing Flow

All provider requests route through executeProviderRequest() which selects the appropriate provider implementation based on ProviderId. Block handlers call this function directly or via a proxy.

Provider System Architecture Diagram

Sources: apps/sim/executor/handlers/agent/agent-handler.ts49-113 apps/sim/executor/handlers/router/router-handler.ts76-112 apps/sim/executor/handlers/evaluator/evaluator-handler.ts109-140 apps/sim/providers/utils.ts214-248 apps/sim/providers/index.ts34

ProviderConfig Interface

All providers implement the ProviderConfig interface, which defines the contract for LLM provider implementations:

Key Method: executeRequest()

The executeRequest() method is the core abstraction. It accepts a normalized ProviderRequest object and returns one of three types:

• ProviderResponse - Synchronous completion with full response apps/sim/providers/types.ts73-102
• ReadableStream<any> - Legacy streaming pattern (deprecated)
• StreamingExecution - Current streaming pattern with metadata wrapper apps/sim/executor/types.ts

Sources: apps/sim/providers/types.ts40-51 apps/sim/providers/types.ts73-102

---

Provider Registry (PROVIDER_DEFINITIONS)

The PROVIDER_DEFINITIONS object in apps/sim/providers/models.ts is the single source of truth for all provider and model metadata. It maps ProviderId to ProviderDefinition which contains model lists, pricing, and capabilities.

Registry Structure

PROVIDER_DEFINITIONS Structure Diagram

Sources: apps/sim/providers/models.ts73-100 apps/sim/providers/models.ts54-71 apps/sim/providers/models.ts29-52 apps/sim/providers/types.ts20-25

Model Lookup and Capability Functions

The providers/utils.ts module provides utility functions for querying PROVIDER_DEFINITIONS:

Function	Location	Return Type	Purpose
getProviderFromModel(model)	apps/sim/providers/utils.ts214-248	ProviderId	Maps model name to provider ID via exact match or RegExp pattern
getProviderModels(providerId)	apps/sim/providers/utils.ts269-271	string[]	Returns all model IDs for a provider
supportsTemperature(model)	apps/sim/providers/utils.ts316-335	boolean	Checks if model supports temperature parameter
calculateCost(model, in, out)	apps/sim/providers/utils.ts605-651	CostObject	Calculates cost in USD based on token usage

Sources: apps/sim/providers/utils.ts214-271 apps/sim/providers/utils.ts605-651 apps/sim/providers/models.ts1-50

---

Request Flow Architecture

Provider requests follow a consistent flow from block handlers to provider implementations.

Provider Request Execution Sequence

Sources: apps/sim/executor/handlers/agent/agent-handler.ts49-113 apps/sim/providers/index.ts34 apps/sim/providers/utils.ts605-651

ProviderRequest and ProviderResponse

The ProviderRequest interface apps/sim/providers/types.ts140-182 normalizes all provider-specific parameters into a common format:

Key Fields:

• model: string - Model identifier (e.g., "gpt-4o", "claude-3-5-sonnet")
• messages?: Message[] - Conversation history with role and content
• tools?: ProviderToolConfig[] - Tools available for function calling
• temperature?: number - Sampling temperature
• responseFormat?: {name, schema, strict} - JSON schema for structured output
• stream?: boolean - Enable streaming responses
• reasoningEffort?: string - For models like OpenAI o1/o3 apps/sim/providers/types.ts174
• thinkingLevel?: string - For models with explicit thinking capabilities apps/sim/providers/types.ts176

The ProviderResponse interface apps/sim/providers/types.ts73-102 standardizes outputs:

Key Fields:

• content: string - Generated text content
• model: string - Actual model used
• tokens?: {input, output, total} - Token usage
• toolCalls?: FunctionCallResponse[] - Tool calls made by the model
• cost?: {input, output, total, pricing} - Cost breakdown

Sources: apps/sim/providers/types.ts140-182 apps/sim/providers/types.ts73-102

---

Provider Implementations

Each provider implements the ProviderConfig interface with an executeRequest function.

Provider Implementation Matrix

Provider	Entry Point	Core API	Notes
OpenAI	providers/openai/index.ts	Chat Completions	Supports o1/o3 reasoning effort apps/sim/providers/models.ts263-278
Anthropic	providers/anthropic/index.ts	Messages API	Supports Computer Use and Thinking apps/sim/providers/models.ts365-375
Google / Vertex	providers/vertex/index.ts	Vertex AI API	Shared core logic for Gemini apps/sim/providers/vertex/index.ts15-39
Ollama	providers/ollama/index.ts	OpenAI-compatible	Local models, auto-discovery via updateOllamaModels apps/sim/providers/utils.ts152-155
OpenRouter	providers/openrouter/index.ts	Unified API	Access to hundreds of models apps/sim/providers/utils.ts163-167
DeepSeek	providers/deepseek/index.ts	DeepSeek API	Native support for V3 and R1 apps/sim/providers/models.ts515-535

Common Implementation Patterns:

1. Format Request: Convert ProviderRequest to provider-specific SDK payload.
2. Tool Preparation: Call prepareToolsWithUsageControl() for function calling apps/sim/providers/utils.ts34-39
3. Execute API Call: Use provider SDK (e.g., new Anthropic(...)).
4. Tool Call Loop: Handle iterative tool calls within the provider core logic.
5. Process Response: Extract content, tokens, and calculate cost using calculateCost() apps/sim/providers/utils.ts605

Sources: apps/sim/providers/anthropic/index.ts10-34 apps/sim/providers/utils.ts34-43 apps/sim/providers/models.ts73-450

---

Tool Calling Architecture

The provider system implements a unified tool calling interface. The AgentBlockHandler handles the resolution of tools before passing them to the provider.

Tool Preparation and Execution Flow

Sources: apps/sim/executor/handlers/agent/agent-handler.ts178-220 apps/sim/providers/utils.ts34-43 apps/sim/providers/index.ts34

---

Cost Tracking and Billing

The system tracks cost per execution based on token counts defined in PROVIDER_DEFINITIONS.

Cost Update Flow

1. Calculation: Providers call calculateCost(model, in, out) apps/sim/providers/utils.ts605
2. Pricing Source: Fetches rates from ModelPricing in model definitions apps/sim/providers/models.ts56-59
3. Conversion: Converts USD to credits using dollarsToCredits() apps/sim/providers/utils.ts5
4. Reporting: The execution engine logs these costs per block in the ExecutionContext apps/sim/executor/types.ts

Sources: apps/sim/providers/utils.ts605-651 apps/sim/providers/models.ts56-59 apps/sim/providers/utils.ts5

---

Credential Resolution

The system supports API keys, OAuth tokens, and server-side rotating keys.

Provider	Credential Source	Key Function
Standard	getApiKey(provider, model, userKey)	apps/sim/providers/utils.ts709
Vertex AI	Vertex specific credential resolution	apps/sim/executor/utils/vertex-credential.ts
Ollama	Local (returns "empty")	apps/sim/providers/utils.ts152
Hosted	getRotatingApiKey(provider)	apps/sim/providers/utils.test.ts47

Sources: apps/sim/providers/utils.ts709-758 apps/sim/providers/utils.test.ts50-171 apps/sim/executor/handlers/agent/agent-handler.ts33