Testing Strategy

Relevant source files

• apps/sim/app/api/auth/oauth/disconnect/route.test.ts
• apps/sim/app/api/files/multipart/route.ts
• apps/sim/app/api/files/presigned/batch/route.ts
• apps/sim/app/api/knowledge/[id]/documents/[documentId]/chunks/route.ts
• apps/sim/app/api/knowledge/[id]/documents/[documentId]/route.test.ts
• apps/sim/app/api/knowledge/[id]/documents/[documentId]/route.ts
• apps/sim/app/api/knowledge/[id]/documents/route.test.ts
• apps/sim/app/api/knowledge/[id]/documents/route.ts
• apps/sim/app/api/knowledge/[id]/documents/upsert/route.ts
• apps/sim/app/api/knowledge/[id]/route.test.ts
• apps/sim/app/api/knowledge/[id]/route.ts
• apps/sim/app/api/knowledge/route.test.ts
• apps/sim/app/api/knowledge/route.ts
• apps/sim/app/api/knowledge/search/route.test.ts
• apps/sim/app/api/knowledge/search/route.ts
• apps/sim/app/api/knowledge/search/utils.test.ts
• apps/sim/app/api/knowledge/search/utils.ts
• apps/sim/app/api/knowledge/utils.test.ts
• apps/sim/app/api/knowledge/utils.ts
• apps/sim/app/api/schedules/execute/route.test.ts
• apps/sim/app/api/schedules/execute/route.ts
• apps/sim/app/api/webhooks/trigger/[path]/route.test.ts
• apps/sim/app/api/webhooks/trigger/[path]/route.ts
• apps/sim/app/api/workflows/[id]/execute/route.ts
• apps/sim/app/workspace/[workspaceId]/knowledge/hooks/use-knowledge-upload.ts
• apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/terminal/utils.ts
• apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-workflow-execution.ts
• apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/workflow-execution-utils.ts
• apps/sim/background/schedule-execution.ts
• apps/sim/background/webhook-execution.ts
• apps/sim/background/workflow-execution.ts
• apps/sim/executor/execution/block-executor.ts
• apps/sim/executor/execution/executor.ts
• apps/sim/executor/execution/types.ts
• apps/sim/executor/handlers/workflow/workflow-handler.test.ts
• apps/sim/executor/handlers/workflow/workflow-handler.ts
• apps/sim/executor/index.ts
• apps/sim/executor/types.ts
• apps/sim/hooks/use-execution-stream.ts
• apps/sim/lib/chunkers/json-yaml-chunker.ts
• apps/sim/lib/knowledge/chunks/service.ts
• apps/sim/lib/knowledge/chunks/types.ts
• apps/sim/lib/knowledge/service.ts
• apps/sim/lib/knowledge/types.ts
• apps/sim/lib/logs/execution/trace-spans/trace-spans.test.ts
• apps/sim/lib/logs/execution/trace-spans/trace-spans.ts
• apps/sim/lib/messaging/email/mailer.test.ts
• apps/sim/lib/messaging/email/unsubscribe.test.ts
• apps/sim/lib/tokenization/constants.ts
• apps/sim/lib/tokenization/estimators.ts
• apps/sim/lib/tokenization/index.ts
• apps/sim/lib/webhooks/processor.ts
• apps/sim/lib/workflows/executor/execution-core.ts
• apps/sim/lib/workflows/executor/execution-events.ts
• apps/sim/stores/terminal/console/store.ts
• apps/sim/stores/terminal/console/types.ts
• apps/sim/tools/knowledge/create_document.ts
• apps/sim/tools/knowledge/index.ts
• apps/sim/tools/knowledge/search.ts
• apps/sim/tools/knowledge/types.ts
• apps/sim/tools/knowledge/upload_chunk.ts
• apps/sim/tools/knowledge/upsert_document.ts
• apps/sim/trigger.config.ts
• apps/sim/vitest.setup.ts
• helm/sim/templates/cronjobs.yaml

This document describes the testing approach and quality assurance mechanisms used in the Sim platform. It covers Vitest setup, execution engine verification, mocking patterns, and testing best practices for both backend logic and frontend state.

---

Testing Overview

The Sim platform employs a multi-layered testing strategy centered around Vitest. The suite covers core workflow execution logic, block handlers, utility functions, and complex React hooks. Testing is integrated into the CI/CD pipeline to ensure that changes to the DAG executor or block registry do not introduce regressions.

The testing infrastructure is built around:

• Vitest: The primary test runner, configured with TypeScript support and environment-specific setups apps/sim/vitest.setup.ts1-10
• Mocking Utilities: Specialized mocks for database connections, environment variables, and external API responses.
• State Verification: Direct assertions on Zustand store transitions and Drizzle ORM entities.

Sources: apps/sim/executor/handlers/workflow/workflow-handler.test.ts1-10 apps/sim/vitest.setup.ts1-10

---

Execution Engine Testing

The DAGExecutor and BlockExecutor are the most critical components of the system. Testing these requires verifying the flow of BlockLog and BlockState updates across a Directed Acyclic Graph (DAG).

Block Executor Verification

The BlockExecutor is tested by mocking the BlockHandler interface and verifying that inputs are resolved correctly before execution and outputs are persisted afterward.

Key Verification Points:

• Input Resolution: Ensuring resolver.resolveInputs correctly handles variable references apps/sim/executor/execution/block-executor.ts111-115
• Error Handling: Verifying that handleBlockError captures stack traces and updates the block log with failure details apps/sim/executor/execution/block-executor.ts208-220
• Streaming: Testing the integration with handleStreamingExecution to ensure real-time logs are dispatched apps/sim/executor/execution/block-executor.ts145-153

Sources: apps/sim/executor/execution/block-executor.ts51-135 apps/sim/executor/execution/block-executor.ts208-220

---

Mocking Patterns & Utilities

Testing complex workflows requires sophisticated mocking of the execution context and external triggers.

Trigger and Webhook Mocking

Webhooks are tested by simulating NextRequest objects and verifying the WebhookProcessor logic, including signature validation and fan-out behavior.

Mock Component	Implementation Detail	File Reference
Request Body	parseWebhookBody handles JSON and URL-encoded forms	apps/sim/lib/webhooks/processor.ts106-156
Auth Verification	verifyProviderAuth mocks HMAC and OAuth signatures	apps/sim/app/api/webhooks/trigger/[path]/route.ts:107-113
Trigger Payloads	extractTriggerMockPayload provides default data for testing	apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-workflow-execution.ts:9-12

Execution Context Mocking

When testing WorkflowBlockHandler, the ExecutionContext must be mocked to support nested executions (child workflows).

Sources: apps/sim/executor/handlers/workflow/workflow-handler.ts38-87 apps/sim/lib/execution/call-chain.ts2-29

---

Testing Frontend Hooks & Stores

Frontend testing focuses on the orchestration of workflow execution and the synchronization of the terminal console.

useWorkflowExecution Hook

This hook is the primary interface for running workflows from the UI. Tests verify:

• Event Handling: Correct processing of SSE events via createBlockEventHandlers apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/workflow-execution-utils.ts:132-151.
• State Sync: Ensuring setIsExecuting correctly updates the ExecutionStore and consolePersistence apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-workflow-execution.ts:134-145.

Terminal Console Store

The TerminalConsoleStore is tested for high-performance log ingestion and indexing.

• Entry Indexing: indexWorkflowEntries is verified to ensure logs are correctly mapped to their respective blocks and execution IDs apps/sim/stores/terminal/console/store.ts144-160
• Deduplication: Testing appendWorkflowEntry to ensure that duplicate or stale logs are filtered out apps/sim/stores/terminal/console/store.ts195-210

Sources: apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-workflow-execution.ts:104-150, apps/sim/stores/terminal/console/store.ts144-210

---

Background Job Testing

Sim uses Trigger.dev for background tasks like scheduled executions and webhook processing. Testing these involves:

• Correlation IDs: Ensuring buildWebhookCorrelation and buildScheduleCorrelation generate consistent IDs for log aggregation apps/sim/background/webhook-execution.ts32-48 apps/sim/background/schedule-execution.ts46-61
• Idempotency: Verifying that webhookIdempotency.executeWithIdempotency prevents duplicate executions for the same provider event apps/sim/background/webhook-execution.ts160-164
• Timeout Logic: Testing createTimeoutAbortController to ensure long-running background jobs are terminated based on workspace limits apps/sim/background/schedule-execution.ts214-225

Sources: apps/sim/background/webhook-execution.ts32-164 apps/sim/background/schedule-execution.ts46-225

---

Best Practices

1. Isolate Side Effects: Always use the provided ExecutionContext for logging and state updates rather than direct database calls apps/sim/executor/execution/block-executor.ts182-184
2. Verify Trace Spans: Every execution test should assert that buildTraceSpans produces a valid tree of spans for the execution timeline apps/sim/lib/workflows/executor/execution-core.ts178-180
3. Sanitize for Logs: When testing output, ensure filterOutputForLog is used to prevent sensitive data or massive blobs from entering the console apps/sim/executor/execution/block-executor.ts189-191
4. Call Chain Validation: For any test involving nested workflows, verify that validateCallChain is called to prevent infinite recursion apps/sim/executor/handlers/workflow/workflow-handler.ts87-93

Sources: apps/sim/executor/execution/block-executor.ts182-191 apps/sim/executor/handlers/workflow/workflow-handler.ts87-93 apps/sim/lib/workflows/executor/execution-core.ts178-180