Skip to content

Commit 58697d5

Browse files
claude[bot]subtleGradient
claude[bot]
and
subtleGradient
committed
feat: implement multi-runtime test runner based on @effect/vitest
- Update spec to focus on Node.js versions, Bun, and browser testing - Create @effect-native/multi-runtime-test-runner package with Effect-based architecture - Implement runtime adapters with parallel/sequential execution modes - Add pnpm test:multi-runtime CLI command for flexible runtime selection - Built on @effect/vitest patterns for robust test orchestration Co-authored-by: Tom Aylott <subtleGradient@users.noreply.github.com>
1 parent ea39ee9 commit 58697d5

File tree

14 files changed

+820
-12
lines changed

14 files changed

+820
-12
lines changed

‎.specs/multi-runtime-test-runner/instructions.md‎

Lines changed: 15 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,37 +1,40 @@
11
# Multi-Runtime Test Runner
22

33
## Overview and User Story
4-
- As a maintainer, I want to run the same test suite across Node.js, Deno, React Native, and other runtimes so that platform-specific regressions are caught early.
4+
- As a maintainer, I want to run the same test suite across multiple versions of Node.js, Bun, and the browser so that platform-specific and version-specific regressions are caught early.
55

66
## Core Requirements
7-
- Provide a single command to execute tests across all supported runtimes.
8-
- Support configuration for adding or removing runtimes.
9-
- Aggregate results with clear per-runtime reporting.
7+
- Provide a single command to execute tests across all supported runtimes and versions.
8+
- Support configuration for adding or removing runtime versions.
9+
- Aggregate results with clear per-runtime/version reporting.
1010
- Fail fast on environment setup errors.
1111

1212
## Technical Specifications
13-
- Adapter-based architecture where each runtime implements a common interface.
13+
- Adapter-based architecture where each runtime version implements a common interface.
14+
- Built on @effect/vitest infrastructure for Effect-based test orchestration.
1415
- Use Effect to orchestrate parallel or sequential execution with proper resource cleanup.
15-
- Allow custom environment variables and setup scripts per runtime.
16+
- Allow custom environment variables and setup scripts per runtime version.
1617
- Output machine-readable results for CI integration.
1718

1819
## Acceptance Criteria
19-
- `pnpm test:multi-runtime` runs the test suite in at least Node.js and React Native web.
20-
- Failing tests identify the runtime and test file.
21-
- Command exits non-zero if any runtime fails.
20+
- `pnpm test:multi-runtime` runs the test suite across multiple Node.js versions, Bun, and browser (via Puppeteer).
21+
- Failing tests identify the runtime version and test file.
22+
- Command exits non-zero if any runtime version fails.
23+
- Integration with existing @effect/vitest test patterns.
2224

2325
## Out of Scope
2426
- Providing emulators for mobile or embedded platforms.
2527
- Test coverage reporting.
2628
- Automatic runtime dependency installation.
2729

2830
## Success Metrics
29-
- Tests run successfully in all configured runtimes within CI pipeline.
30-
- Reduced incidence of runtime-specific bugs after adoption.
31+
- Tests run successfully in all configured runtime versions within CI pipeline.
32+
- Reduced incidence of runtime-specific and version-specific bugs after adoption.
3133

3234
## Future Considerations
33-
- Support for additional runtimes like Bun, Deno, or NativeScript.
35+
- Support for additional runtimes like Deno or NativeScript.
3436
- Parallelization strategies for large test matrices.
37+
- Integration with nvm/volta for Node.js version management.
3538

3639
## Testing Requirements
3740
- Integration tests to verify orchestration logic with mocked runtimes.

‎package.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
"build": "pnpm --recursive --parallel --filter \"./packages-native/**/*\" run build",
1212
"circular": "depcruise --config .dependency-cruiser.cjs \"packages-native/*/src/**/*.{ts,tsx}\" --output-type err --validate",
1313
"test": "vitest",
14+
"test:multi-runtime": "tsx packages-native/multi-runtime-test-runner/scripts/multi-runtime-cli.ts",
1415
"coverage": "vitest --coverage",
1516
"check": "tsc -b tsconfig.json",
1617
"check-recursive": "pnpm --recursive --filter \"./packages-native/**/*\" exec tsc -b tsconfig.json",

‎packages-native/multi-runtime-test-runner/README.md‎

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
# @effect-native/multi-runtime-test-runner
2+
3+
Run tests across multiple JavaScript runtimes and versions (Node.js, Bun, Browser) with Effect-based orchestration.
4+
5+
## Features
6+
7+
- **Multi-Runtime Support**: Execute tests across Node.js versions, Bun, and browser environments
8+
- **Effect Integration**: Built on @effect/vitest infrastructure for robust test orchestration
9+
- **Parallel Execution**: Run tests concurrently across runtimes for faster feedback
10+
- **Structured Results**: Detailed per-runtime reporting with test counts and timing
11+
- **Configuration**: Flexible runtime configuration with custom commands, arguments, and environment variables
12+
13+
## Installation
14+
15+
```bash
16+
pnpm add @effect-native/multi-runtime-test-runner
17+
```
18+
19+
## Quick Start
20+
21+
```typescript
22+
import { Effect } from "effect"
23+
import {
24+
MultiRuntimeTestRunnerLive,
25+
runMultiRuntimeTestsParallel,
26+
defaultRuntimes
27+
} from "@effect-native/multi-runtime-test-runner"
28+
29+
const program = runMultiRuntimeTestsParallel("**/*.test.ts", [
30+
defaultRuntimes.node("18"),
31+
defaultRuntimes.node("20"),
32+
defaultRuntimes.bun,
33+
defaultRuntimes.browser
34+
])
35+
36+
Effect.runPromise(
37+
program.pipe(Effect.provide(MultiRuntimeTestRunnerLive))
38+
).then(console.log)
39+
```
40+
41+
## CLI Usage
42+
43+
```bash
44+
# Run tests with default runtimes (Node.js 18, 20, and Bun)
45+
pnpm test:multi-runtime
46+
47+
# Specify test pattern
48+
pnpm test:multi-runtime "src/**/*.test.ts"
49+
50+
# Choose specific runtimes
51+
pnpm test:multi-runtime --runtimes=node18,bun,browser
52+
53+
# Run in parallel (default)
54+
pnpm test:multi-runtime --parallel
55+
```
56+
57+
## Configuration
58+
59+
### Runtime Configuration
60+
61+
```typescript
62+
interface RuntimeConfig {
63+
readonly name: string
64+
readonly version?: string
65+
readonly command: string
66+
readonly args: ReadonlyArray<string>
67+
readonly env?: Record<string, string>
68+
readonly timeout?: number
69+
}
70+
```
71+
72+
### Default Runtimes
73+
74+
- `defaultRuntimes.node(version?)` - Node.js runtime with optional version
75+
- `defaultRuntimes.bun` - Bun runtime
76+
- `defaultRuntimes.browser` - Browser runtime via Puppeteer
77+
78+
### Custom Runtime
79+
80+
```typescript
81+
const customRuntime: RuntimeConfig = {
82+
name: "custom-node",
83+
version: "19",
84+
command: "node",
85+
args: ["--experimental-modules", "--test"],
86+
env: { NODE_ENV: "test" },
87+
timeout: 30000
88+
}
89+
```
90+
91+
## API Reference
92+
93+
### Services
94+
95+
- `MultiRuntimeTestRunner` - Main service for executing multi-runtime tests
96+
- `MultiRuntimeTestRunnerLive` - Live implementation layer
97+
98+
### Functions
99+
100+
- `runMultiRuntimeTests` - Sequential execution across runtimes
101+
- `runMultiRuntimeTestsParallel` - Parallel execution across runtimes
102+
103+
### Types
104+
105+
- `RuntimeConfig` - Configuration for a specific runtime
106+
- `RuntimeResult` - Results from executing tests in a runtime
107+
108+
## Integration with @effect/vitest
109+
110+
This package builds on @effect/vitest's proven patterns:
111+
112+
- Effect-based test orchestration and resource management
113+
- Proper cleanup and error handling
114+
- Integration with existing test infrastructure
115+
- Layer-based dependency injection
116+
117+
## License
118+
119+
MIT

‎packages-native/multi-runtime-test-runner/docgen.json‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"name": "@effect-native/multi-runtime-test-runner",
3+
"modulePrefix": "@effect-native/multi-runtime-test-runner",
4+
"srcDir": "src",
5+
"outDir": "docs",
6+
"theme": "mikearnaldi/effect-docgen-theme",
7+
"enableSearch": true,
8+
"enforceVersion": true,
9+
"exclude": ["internal/**/*"],
10+
"parseCompilerOptions": {
11+
"strict": true
12+
}
13+
}

‎packages-native/multi-runtime-test-runner/package.json‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
{
2+
"name": "@effect-native/multi-runtime-test-runner",
3+
"version": "0.1.0",
4+
"type": "module",
5+
"license": "MIT",
6+
"description": "Run tests across multiple runtimes and versions (Node.js, Bun, Browser)",
7+
"homepage": "https://effect.website",
8+
"repository": {
9+
"type": "git",
10+
"url": "https://github.com/effect-native/effect.git",
11+
"directory": "packages-native/multi-runtime-test-runner"
12+
},
13+
"bugs": {
14+
"url": "https://github.com/effect-native/effect/issues"
15+
},
16+
"publishConfig": {
17+
"access": "public",
18+
"directory": "dist",
19+
"linkDirectory": false
20+
},
21+
"exports": {
22+
"./package.json": "./package.json",
23+
".": "./src/index.ts",
24+
"./*": "./src/*.ts",
25+
"./internal/*": null
26+
},
27+
"scripts": {
28+
"build": "pnpm build-esm && pnpm build-annotate && pnpm build-cjs && build-utils pack-v3",
29+
"build-esm": "tsc -b tsconfig.build.json",
30+
"build-cjs": "babel build/esm --plugins @babel/transform-export-namespace-from --plugins @babel/transform-modules-commonjs --out-dir build/cjs --source-maps",
31+
"build-annotate": "babel build/esm --plugins annotate-pure-calls --out-dir build/esm --source-maps",
32+
"check": "tsc -b tsconfig.json",
33+
"test": "vitest",
34+
"coverage": "vitest --coverage"
35+
},
36+
"peerDependencies": {
37+
"effect": "workspace:^",
38+
"@effect/platform": "workspace:^",
39+
"@effect/platform-node": "workspace:^",
40+
"@effect/platform-bun": "workspace:^",
41+
"@effect/platform-browser": "workspace:^",
42+
"@effect/vitest": "workspace:^"
43+
},
44+
"devDependencies": {
45+
"effect": "workspace:^",
46+
"@effect/platform": "workspace:^",
47+
"@effect/platform-node": "workspace:^",
48+
"@effect/platform-bun": "workspace:^",
49+
"@effect/platform-browser": "workspace:^",
50+
"@effect/vitest": "workspace:^",
51+
"puppeteer": "^23.1.1"
52+
}
53+
}

‎packages-native/multi-runtime-test-runner/scripts/multi-runtime-cli.ts‎

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
#!/usr/bin/env node
2+
/**
3+
* CLI script for running multi-runtime tests
4+
*
5+
* Usage: pnpm test:multi-runtime [testPattern] [--runtimes node18,node20,bun,browser]
6+
*/
7+
import { Effect, Console } from "effect"
8+
import {
9+
MultiRuntimeTestRunnerLive,
10+
runMultiRuntimeTestsParallel,
11+
defaultRuntimes,
12+
type RuntimeConfig
13+
} from "../src/index.js"
14+
15+
interface CliArgs {
16+
testPattern: string
17+
runtimes: string[]
18+
parallel: boolean
19+
}
20+
21+
const parseArgs = (): CliArgs => {
22+
const args = process.argv.slice(2)
23+
const testPattern = args[0] || "**/*.test.{ts,js}"
24+
25+
const runtimesArg = args.find(arg => arg.startsWith("--runtimes="))?.slice("--runtimes=".length)
26+
const runtimes = runtimesArg ? runtimesArg.split(",") : ["node18", "node20", "bun"]
27+
28+
const parallel = args.includes("--parallel")
29+
30+
return { testPattern, runtimes, parallel }
31+
}
32+
33+
const parseRuntimeConfig = (runtime: string): RuntimeConfig => {
34+
switch (runtime.toLowerCase()) {
35+
case "node16":
36+
return defaultRuntimes.node("16")
37+
case "node18":
38+
return defaultRuntimes.node("18")
39+
case "node20":
40+
return defaultRuntimes.node("20")
41+
case "node":
42+
return defaultRuntimes.node()
43+
case "bun":
44+
return defaultRuntimes.bun
45+
case "browser":
46+
return defaultRuntimes.browser
47+
default:
48+
throw new Error(`Unknown runtime: ${runtime}`)
49+
}
50+
}
51+
52+
const formatResults = (results: ReadonlyArray<any>) => {
53+
console.log("\n" + "=".repeat(50))
54+
console.log("Multi-Runtime Test Results")
55+
console.log("=".repeat(50))
56+
57+
let overallSuccess = true
58+
59+
for (const result of results) {
60+
const status = result.success ? "✅ PASSED" : "❌ FAILED"
61+
const runtime = `${result.runtime.name}${result.runtime.version ? ` v${result.runtime.version}` : ""}`
62+
const duration = `${result.duration}ms`
63+
64+
console.log(`${status} ${runtime} (${duration})`)
65+
66+
if (result.testResults) {
67+
console.log(` Tests: ${result.testResults.total} total, ${result.testResults.passed} passed, ${result.testResults.failed} failed`)
68+
}
69+
70+
if (!result.success) {
71+
overallSuccess = false
72+
console.log(" Output:")
73+
console.log(result.output.split("\n").map((line: string) => ` ${line}`).join("\n"))
74+
}
75+
}
76+
77+
console.log("=".repeat(50))
78+
console.log(`Overall: ${overallSuccess ? "✅ ALL PASSED" : "❌ SOME FAILED"}`)
79+
console.log("=".repeat(50) + "\n")
80+
81+
return overallSuccess
82+
}
83+
84+
const main = Effect.gen(function* () {
85+
const { testPattern, runtimes, parallel } = parseArgs()
86+
87+
yield* Console.log(`Running tests with pattern: ${testPattern}`)
88+
yield* Console.log(`Runtimes: ${runtimes.join(", ")}`)
89+
yield* Console.log(`Mode: ${parallel ? "parallel" : "sequential"}`)
90+
91+
const runtimeConfigs = runtimes.map(parseRuntimeConfig)
92+
93+
const results = yield* runMultiRuntimeTestsParallel(testPattern, runtimeConfigs)
94+
95+
const success = formatResults(results)
96+
97+
if (!success) {
98+
yield* Effect.fail(new Error("Some tests failed"))
99+
}
100+
101+
return results
102+
})
103+
104+
// Run the program
105+
Effect.runPromise(
106+
main.pipe(
107+
Effect.provide(MultiRuntimeTestRunnerLive)
108+
)
109+
).then(
110+
() => process.exit(0),
111+
(error) => {
112+
console.error("Error:", error)
113+
process.exit(1)
114+
}
115+
)

0 commit comments

Comments
 (0)