Skip to content

Commit 1ae1867

Browse files
cameroncookecodex
cameroncooke
and
codex
authored
feat(test): Expose xcresult bundle paths (#397)
* feat(test): Expose xcresult bundle paths Include xcresult bundle paths in test result artifacts when xcodebuild reports or receives a result bundle path. This lets MCP clients open the result bundle directly from structured output and text renderers. Fixes #392 Co-Authored-By: Codex <noreply@openai.com> * docs: Clarify streaming fragment output contract * fix(test): Handle malformed xcresult summaries Return null when xcresult summary parsing receives malformed JSON so the exported parser matches its nullable contract and callers can safely fall back to streamed counts. Refs #397 Co-Authored-By: Codex <noreply@openai.com> * ref(test): Share result bundle argument parsing Move result bundle path parsing into a shared helper so single-phase and two-phase test execution paths use the same validation and precedence rules. Refs #397 Co-Authored-By: Codex <noreply@openai.com> --------- Co-authored-by: Codex <noreply@openai.com>
1 parent e1e0859 commit 1ae1867

23 files changed

Lines changed: 668 additions & 79 deletions

‎AGENTS.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -56,9 +56,9 @@ ESM TypeScript project (`type: module`). Key layers:
5656

5757

5858
## Rendering and Streaming Contract
59-
- Streaming fragments are transient output only. They MUST NOT be used as internal state, cached for final responses, or promoted into final MCP/JSON/CLI text output.
60-
- Non-streaming runtimes/output modes, including MCP final responses, MUST render only from the final structured result and next-step metadata. If final output needs data, add it to the final result type instead of reading it from fragments.
61-
- Only streaming-capable renderers may observe fragment callbacks, and only to print live progress. Their fragment handling must not affect final structured output or final rendered text.
59+
- Streaming fragments are transient live-progress output only. They may be displayed while a tool is running, but MUST NOT provide final settled MCP/JSON/CLI text.
60+
- Final settled output MUST render from the final structured/domain result and next-step metadata. If final output needs data, add it to the final result type instead of reading it from fragments.
61+
- Streaming-capable renderers may observe fragment callbacks only for live progress. Fragment handling must not affect final structured output or final settled text.
6262

6363
## Test Conventions
6464
- Vitest with colocated `__tests__/` directories using `*.test.ts`

‎CHANGELOG.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,8 @@
1111
- Fixed CLI test summaries showing false-positive compiler errors from xcodebuild NSError dump lines, and added compiler-error snapshot coverage for simulator, device, and macOS build-style flows ([#383](https://github.com/getsentry/XcodeBuildMCP/issues/383)).
1212
- Fixed simulator OSLog helper cleanup so server and daemon startup reconcile same-workspace orphaned log streams without stopping helpers owned by live sessions in other workspaces ([#382](https://github.com/getsentry/XcodeBuildMCP/issues/382)).
1313
- Fixed Weather example test discovery and made CLI test progress visible while tests are running instead of leaving the last build phase displayed.
14+
- Exposed xcresult bundle paths in test result structured output and text output when xcodebuild reports or is given a result bundle path, so agents can inspect test artifacts after simulator, device, and macOS test runs ([#392](https://github.com/getsentry/XcodeBuildMCP/issues/392)).
15+
- Fixed final test summaries to use xcresult top-level test declaration counts when available, avoiding overcounting dynamic-parameter test runs ([#392](https://github.com/getsentry/XcodeBuildMCP/issues/392)).
1416

1517
### Changed
1618

‎CLAUDE.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -66,9 +66,9 @@ Use these sections under `## [Unreleased]`:
6666

6767

6868
## Rendering and Streaming Contract
69-
- Streaming fragments are transient output only. They MUST NOT be used as internal state, cached for final responses, or promoted into final MCP/JSON/CLI text output.
70-
- Non-streaming runtimes/output modes, including MCP final responses, MUST render only from the final structured result and next-step metadata. If final output needs data, add it to the final result type instead of reading it from fragments.
71-
- Only streaming-capable renderers may observe fragment callbacks, and only to print live progress. Their fragment handling must not affect final structured output or final rendered text.
69+
- Streaming fragments are transient live-progress output only. They may be displayed while a tool is running, but MUST NOT provide final settled MCP/JSON/CLI text.
70+
- Final settled output MUST render from the final structured/domain result and next-step metadata. If final output needs data, add it to the final result type instead of reading it from fragments.
71+
- Streaming-capable renderers may observe fragment callbacks only for live progress. Fragment handling must not affect final structured output or final settled text.
7272

7373
## Test Execution Rules
7474
- When running long test suites (snapshot tests, smoke tests), ALWAYS write full output to a log file and read it afterwards. NEVER pipe through `tail` or `grep` directly — that loses output you may need to debug failures.

‎schemas/structured-output/xcodebuildmcp.output.test-result/1.schema.json‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,9 @@
7373
},
7474
"packagePath": {
7575
"type": "string"
76+
},
77+
"xcresultPath": {
78+
"type": "string"
7679
}
7780
},
7881
"required": [],

‎src/mcp/tools/device/__tests__/test_device.test.ts‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -292,6 +292,34 @@ describe('test_device plugin', () => {
292292
expect(result.isError()).toBeFalsy();
293293
});
294294

295+
it('should expose user-provided result bundle paths in test output', async () => {
296+
const mockExecutor = createMockExecutor({
297+
success: true,
298+
output: 'Test Succeeded',
299+
});
300+
301+
const { result } = await runTestDeviceLogic(
302+
{
303+
projectPath: '/path/to/project.xcodeproj',
304+
scheme: 'MyScheme',
305+
deviceId: 'test-device-123',
306+
configuration: 'Debug',
307+
extraArgs: [
308+
'-resultBundlePath',
309+
'/tmp/Stale Device Tests.xcresult',
310+
'-resultBundlePath=/tmp/Device Tests.xcresult',
311+
],
312+
preferXcodebuild: false,
313+
platform: 'iOS',
314+
},
315+
mockExecutor,
316+
mockFs(),
317+
);
318+
319+
expectPendingBuildResponse(result);
320+
expect(result.text()).toContain('Result Bundle: /tmp/Device Tests.xcresult');
321+
});
322+
295323
it('should handle workspace testing successfully', async () => {
296324
const mockExecutor = createMockExecutor({
297325
success: true,

‎src/mcp/tools/swift-package/__tests__/swift_package_test.test.ts‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,24 @@ describe('swift_package_test plugin', () => {
147147
expect(result.isError()).toBeFalsy();
148148
});
149149

150+
it('should not expose xcresult paths from SwiftPM test output', async () => {
151+
const mockExecutor: CommandExecutor = async (_args, _name, _hideOutput, opts) => {
152+
opts?.onStdout?.('Result bundle written to: /tmp/SwiftPM.xcresult\n');
153+
return createMockCommandResponse({
154+
success: true,
155+
output: 'All tests passed.',
156+
});
157+
};
158+
159+
const { result } = await runSwiftPackageTestLogic(
160+
{ packagePath: '/test/package' },
161+
mockExecutor,
162+
);
163+
164+
expect(result.isError()).toBeFalsy();
165+
expect(result.text()).not.toContain('Result Bundle: /tmp/SwiftPM.xcresult');
166+
});
167+
150168
it('should return error response for test failure', async () => {
151169
const mockExecutor = createMockExecutor({
152170
success: false,

‎src/rendering/__tests__/text-render-parity.test.ts‎

Lines changed: 54 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -249,7 +249,10 @@ describe('text render parity', () => {
249249
durationMs: 2200,
250250
counts: { passed: 1, failed: 1, skipped: 0 },
251251
},
252-
artifacts: { buildLogPath: '/tmp/Test.log' },
252+
artifacts: {
253+
buildLogPath: '/tmp/Test.log',
254+
xcresultPath: '/tmp/App Tests.xcresult',
255+
},
253256
diagnostics: {
254257
warnings: [],
255258
errors: [],
@@ -284,9 +287,59 @@ describe('text render parity', () => {
284287
expect(output.match(/Discovered 2 test\(s\):/g)).toHaveLength(1);
285288
expect(output.match(/MCPTestTests\n ✗ testTwo\(\):/g)).toHaveLength(1);
286289
expect(output.match(/1 test failed, 1 passed, 0 skipped/g)).toHaveLength(1);
290+
expect(output).toContain('Result Bundle: /tmp/App Tests.xcresult');
287291
expect(output).toContain('Build Logs: /tmp/Test.log');
288292
});
289293

294+
it('matches cli text and uses structured build summary when streamed build-summary disagrees', () => {
295+
const fixture: TranscriptFixture = {
296+
progressEvents: [
297+
{
298+
kind: 'build-result',
299+
fragment: 'invocation',
300+
operation: 'BUILD',
301+
request: {
302+
scheme: 'MyApp',
303+
projectPath: '/tmp/MyApp.xcodeproj',
304+
configuration: 'Debug',
305+
platform: 'iOS Simulator',
306+
},
307+
},
308+
{
309+
kind: 'build-result',
310+
fragment: 'build-summary',
311+
operation: 'BUILD',
312+
status: 'FAILED',
313+
durationMs: 9900,
314+
},
315+
],
316+
structuredOutput: {
317+
schema: 'xcodebuildmcp.output.build-result',
318+
schemaVersion: '1.0.0',
319+
result: {
320+
kind: 'build-result',
321+
didError: false,
322+
error: null,
323+
summary: { status: 'SUCCEEDED', durationMs: 3200 },
324+
artifacts: { scheme: 'MyApp', buildLogPath: '/tmp/build.log' },
325+
diagnostics: { warnings: [], errors: [] },
326+
},
327+
},
328+
};
329+
330+
const rendered = renderTranscript(
331+
{
332+
items: fixture.progressEvents,
333+
structuredOutput: fixture.structuredOutput,
334+
},
335+
'text',
336+
);
337+
338+
expect(rendered).toBe(captureCliText(fixture));
339+
expect(rendered).toContain('✅ Build succeeded. (⏱️ 3.2s)');
340+
expect(rendered).not.toContain('❌ Build failed. (⏱️ 9.9s)');
341+
});
342+
290343
it('renders next steps in MCP tool-call syntax for MCP runtime text transcripts', () => {
291344
const fixture: TranscriptFixture = {
292345
progressEvents: [],

‎src/snapshot-tests/__tests__/json-normalize.test.ts‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,24 @@ describe('normalizeStructuredEnvelope', () => {
3535
});
3636
});
3737

38+
it('preserves xcresult paths in test result artifacts', () => {
39+
const envelope: StructuredOutputEnvelope<unknown> = {
40+
schema: 'xcodebuildmcp.output.test-result',
41+
schemaVersion: '1',
42+
didError: false,
43+
error: null,
44+
data: {
45+
summary: { target: 'simulator' },
46+
artifacts: {
47+
buildLogPath: '/tmp/build.log',
48+
xcresultPath: '/tmp/App Tests.xcresult',
49+
},
50+
},
51+
};
52+
53+
expect(normalizeStructuredEnvelope(envelope)).toEqual(envelope);
54+
});
55+
3856
it('keeps suite-less passed test cases for non-simulator results', () => {
3957
const envelope: StructuredOutputEnvelope<unknown> = {
4058
schema: 'xcodebuildmcp.output.test-result',

‎src/types/domain-results.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -168,6 +168,7 @@ export type TestResultArtifacts = AtLeastOne<{
168168
deviceId: string;
169169
buildLogPath: string;
170170
packagePath: string;
171+
xcresultPath: string;
171172
}>;
172173
export interface CoverageSummary extends StatusSummary {
173174
coveragePct?: number;

‎src/utils/__tests__/simulator-test-execution.test.ts‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,7 @@ describe('createSimulatorTwoPhaseExecutionPlan', () => {
6262
'/tmp/Calculator.xcresult',
6363
]);
6464
expect(plan.usesExactSelectors).toBe(false);
65+
expect(plan.resultBundlePath).toBe('/tmp/Calculator.xcresult');
6566
});
6667

6768
it('preserves user-supplied selector arguments in both simulator test phases', () => {
@@ -90,5 +91,32 @@ describe('createSimulatorTwoPhaseExecutionPlan', () => {
9091

9192
expect(plan.buildArgs).toEqual([]);
9293
expect(plan.testArgs).toEqual(['-resultBundlePath', '/tmp/UserProvided.xcresult']);
94+
expect(plan.resultBundlePath).toBe('/tmp/UserProvided.xcresult');
95+
});
96+
97+
it('supports equals-form resultBundlePath arguments', () => {
98+
const plan = createSimulatorTwoPhaseExecutionPlan({
99+
extraArgs: ['-resultBundlePath=/tmp/EqualsProvided.xcresult'],
100+
});
101+
102+
expect(plan.buildArgs).toEqual([]);
103+
expect(plan.testArgs).toEqual(['-resultBundlePath', '/tmp/EqualsProvided.xcresult']);
104+
expect(plan.resultBundlePath).toBe('/tmp/EqualsProvided.xcresult');
105+
});
106+
107+
it('uses the last valid resultBundlePath argument', () => {
108+
const plan = createSimulatorTwoPhaseExecutionPlan({
109+
extraArgs: [
110+
'-resultBundlePath',
111+
'-quiet',
112+
'-resultBundlePath',
113+
'/tmp/First.xcresult',
114+
'-resultBundlePath=/tmp/Last.xcresult',
115+
],
116+
});
117+
118+
expect(plan.buildArgs).toEqual(['-quiet']);
119+
expect(plan.testArgs).toEqual(['-quiet', '-resultBundlePath', '/tmp/Last.xcresult']);
120+
expect(plan.resultBundlePath).toBe('/tmp/Last.xcresult');
93121
});
94122
});

0 commit comments

Comments
 (0)