Skip to content

feat(tester): agent-friendly bit test output with per-component rollup#10313

Merged
davidfirst merged 17 commits into
masterfrom
feat/bit-test-agent-friendly-output
Apr 21, 2026
Merged

feat(tester): agent-friendly bit test output with per-component rollup#10313
davidfirst merged 17 commits into
masterfrom
feat/bit-test-agent-friendly-output

Conversation

@davidfirst
Copy link
Copy Markdown
Member

@davidfirst davidfirst commented Apr 17, 2026

Rework bit test output so humans and AI agents can tell pass/fail at a glance:

  • Per-component rollup after native tester output: one indented line per component ( bullet for pass, for fail, for tester error), following the shared CLI style guide.
  • Numeric headline summary (✔ 359/359 tests passed across 17 components, 79 without tests).
  • Collapse components with zero spec files into a single hint line (--verbose expands to the full list).
  • Skip empty-component invocations in TesterService, removing Mocha's 0 passing (0ms) noise.
  • Distinguish components affected by env/tester errors from components with no tests.
  • New --summary flag: suppress tester output, print only the final headline. With --json, data becomes a structured summary (totals, per-component rollup, components-without-tests, env errors) — ideal for agents that just need to grep pass/fail.
  • Remove the per-env no tests found for components using environment X warning — the new summary covers it.

Uses the shared @teambit/cli output toolkit. Unit tests cover aggregation and rendering.

Copilot AI review requested due to automatic review settings April 17, 2026 20:08
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reworks bit test output to provide an agent-/human-friendly rollup of results across components and environments, including a concise final summary and an additive JSON summary block.

Changes:

  • Skip invoking testers when an env has zero spec files (reduces “0 passing (0ms)” noise).
  • Add a new test-output-formatter module to aggregate results and render a per-component rollup + final summary (with --verbose expanding the “no tests” list).
  • Extend bit test --json output with an additional summary object (totals + per-component counts + env errors).

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File Description
scopes/defender/tester/tester.service.ts Filters out components with zero spec files before running the tester; returns empty results when none have tests.
scopes/defender/tester/test.cmd.ts Adds --verbose, uses the new aggregator/formatter for human output, and adds summary to JSON output.
scopes/defender/tester/test-output-formatter.ts New aggregation + rendering logic for per-component rollup, env errors, no-tests section, and final headline.
scopes/defender/tester/test-output-formatter.spec.ts Unit tests covering aggregation and report rendering.

Comment thread scopes/defender/tester/test-output-formatter.ts
Comment thread scopes/defender/tester/test-output-formatter.ts Outdated
Comment thread scopes/defender/tester/test-output-formatter.ts
Comment thread scopes/defender/tester/test-output-formatter.ts Outdated
Comment thread scopes/defender/tester/test.cmd.ts Outdated
Comment thread scopes/defender/tester/test-output-formatter.ts Outdated
Copilot AI review requested due to automatic review settings April 17, 2026 20:49
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Comment thread scopes/defender/tester/test-output-formatter.ts Outdated
Comment thread scopes/defender/tester/test.cmd.ts
Comment thread scopes/defender/tester/test.cmd.ts Outdated
Copilot AI review requested due to automatic review settings April 20, 2026 13:31
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Comment thread scopes/defender/tester/test.cmd.ts Outdated
Comment thread scopes/defender/tester/test.cmd.ts
Comment thread scopes/defender/tester/test-output-formatter.ts Outdated
Copilot AI review requested due to automatic review settings April 20, 2026 13:49
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Comment thread scopes/defender/tester/test.cmd.ts
Copilot AI review requested due to automatic review settings April 20, 2026 14:38
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Comment thread scopes/defender/tester/test-output-formatter.ts
Comment thread scopes/defender/tester/test-output-formatter.ts
Copilot AI review requested due to automatic review settings April 20, 2026 16:39
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Comment thread scopes/defender/tester/test-output-formatter.ts Outdated
Comment thread scopes/defender/tester/test.cmd.ts
Comment thread scopes/defender/tester/test.cmd.ts
Copilot AI review requested due to automatic review settings April 20, 2026 19:15
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Comment thread scopes/defender/tester/test.cmd.ts Outdated
Copilot AI review requested due to automatic review settings April 21, 2026 14:10
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

scopes/defender/tester/tester.service.ts:156

  • specFiles is filtered down to components with spec files, but patterns (and testerContext.components via Object.assign(context, ...)) still include all context.components. Testers like the built-in Jest tester derive the executed spec patterns from context.patterns (scopes/defender/jest/jest.tester.ts:303-308), so components with zero spec files will still be passed to the tester, undermining the “skip empty-component invocations” goal and potentially reintroducing noise/extra work. Consider filtering patterns (and overriding testerContext.components) to match the filtered specFiles.components set so empty components are truly excluded from the tester run.
    const allSpecFiles = ComponentMap.as(context.components, (component) => {
      return detectTestFiles(component, this.devFiles);
    });
    // drop components with zero spec files before handing them to the tester — otherwise testers
    // like Mocha invoke their reporter once per component and print "0 passing (0ms)" noise.
    const specFiles = allSpecFiles.filter((files) => files.length > 0);
    const componentWithTests = specFiles.toArray().length;

    if (componentWithTests === 0 && !options.ui) {
      // silent: the final summary in `bit test` collapses no-test components into a single line.
      return new Tests([]);
    }

    if (!options.ui)
      this.logger.console(`testing ${componentWithTests} components with environment ${chalk.cyan(context.id)}\n`);

    const patterns = await ComponentMap.asAsync(context.components, async (component) => {
      const componentDir = this.workspace.componentDir(component.id);
      const componentPatterns = this.devFiles.getDevPatterns(component, TesterAspect.id);
      const packageRootDir = await this.workspace.getComponentPackagePath(component);

      return {
        componentDir,
        packageRootDir,
        paths:
          componentPatterns.map((pattern: string) => ({
            path: resolve(componentDir, pattern),
            relative: pattern,
          })) || [],
      };
    });

    let additionalHostDependencies = [];
    if (
      context.env.getAdditionalTestHostDependencies &&
      typeof context.env.getAdditionalTestHostDependencies === 'function'
    ) {
      additionalHostDependencies = await context.env.getAdditionalTestHostDependencies();
    }

    const testerContext = Object.assign(context, {
      release: false,
      specFiles,
      patterns,
      sourcePatterns: patterns,

Comment thread scopes/defender/tester/test-output-formatter.ts
@davidfirst davidfirst merged commit 971263d into master Apr 21, 2026
16 checks passed
@davidfirst davidfirst deleted the feat/bit-test-agent-friendly-output branch April 21, 2026 14:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants