Skip to content

Assess PIT (pitest) for mutation testing聽#584

Open

Description

@manoelcampos

Context

The project measures test coverage with JaCoCo. Coverage only shows which lines the tests run, not whether the tests would detect a bug in them. A line can be fully covered by a test with weak or missing assertions.

PIT (pitest) does mutation testing: it changes the bytecode in small ways (for instance, < to <=, + to -, removing a method call or changing a return value), runs the tests for each change (mutant), and reports the mutants no test detected. This shows weak tests and untested behaviour, which matters for a simulation framework where small mistakes in time or resource math give wrong results without failing.

What to assess

  • Setup: add the pitest-maven plugin with the pitest-junit5-plugin, in a separate Maven profile (such as -Pmutation) so the normal build doesn't get slower.
  • Run time: PIT runs the tests many times. Measure how long a full run takes, and check options to reduce it: incremental analysis (withHistory), limiting targetClasses to some packages (such as schedulers, cloudlets, resources), and running in parallel (threads).
  • Compatibility: Java 25, Lombok-generated code (it should be excluded from mutation), and the --add-opens/module settings the tests already use.
  • Results: run it on a few core packages and review the surviving mutants, to see if they show real gaps in the tests or just noise (equivalent mutants).
  • CI: decide if it should run in CI (for instance, on changed classes in PRs) or only locally, and if a minimum mutation score should be required.

Expected outcome

A recommendation (with the measured run time and the mutation score of the assessed packages) on whether to adopt PIT, and how: profile, scope, and CI usage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions