Assess PIT (pitest) for mutation testing聽#584
Description
Context
The project measures test coverage with JaCoCo. Coverage only shows which lines the tests run, not whether the tests would detect a bug in them. A line can be fully covered by a test with weak or missing assertions.
PIT (pitest) does mutation testing: it changes the bytecode in small ways (for instance, < to <=, + to -, removing a method call or changing a return value), runs the tests for each change (mutant), and reports the mutants no test detected. This shows weak tests and untested behaviour, which matters for a simulation framework where small mistakes in time or resource math give wrong results without failing.
What to assess
- Setup: add the
pitest-mavenplugin with thepitest-junit5-plugin, in a separate Maven profile (such as-Pmutation) so the normal build doesn't get slower. - Run time: PIT runs the tests many times. Measure how long a full run takes, and check options to reduce it: incremental analysis (
withHistory), limitingtargetClassesto some packages (such asschedulers,cloudlets,resources), and running in parallel (threads). - Compatibility: Java 25, Lombok-generated code (it should be excluded from mutation), and the
--add-opens/module settings the tests already use. - Results: run it on a few core packages and review the surviving mutants, to see if they show real gaps in the tests or just noise (equivalent mutants).
- CI: decide if it should run in CI (for instance, on changed classes in PRs) or only locally, and if a minimum mutation score should be required.
Expected outcome
A recommendation (with the measured run time and the mutation score of the assessed packages) on whether to adopt PIT, and how: profile, scope, and CI usage.
Activity