🤝 Top Collaborators
🤓 Latest Submissions
Mutation Score Climber
Line coverage is the metric most teams trust, and it lies. A test can execute a line without checking that the code did the right thing. On a small Java utility library, line coverage was already at 95% -- green across the board -- but PIT mutation testing showed the real picture: only 81.8% of deliberately-broken code variants (mutants) were caught. Roughly one in five logic changes could slip into production with every test still passing. Mutation Score Climber turns that gap into a repeatable, one-command loop. sh run_round.sh compiles the project, runs PIT, and writes two files: survivors.md (every mutant the tests missed, with file, line and mutation type) and a new row in rounds.csv. IBM Bob then reads survivors.md through a custom mutation-climb skill and a Test Engineer custom mode scoped to src/test/java/, writes the exact JUnit assertion each survivor needs, and the loop repeats. Any developer with a JUnit suite and a coverage number they don't fully trust can run this on their own project. The loop is language-agnostic in concept -- swap PIT for Stryker or mutmut and the same skill drives it. Results, all from a clean build so they reproduce from a fresh clone: mutation score rose from 81.8% to 95.8% (+14.0 pp) across 4 measured rounds; line coverage from 95% to 99%; survivors from 48 to 11, seven of which are proven equivalent mutants no test can ever kill. 37 of 48 baseline survivors are now caught.
27 Sep 2026
