🤝 Top Collaborators
.png&w=640&q=75)
🤓 Latest Submissions
PatchBench — Make AI Patches Prove Themselves
AI can generate code fixes quickly, but deciding whether those fixes are actually safe is still slow and uncertain. A patch may solve the reported bug while quietly breaking another behavior elsewhere in the repository. PatchBench is a local-first developer tool built around one invariant: REPRODUCE BEFORE REPAIR. A developer provides a bug report and repository. PatchBench records an immutable baseline commit, creates an isolated reproduction workspace, and asks IBM Bob to produce a regression test for the reported defect. PatchBench does not simply trust that result. It independently runs the proposed regression against the untouched baseline and only continues if the bug is reproduced for the expected reason. Once verified, the regression artifact is frozen and integrity-checked. Multiple candidate fixes can then be evaluated from the same baseline in isolated Git worktrees. Every candidate receives the same frozen regression and the same verification commands. PatchBench compares each candidate against the baseline rather than treating pre-existing failures as new defects. Candidates are rejected for newly introduced failures or failed required gates, while informational checks remain visible as evidence. The final result is an evidence matrix rather than an AI-generated quality score. The developer can inspect the regression result, new failures, type checking, build status, diffs and logs for each candidate and make the final decision. Our deterministic showcase demonstrates the value directly. An expired refresh token incorrectly returns HTTP 500 instead of 401. Three candidate patches all address the reported defect. Candidates A and C remain eligible, while Candidate B is automatically rejected because it introduces exactly two new regressions despite fixing the original bug. The hosted showcase serves a sanitized captured run; repository execution remains local. PatchBench is designed for developers and teams adopting AI coding agents who need stronger evidence before accepting generated patches.
27 Sep 2026
