PatchLens

PROBLEM AI coding agents can generate and apply software fixes in seconds, but a critical question remains: how do we know the fix actually solved the original problem? A green test suite or an AI agent saying “fixed” does not necessarily prove that the original failure disappeared. A patch could hardcode a failing input, swallow an exception, weaken or modify tests, or otherwise make the observed failure disappear without providing a trustworthy repair. This creates a verification gap for developers and teams increasingly relying on AI-generated code. SOLUTION PatchLens is a verification layer for AI-generated code fixes. Instead of trusting the AI's claim, PatchLens independently tests the proposed repair against the original failure. The workflow is: Capture the failure → Reproduce it → Apply the candidate patch → Re-run the original failure → Compare evidence → Run regression tests → Produce a verdict. PatchLens creates a structured failure signature containing evidence such as the exception type, message, source location, stack frames, reproduction input, HTTP status, and program output. After the patch is applied, the same scenario is executed again and the resulting evidence is compared with the original failure. PatchLens returns a deterministic result: 🟢 VERIFIED — The original failure disappeared under the tested conditions and supporting checks passed. 🟡 INCONCLUSIVE — Available evidence is insufficient or the execution cannot establish a reliable result. 🔴 NOT FIXED — The original failure still occurs. PatchLens also analyzes the candidate patch for suspicious repair patterns such as hardcoded failing inputs, swallowed exceptions, and modified or weakened tests. This separates two important questions: Did the failure disappear? And does the repair deserve scrutiny? TARGET USERS & INTERACTION PatchLens is designed for developers, engineering teams, AI-assisted coding workflows, and anyone using coding agents to generate software fixes. Users can start with the built-in demo or upload their own Python project, candidate patch, and reproduction script. PatchLens then executes the verification workflow and presents the evidence and final verdict through a simple interface. WHAT MAKES IT UNIQUE PatchLens is designed around a simple principle: AI writes the fix. PatchLens verifies it. The final verdict is not decided by an LLM's opinion. Execution produces the evidence, while deterministic logic evaluates that evidence. AI can assist with analysis and explanation, but it does not get to declare its own fix successful. This makes PatchLens a practical trust and verification layer for agent-generated software changes. EFFICIENCY & VALIDATION PatchLens uses deterministic execution for verification, targeted AI assistance where useful, controlled command execution, timeouts, temporary workspaces, patch validation, and secure ZIP handling. The implementation was validated with 45 backend tests and a 12-evaluation benchmark across six deliberately designed patch scenarios, including both valid fixes and suspicious repairs. PatchLens was built with IBM Bob 2.0 throughout development, from implementation and debugging to benchmarking, security hardening, and final polish, while keeping Bob outside the runtime verification decision itself.