🤓 Latest Submissions
Never Again: Postmortem to Regression Guard
Problem. After every serious incident, teams write a post-mortem with action items: add a test, check for similar code, update the runbook. Those items live in a document, drift, and are rarely closed, which makes the same incident far more likely to come back. Two gaps make it worse. Even when someone says "we added a test", nothing proves that test would have caught the original bug. And the root-cause pattern almost always exists in other files, where nobody goes looking. Target users. SRE, platform and backend teams who run post-mortems and want proof the follow-up happened. Solution. Never Again turns a post-mortem into a guard the codebase can't forget, and proves it. Inside IBM Bob 2.0, a custom "Post-mortem Guard" mode reads the post-mortem and extracts the root cause, trigger, seed location and action items. For each incident Bob generates a regression test, a Semgrep rule generalized from the root cause, and a guard manifest that audits every action item. How it works. An independent, deterministic verifier, not the AI, decides whether the guard is real. It checks out the buggy commit in a throwaway git work-tree and requires the test to fail on an assertion (crashes don't count). It checks out the fix and requires the test to pass. It confirms the rule matches the original bug and not the fix, then scans the whole app for variants: other files with the same mistake. Anti-cheat checks reject skipped, assertion-free or self-mocking tests. Bob fixes the variants and the verifier reruns until every incident is GUARDED; its exit code is 0 only then. Users interact through Bob in the IDE, one CLI command (npm run guard -- --all), and a 3D dashboard that reads only the verifier's JSON reports. What makes it unique. Most AI coding tools ask you to trust their output. Here Bob generates and a verifier judges, so the AI never grades its own homework. It also brings variant analysis, a security-research technique, to reliability incidents, and it audits post-mortem action items honestly, marking runbook and on-call items as process-only instead of inventing code evidence. Results on our demo app (three incidents modelled on real-world failures: a time-zone bug that promised midnight orders the wrong delivery date, a retry storm that took down a payment gateway, and a null price that produced ₹NaN checkouts): - All 3 regression tests proven both ways: fail on an assertion at the bug commit, pass at the fix commit. - 7 hidden variants found by the rules in run 1; all 7 fixed and proven closed in run 2. - Code action items with file evidence went from 6 of 11 to 11 of 11; 4 process-only items labelled honestly. - Full verification of all three incidents runs in under a minute, also on a fresh clone. Impact. Fewer repeat incidents, post-mortem follow-up in minutes instead of sprints, and evidence instead of claims. Next: a GitHub Action that runs the guard on every merged post-mortem.
27 Sep 2026
