🤓 Latest Submissions
Runbook Compiler
Release runbooks live in PDFs and wiki pages. On release day, someone reads each step and ticks a box. A tick shows that a person looked. It doesn't show that the step holds. Our sample service shows the gap. Step 5 of its runbook says /health must confirm the database. /health returned 200 with {"status":"ok"} and never touched the database. Anyone skimming sees a 200 and ticks the box. Step 6 says the smoke suite covers login, list shifts, and create shift. Create shift was missing from the suite. Both steps would have been signed off, and both were false. People also skip steps. In my own manual runs of this runbook, I missed blank config variables once and skipped the container-user check once, with the checklist open in front of me. Runbook Compiler uses IBM Bob to turn a release runbook into a release gate that runs on every release. Bob reads the runbook PDF and splits it into numbered requirements. Two read-only subagents then run in parallel. Step Mapper finds the code each step refers to, and Gap Analyst decides whether that code actually meets the step. Bob writes three things: release-gate.yaml, one JavaScript check module per step a machine can verify, and docs/gap-report.md listing every step the repo fails. Steps no machine can verify, like watching a dashboard for 15 minutes after deploy, stay in the gate as POST-DEPLOY items for a person. The gate runs with npm run release-check. There is no AI at release time. The checks are plain JavaScript, give the same answer every run, and go through code review like any other code. Bob does the expensive thinking once, at compile time. Results on shiftboard-api, a Node/Express shift API: The first gate run, at commit b015bc0, failed steps 5 and 6 and passed the rest, with step 10 flagged for a person. Step 3 passed because the variables were set in my shell, but the gap report caught that the app never checked them itself. Bob then fixed all three code gaps in one turn: /health now runs a real database query, the smoke suite covers create shift, and the server refuses to start without its required environment variables. The second run, at 21c8dcd, passed all 9 automated checks in 7.90 seconds by stopwatch. My fastest measured manual run took 44.19 seconds, and that was with commands pasted, CI not opened, and one step skipped. The whole build, including the fixes and a code review, used 9.48 of 40 Bobcoins. This is for small teams that release from a runbook and have no release engineer. The runbook stops being a document people agree with and becomes a check that blocks a bad release. The gate checks contracts. It can't always prove how something is implemented, so the README lists six known weaknesses, including ones Bob's own review missed.
27 Sep 2026
