.png&w=256&q=75)
🤝 Top Collaborators
.png&w=640&q=75)
.png&w=640&q=75)
.png&w=640&q=75)
🤓 Latest Submissions
Hall Monitor: Code Review of AI-Written Changes
Hall Monitor improves one developer workflow: code review of AI-written changes. The problem: a coding agent says "wired it in", "added tests", "didn't touch auth", and a reviewer has to re-check each claim from scratch. Of 23,247 agent-written pull requests, 1.7% had descriptions that didn't match the code. The most common mismatch (45.4%) claimed changes that were never made, and those PRs were accepted 28.3% of the time, against 80.0% for the rest (MSR 2026). Agents also drift off task and talk themselves past rules ("just this once"). The solution: Hall Monitor supervises IBM Bob while it works, built from Bob's own hooks, MCP server, modes, skills and subagents. - Rules from documents: /decisions docs/security-policy.pdf has Bob read the policy and record each rule with its section and a quote. - Declared intents: before each edit or command, Bob says what it will do. Hall Monitor checks the intent against the goal and the rules, names excuses (exception, deferral, reinterpretation, goal substitution) and rejects them before anything runs. A hook blocks any edit no approved intent covers. - Subagents: each brief is checked before the subagent starts, and its summary when it returns. - Receipts: every edit and command becomes a numbered receipt (E1, E2...), and each claim Bob makes must cite them. Code checks them first (the file exists, it is in the diff, tests passed after the last edit), then fresh test runs, sabotage mutants and a check that new tests fail without the change, then Jev. Uncertain claims go to a read-only Bob auditor, then to a person. After 2 send-backs, Hall Monitor stops and names the checkpoint to restore. - The Hall Pass: a one-page report of the session that Bob publishes as an artifact. Who uses it, and how: developers and reviewers who let Bob write code. They install it with one script, pick the Supervised mode, load their policy with /decisions, and give Bob the task. In CI, scripts/headless.py runs Bob Shell under supervision and fails the build unless the receipts verify. What's new: review stops being a re-check of a finished diff. Work is checked as it happens, against evidence Hall Monitor gathered itself rather than the agent's word. Every question about a step goes to TypeSafe's Jev model in one parallel call, and code picks the action with the lowest expected loss using a harm table the team controls, so one confident wrong guess can't wave through a destructive command. The auditor mode has no edit permission, so it can't alter the evidence it checks. Results so far: on claims we seeded ourselves (n=60), Hall Monitor caught 21 of 22 false claims, with 2 false alarms on 38 true claims. It scores 20/20 on a control set of 10 must-block and 10 must-allow actions. In 10 real Bob Shell runs, 3 ended verified, including the rate-limit demo task. Parallel-subagent runs still get stuck more often than we want; each failure became a fix. Jev costs about $0.003 per supervised session.
27 Sep 2026
