🤓 Latest Submissions
ControlProof
**The workflow: guardrail verification before merge.** Teams now put guardrails around AI coding agents. In IBM Bob, that is often a hook in `.bob/settings.json`: a script that runs at a lifecycle point and can refuse a tool call, for example "Bob must not write under `protected/`." Before such a change is merged, a reviewer needs to know: did the guardrail actually stop the action, or did it only look like it did? **The problem.** Configuration alone cannot answer that. Two small mistakes produce a guardrail that looks correct and protects nothing: - **PostToolUse** instead of PreToolUse: the hook runs, sees the violation and objects, but only after the write has happened. Bob logs that PostToolUse hooks cannot block. - **Broken matcher** regex: Bob logs one warning and never runs the hook. In both cases Bob's chat can show two successful writes and no error. Today, telling "enforced" from "observed too late" from "never ran" means reading, by hand, the settings, Bob-version lifecycle semantics, Bob's task record, the hook ledger, Bob runtime log, and before/after workspace state. The costly mistake is false assurance: approving a guardrail that never enforced anything. **The solution.** ControlProof turns that evidence into one receipt per control with five separate states, never collapsed into pass/fail: - **Configured**: present in the applicable Bob configuration. - **Executed**: it ran in this session, and its own report is matched against Bob's record of the same tool calls. - **Observed**: it received the action it governs. - **Can enforce**: the lifecycle point can stop that action, taken from a per-version Bob semantics table and never inferred from outcome. - **Enforced**: it triggered, the protected write did not happen, and Bob recorded the block citing this control. When evidence is missing or contradictory, ControlProof says so: NOT DETERMINED, CONFLICTING EVIDENCE, or EVIDENCE NOT VERIFIED. The bundle is hashed into a manifest, pinned in the repository, and if it does not match, ControlProof gives no verdict. **The demo is live evidence, not a mock-up.** One policy, one prompt, three real IBM Bob 2.2.0 sessions: PreToolUse -> ENFORCEMENT VERIFIED; PostToolUse -> OBSERVATIONAL ONLY; broken matcher -> CONFIGURED - NOT EXECUTED. A reviewer runs `python controlproof.py capture` after the sessions and `python controlproof.py demo` to get a receipt stating only what the evidence supports. ControlProof also caught our own mistake: on the live run it withheld a verdict because Bob 2.2.0 stores a blocked call differently from Bob 2.1.0. After Bob fixed the reader, the unchanged evidence showed ENFORCEMENT VERIFIED.
27 Sep 2026
