The Felt Five: a factory that tests its own tests

Created by team The Felt Five on October 06, 2026
Pocketful

The checks that ship with each stage are a sample of the judges' suite: 79% of stage 1, 9% of stage 3. Green on them proves little. The Felt Five is a factory that measures its own evidence: a spec-auditor breaks one requirement at a time in a throwaway copy and counts how many breaks the band's evidence catches, against how many the shipped checks catch. In the submitted pocketful run, across four stages, it seeded 81 faults. The band caught 75; the shipped checks caught 8, all of them among those 75; the other 6 change nothing a test can observe. The implementer's own tests caught 71 of the 75, so the result does not rest on the auditor's probes. A sixth seat, the cross-auditor, runs on Codex and audits every revision blind to the other verifiers. In the submitted run it alone rejected five revisions that all three Claude verifiers had accepted, among them a fractional amount rounded into a valid payment and a paid request that a hand-edited import allowed to be paid twice. It is not one-way: the customer seat rejected two screens the reviewer had accepted. The mandates are generic. With the same six files and only the dispatch changed, the factory also built the other scored track, tablekeeper: all four stages in 2 h 39 min, one human message, every stage folder claiming its stage in an isolated run. Every design choice is an ADR with its options measured, and FACTORY.md lists every action the operator took outside the room, including a restart script replaced during the run and an Xcode licence accepted on the host. Stage 4 closed with one recorded limitation, an import of about 300 older saved statements that exceeds the 5 s limit. Repo: github.com/kosesena/dark-factory-pocketful. Live demo: felt-five-pocketful.onrender.com (demo account [email protected] / felt-demo-ada).

Category tags: