Lishbor Trace: Green CI, Broken Requirement

Problem: A traceability matrix says which test proves each requirement. It stays green as long as that test passes, even when the test no longer checks what the requirement says. In August 2024, PyJWT, Python's most downloaded JWT library, merged a typing cleanup (#970) that changed isinstance(issuer, list) to isinstance(issuer, Sequence). A string is a Sequence, so a single expected issuer became a substring check. All 271 PyJWT tests passed, including the issuer test. The change sat on main for 101 days, shipped in 2.10.0 and became CVE-2024-53861. Solution: Lishbor Trace answers two questions on every merge request: would the tests notice if a requirement broke, and did this change break one? 1. Map: IBM Bob reads the specification (the RFC 7519 PDF) and maps each normative requirement to the implementing method and the test that claims to prove it, quoting the document. Lishbor accepts a mapping only if the quote appears word for word and the test passes. 2. Probe: parallel Bob tasks write, for each requirement, a witness test plus a realistic change that breaks it. Lishbor keeps a probe only if fresh checkouts show the witness passing on approved code and failing on the broken code. For PyJWT all eight probes held, every mapped test missed its probe's violation, and for five of the eight the entire suite still passed. 3. Review: on each merge request, Lishbor runs mapped tests, probes and the project suite on base and head. On the replayed #970, PyJWT's suite passes and Lishbor marks the issuer requirement violated. 4. Repair: Bob fixes the violation. Lishbor shows the patch only after a fresh checkout passes every probe, every mapped test and the full suite with no new skips. 5. Strengthen: Bob writes the regression test the project was missing. Lishbor keeps it only if it passes on approved code and fails on every probe mutant. PyJWT's new test fails on #970's commit: with it merged, the change behind the CVE could not have passed CI. Target users: teams shipping regulated or security-critical software who must show that each change was tested against its requirements (SOC 2 change management, IEC 62304, the EU Cyber Resilience Act). How they use it: paste a GitLab merge request URL into the web app, or run it as a CI gate that fails the pipeline on a requirement regression. The report shows the matrix, each requirement's evidence strength, Bob's verified repair and an audit manifest of commits, input hashes and Bob task IDs. What makes it unique: spec-driven tools write requirements first; test generators add tests. Lishbor aims every mutant at a quoted normative sentence, confirms it on fresh checkouts, and replays it as a gate on each merge request. Bob proposes; fresh checkouts decide. Efficiency: probes are written once per approved baseline and replayed free on every merge request. A verified repair or test costs about 0.2 Bobcoins, and the whole PyJWT scan and repair runs in 39 seconds on a laptop.