.png&w=256&q=75)
🤓 Latest Submissions
Test Coverage Copilot
The Problem Test coverage tools measure lines executed, not behavior verified. A codebase can show high line coverage while its tests are shallow — calling a function without checking it did the right thing. Worse, standard coverage tools have no concept of requirements: they can't tell you when a stated feature has zero tests behind it. And when AI generates tests, there's usually no independent check on whether those tests are actually meaningful, or just plausible-looking padding. The Solution Test Coverage Copilot is an agentic Bob workflow that scans a real, multi-service codebase (Galaxium Travels — Python FastAPI backend, Java Spring Boot service, React frontend), identifies untested surface against the project's stated requirements, generates targeted tests, and independently validates them before producing a consolidated report. The core of the system is two custom Bob skills working as an actor-critic pair: The Actor generates new tests, following the repo's existing conventions (fixtures, mocking patterns, Union-return-type error handling), and maps each test to a specific requirement. The Critic independently reviews the Actor's output — seeing only the generated test file and source code, never the Actor's reasoning or the original plan. It checks whether each test verifies real behavior, maps to a genuine requirement, and covers meaningful edge cases, issuing a structured ACCEPT/REJECT/IMPROVE verdict per test and a module-level verdict overall. This isolation is deliberate: an AI reviewing its own work tends to rubber-stamp it. Separating generation from review, with no shared context, produces a genuinely independent check. What We Found Running this against Galaxium Travels surfaced real, concrete gaps: a service function (update_user) with zero tests and no endpoint even calling it, seat-class booking logic tested only via its default path, and — most significantly — a completely untested integration seam between the Java and Python services (POST /internal/bookings/from-hold). Writing tests for that seam exposed a genuine production bug: a missing payload key causes an unhandled server crash (HTTP 500) instead of a clean error response — something a plain coverage tool would never have surfaced, because there was no test at all in that path to begin with. Target Users Development teams working on multi-service or legacy codebases who need to know not just "what percentage is covered" but "what did we actually forget to test." Particularly valuable for onboarding, pre-release audits, and codebases where requirements and tests have drifted apart over time. How It's Used A developer points Bob at their repository and a requirements source (README, spec doc). Bob's Plan mode scopes the work; the Actor and Critic skills run per module; results compile into a single test-coverage-report.md containing a before/after test count, a requirement traceability matrix, gap severity ratings, and a three-dimension scorecard (Requirement Traceability, Critical Path Coverage, Edge Case Handling). What Makes It Unique Most "AI writes tests" tools stop at generation. This system adds structured, isolated review as a first-class step, ties every test back to a named requirement rather than just a line of code, and produces one human-readable report a team lead can act on immediately — backed by verified results: 87 Python tests and 127 Java tests all passing, zero regressions, one real bug caught.
27 Sep 2026
