.png&w=256&q=75)
🤝 Top Collaborators
🤓 Latest Submissions
flakeFinder — AI-Powered Flaky Test Detective
Modern software projects can contain flaky tests: tests that sometimes pass and sometimes fail depending on execution order, timing, randomness, shared state, or code regressions. These failures are difficult and time-consuming to diagnose because a single test run rarely explains why the failure occurs. FlakeFinder addresses this problem by turning flaky-test debugging into an evidence-driven, multi-agent workflow. A developer provides a failing or suspicious pytest test. FlakeFinder first identifies the target and then orchestrates five independent investigation strategies: • Isolation — determines whether the test fails when executed independently. • Shuffle — executes tests in different orders to expose order-dependent failures. • Git Bisect — searches commit history for regressions when historical evidence is relevant. • Static Scan — examines source code for suspicious patterns such as shared mutable state or uncontrolled randomness. • History — analyzes repeated executions and execution-order patterns. The five agents produce structured evidence rather than relying on one AI opinion. A Coordinator then synthesizes the evidence and identifies the most strongly supported root cause. The next stage is the Fix Agent. Instead of allowing the same reasoning process to decide whether its own fix works, the Fix Agent only proposes the smallest targeted change. An independent Verification stage then applies the proposed patch to a scratch copy of the repository and reruns the affected test repeatedly. This produces real before-and-after pass/fail evidence. Our demonstration uses four deliberately seeded flaky scenarios representing different failure mechanisms: shared-state/order dependency, race/timing behavior, uncontrolled randomness, and a historical regression. Test A demonstrates shared-cache contamination, while Test D demonstrates a regression introduced at a specific Git commit. The project is designed primarily for software developers, QA engineers, and engineering teams who spend time diagnosing unreliable automated tests. Instead of manually guessing through logs and repeatedly rerunning tests, users receive a structured investigation pipeline with independent evidence, root-cause synthesis, a proposed minimal fix, and empirical verification. What makes FlakeFinder distinctive is the separation of responsibilities. Investigation agents gather evidence independently, the Coordinator synthesizes it, the Fix Agent proposes a change without declaring success, and Verification determines whether the fix actually works through real execution. This creates a practical bridge between AI-assisted reasoning and deterministic software testing. The result is not simply an AI-generated answer to "why did my test fail?" It is an evidence-based debugging workflow designed to connect AI reasoning with reproducible engineering proof.
27 Sep 2026
