S M Mohaiminul Islam Arik@Arik

Software testing is essential but often repetitive and time-consuming. Developers need to identify untested code, write tests, execute them, diagnose failures, repair tests or code, and repeatedly verify the result. AI can automate parts of this workflow, but blindly allowing an AI agent to modify tests can create a dangerous outcome: the system may change a test simply to make it pass while leaving a genuine application defect hidden. TestPilot is an AI-assisted software testing pipeline designed to automate this workflow while keeping verification measurable and safety-aware. The system analyzes a Python repository to identify testing gaps and target modules. A Test Generation Agent then creates tests using the project's existing testing conventions. Generated tests are extracted and validated through an AST-based validation layer before execution. TestPilot executes the generated tests and analyzes failures. When appropriate, a bounded Repair Agent attempts to repair failing tests through controlled iterations. A Safety Gate distinguishes failures that should not be "fixed" by modifying tests, such as genuine source-code logic defects. The pipeline follows: Analyze → Generate → Validate → Execute → Detect Failure → Analyze Failure → Repair → Re-execute → Evaluate → Metrics Target users are software developers, engineering teams, and researchers interested in AI-assisted software quality and automated testing. Users interact with TestPilot primarily through a CLI. They can provide a repository, select a target, run generation or evaluation, and inspect structured results including generated-test validity, execution outcomes, repair attempts, coverage changes, and benchmark metrics. The creative aspect of TestPilot is not simply generating more tests. It combines autonomous test generation with execution feedback, bounded iterative repair, safety checks, and measurable evaluation. The system is designed so that an AI agent is not rewarded merely for making every test pass. In a real benchmark run, TestPilot generated 8 tests and all 8 were valid. Coverage increased from 65.12% to 69.77%. During repair, the system encountered a genuine logic failure; after two repair attempts it stopped and triggered its safety gate rather than modifying tests to conceal the defect. The final project verification completed with 342 passing tests and mypy reporting no issues across 27 source files.

Six Matches. One leaderboard. Build on AMD, earn points and compete for prizes.

💻 Online Hackathon | ⏳ Build: 48 hours | 📅 September 25–27, 2026 The official IBM Bob 2.0 hackathon. Build what's next in AI-assisted development, alongside the developers and experts shaping it. ⭐ Get help from experts throughout the event 🤝 Join a team or build solo 💰 Prize pool: $12,000 🧑🏻‍💻 Register before the Kick-Off Stream to start building with Bob

🌎 Online Hackathon | 💻 Month-long challenge | Dates: Sep 1–30, 2026 | 🏆 $10,000 Prize Pool ($5k cash + $5k in AAI credits) A month-long online challenge run by lablab.ai together with AssemblyAI, the Voice AI infrastructure company for builders. Every participant builds on AssemblyAI. You can join at any point — registration stays open for the whole build window, so you can start on day one or pick the project up halfway through and still submit. 🌍 Fully online hackathon. Join and build from anywhere in the world.