🤓 Latest Submissions
GAUNTLET: Crash-Test Rig for Real-Time Voice AI
Voice agents rarely fail by saying the wrong thing. They fail by answering two seconds late, talking over a caller who interrupts, leaving dead air, or slowing down under load. None of that shows up in an APM trace, a transcript review or an HTTP load test. GAUNTLET is a load and chaos testing rig for real-time voice AI. It connects to an agent over a WebSocket audio stream or a LiveKit room, and AI callers with different personas (calm, impatient, elderly, on a mobile network) try to complete a real task by voice. One process holds both audio directions on one high-resolution clock, so every latency is measured from the caller's ear with no code inside the agent. Adverse conditions are applied from a seed and can be switched mid-call: packet loss, jitter, delay, cafe noise and interruptions. Results become a deterministic readiness score (A to F) against a versioned threshold profile; no language model assigns it. Two versions can be compared, and a GitHub Action fails a pull request when an agent regresses. An independent Speechmatics transcript lets a judge model mark task success, quoting the exact agent turn for every goal it accepts. The rig publishes its own accuracy: 3.897 ms measurement error over 80 calibration repetitions, and 8 concurrent calls sustained on a laptop. In a 24-call benchmark on a mobile network profile, a well-tuned agent scored A (p95 latency 786 ms) and a slow-endpointing version scored C (1,640 ms, 15.6% dead air); the gate failed it on four breaches. You can also call the bundled agent from the browser and hear the difference yourself. Built with Python, FastAPI, NumPy, React, Groq, ElevenLabs and Speechmatics. Open source.
16 Sep 2026
