Earshot

Every autonomous fleet keeps humans on standby: Waymo runs about 70 remote assistants for 3,000 vehicles, and in robotic warehouses the humans who handle exceptions decide throughput. Their corrections are the most valuable data those companies own, and today they evaporate the moment the ticket closes. Earshot is the voice supervision layer for autonomous agents. An operator watches an agent work and talks to it: “Stop, a bit to the left.” “Stop, squeeze it first.” The word “stop” is caught on AssemblyAI Universal-3.5 Pro partial transcripts, so the agent halts in about 300 ms, before the sentence is over. The rest of the sentence is parsed into a skill and executed on the spot. Every correction is logged with two seconds of prior state and the action the policy was about to take. When the run ends, an LLM distills the corrections into versioned policy rules, each one traceable to the sentence that earned it, while the skill layer keeps the operator’s facts (grasp offsets, packing order, “lower it first”) and enforces them on every future run. The live demo is a simulated gantry gripper packing four objects with hidden quirks the policy cannot see but any human can. Run 1 needs four corrections; run 2 needs none. The Metrics tab shows interventions, operator attention and robots per operator moving in the right direction after a single learning step. Built with AssemblyAI streaming (partial transcripts, keyterm biasing, low-latency turn detection, temporary tokens, multilingual), an OpenAI-compatible LLM for the policy and distillation, Next.js on Vercel and react-three-fiber. The skill interface maps to a ROS action server: anything with skills can be put within earshot.