Talos: Learned Dinner-Table Robotics

Talos turns supported spoken or typed instructions into dinner-table manipulation by two SO-101 arms in MuJoCo. Six learned skills place a bottle, plate and mug, open a passive drawer, and retrieve the fork and spoon. Speechmatics transcribes commands; a bounded language interpreter and classical RGB checks select actions. Neural motor trajectories use camera features and joint feedback. A separate simulator-state monitor stops or scores execution. The original learned dinner system passed 8/10 frozen sequences. Table-supported bottle relays passed 5/5 frozen trials in each direction; a composed relay-plus-dinner workflow passed 8/10. All failures remain. Later stereo correction during late mug placement passed 20/20 narrow workflows, as did the original baseline; frozen corrective images passed 0/20. This establishes bounded visual feedback, not general workspace coverage. A human “Set the table” voice command completed all six skills. The hosted live-mug CPU demo also passed. Six-skill baseline execution is recorded on a legacy Intel laptop with Intel CPU/iGPU inference and UHD rendering. Core Ultra Series 2/3 eligibility remains unresolved; the latest live-mug profile needs separate Intel verification. Broader work includes seven loose items, full tabletop yaw and stable alternative orientations. No complete jointly randomized seven-item scene has passed. Calibrated RGB-D and an adapted CLIPort runtime work on the RTX 4070, but no shared Talos spatial policy has been trained. Arbitrary reachable positions, general visual recovery, unrestricted language, pouring and airborne handoffs remain unfinished. Original Talos contributions are MIT licensed; third-party components retain their own licenses. Source, compact training inputs, model provenance, protocols and successful/failed physical outcomes are preserved.