BusBos — Bimanual Dinner-Table VLA

Created by team Potatos on September 16, 2026
Intel Online

A language-driven bimanual agent: two simulated SO-101 arms in MuJoCo set a dinner table from plain-English commands. Six tasks span bimanual behaviour: single-arm placements (fork, spoon, cup), a simultaneous dual-arm place, a hand-off where the right arm relays an out-of-reach fork to the left arm via a shared pad, and a multi-step full setting. Success requires every object within 3 cm of target and resting on the table, cutlery within 45° of target yaw, the cup within 30° of upright, and all arms released. The agent runs a closed loop. An overhead depth camera gives object poses (mean error 0.6 mm). Qwen3-1.7B, INT4 on OpenVINO GenAI, fills a JSON-schema form per object; a critic checks it against the command (28/29 plans correct). A scheduler groups steps into simultaneous two-arm phases; unplannable steps switch arms or become a hand-off. mink IK + OMPL execute each phase with each arm planned against the other, and the camera re-checks every step and re-plans failures. On 10 held-out randomized seeds per task the agent completes 56/60 (expert with ground-truth poses: 57/60), 56/60 on unseen wording, 49/60 under 1.5x randomization, 56/60 with objects knocked off mid-task, and 45/50 free-form "arm A / arm B" commands. The expert's 293 randomized demos (LeRobot v3) train SmolVLA on all six tasks and an ACT baseline; on a laptop budget both are undertrained (<= 3/60). Both export to OpenVINO IR, including the full 10-step SmolVLA sampler as one graph, with PyTorch parity of 4.8e-4 (ACT) and 3.2e-4 rad (SmolVLA). NNCF INT8 cuts ACT from 66 to 34 MB and 42 to 26 ms per chunk (1.6x) with placements within one episode of PyTorch. All timings are from an Apple M4; scripts/intel_quickstart.sh is built to reproduce everything on Core Ultra CPU, iGPU and NPU in one command.

Category tags: