Table for Two: Bimanual SO-101 on OpenVINO

Created by team suzy su on September 16, 2026

Table for Two turns one instruction, typed or spoken (Speechmatics), into a two-arm table-setting routine in MuJoCo: both arms carry the plate to the placemat, the left arm opens the drawer while the right places the mug, the fork is laid, water is poured into the mug, the bottle goes back, and the spoon is handed from one arm to the other. Nothing is attached to a gripper. Objects move only because the finger pads squeeze them and friction holds them; the gripper uses the stock servo's sustainable torque, the water is 24 beads, and mass and friction are randomised per scene. A Qwen2.5-1.5B INT4 model on OpenVINO GenAI writes the plan in a compact plan language, and every plan is checked (syntax plus a simulation of what each hand holds) before the robot moves: 20/20 paraphrased instructions give valid plans. A LeRobot ACT policy (4 cameras, joint positions and velocities, subtask token) trained on 280 scripted contact demos drives the arms; NNCF INT8 runs in 4.9 ms per call on an Intel CPU, 3.4x faster than FP32. Results on 10 randomised scenes: the scripted pipeline sets the whole table 10/10 (29/30 on unseen scenes). The learned policy does the plate, drawer and mug 10/10 and the fork 9/10, but the pour, bottle return and hand-over still break the chain, so it has not set a full table on its own; from a clean start it returns the bottle 10/10 and hands the spoon over 5/10. In the video those three stages are run by the scripted skills and labelled on screen. Every number, the checkpoint and the OpenVINO models are in the repository and its release.

Category tags: