IronMan: Bimanual VLA for Table Setting

IronMan is a language-conditioned bimanual manipulation system built around a single SmolVLA policy that jointly controls two SO-101 robotic arms in MuJoCo. The policy receives three synchronized RGB views—overhead, left wrist and right wrist—together with a 12-D robot state and a natural-language instruction, and predicts a 12-D action for both arms. We built an automated training pipeline using a scripted physics expert to generate 150 successful demonstrations: 50 each for plate, fork and mug placement. The resulting LeRobot dataset contains 41,940 multimodal frames. SmolVLA was fine-tuned for 50,000 updates on an NVIDIA RTX 4060 Laptop GPU, with the vision encoder frozen and the action expert adapted to the dual-arm task. At runtime, a lightweight state machine executes the sequence Plate → Fork → Mug. Learned actions are smoothed and interpolated before reaching the low-level controller, while contact-based grasp verification and closed-loop success checks confirm that each object is correctly placed, upright, released and stable. Failed stages can be retried autonomously. Across two independently seeded controlled showcase layouts, the learned policy completed all 6/6 object placements with a mean final placement error of 23.2 mm. We also evaluated the system on held-out layouts, where performance dropped outside the demonstrated staging distribution. This identified spatial diversity in the training data as the main remaining limitation rather than basic system integration. The project includes automated evaluation, telemetry, reproducible training and inference pipelines, and 72 passing tests. IronMan demonstrates an end-to-end workflow from simulation-generated demonstrations to a learned, language-conditioned bimanual manipulation policy.