TABLEMIND - Self Correcting Bimanual VLA

TABLEMIND is a conversational Physical AI system for bimanual robotic manipulation in a dinner-table environment. It enables two simulated SO-101 robotic arms to interpret natural-language goals, perceive the current scene, coordinate actions, verify outcomes, and adapt when the user's requirements change. The system follows a closed-loop SEE → THINK → ACT → VERIFY → ADAPT pipeline. Natural-language commands are converted into structured goals, while visual perception identifies table objects and their spatial state. A custom-trained Ultralytics YOLO26n detector recognizes plates and glasses, with OpenCV for image processing and OpenVINO optimization for efficient Intel CPU inference. MuJoCo provides the physics simulation and SO-101 robotic environment, while Cartesian IK and coordinated planning translate goals into physical manipulation. A key capability is bidirectional conversational goal reconciliation. TABLEMIND can begin with one complete place setting, respond when the user changes the requirement to two, physically restore the additional plate and glass, verify the new state, and then respond to another change back to one by physically removing the extra setting. This demonstrates adaptation to a changing user goal rather than simply restarting a predefined workflow. The system also supports voice interaction through Speechmatics, with a Streamlit web interface providing the demonstration application. Every manipulation is verified against the simulated physical state, enabling the system to reason about whether the intended outcome was actually achieved. For hardware validation, the OpenVINO vision pipeline was tested on an Intel Core i5-1135G7 CPU. Core Ultra Series 2/3 hardware was not available for this submission.