DuetShift

DuetShift is an adaptive bimanual Vision-Language-Action system designed for the Intel Physical AI challenge. It controls two simulated SO-101 robotic arms in MuJoCo to execute multi-step tabletop manipulation tasks from natural-language instructions and visual observations. Rather than executing a fixed sequence, DuetShift maintains a structured world state and task graph, assigns work between both arms, verifies important action postconditions, and replans when execution does not match expectations. The system is designed for coordinated reaching, grasping, transport, placement, shared-workspace manipulation, and bimanual handoff. Its signature capability is live goal revision. For example, while one arm is transporting an object, a user can change the requested destination. DuetShift updates the active goal, invalidates only the dependent future actions, dynamically reallocates work between the arms, and can use a safe handoff when the other arm is better positioned to complete the revised task. The same closed-loop architecture supports recovery when objects are displaced or the environment changes unexpectedly. The system combines MuJoCo simulation, camera-based perception, natural-language goal parsing, predicate-based world modeling, task planning, bimanual scheduling, manipulation policies, outcome verification, and recovery logic. The deployment pipeline is being designed around OpenVINO for efficient execution on Intel Core Ultra hardware. DuetShift focuses on robustness, interpretability, reproducibility, and responsive human-robot collaboration rather than one-shot scripted behavior.