@vas1l3vv791

Vino & Dine is a voice-guided bimanual robotics project in which two simulated SO-101 arms complete a multi-step table-setting task in MuJoCo. The goal is to connect a spoken request to a vision-language-action policy that can be evaluated safely before hardware deployment. The system uses an overhead camera, two wrist cameras, domain randomization, deterministic evaluation, and custom grasp-assist logic. Demonstrations generated by an IK-based expert are stored in LeRobot format and used to fine-tune SmolVLA with the SmolVLM2-500M-Video-Instruct backbone. The policy outputs 50-step action chunks for the 12-dimensional bimanual joint state. Speechmatics provides cloud-based real-time speech recognition. A spoken command is transcribed and matched to trigger keywords, which then launch the real SmolVLA rollout using the trained drawer-and-plate instruction. Verified results include 10/10 successful drawer open/close episodes and 7/10 successful plate pick-and-place episodes. Fixing an image-preprocessing mismatch improved evaluation from 2/10 to 7/10 without retraining the checkpoint. For optimized inference, we built a SmolVLA-to-OpenVINO bridge that directly exports the fixed-shape velocity step to OpenVINO IR while retaining PyTorch preprocessing and Python-based Euler updates. CPU parity testing validated the implementation, and our benchmark showed about a 20× speedup for the denoising-step workload versus PyTorch CPU execution. The runtime supports OpenVINO CPU, GPU, and NPU targets. The cup extension is still in progress, currently succeeding on 4/10 evaluation seeds, so it is not treated as a headline result. Future work will improve cup reliability and extend the voice-triggered workflow to the full drawer, plate, and cup sequence.

🌎 Hybrid Hackathon | 💻 Online Build: September 10–16, 2026 | 📍 On-site: September 15–17, 2026 The official hackathon of AI Infra Summit — co-organized by Kisaco Research and lablab.ai. Build production-grade AI applications on the modern AI infrastructure stack, alongside 8,000+ attendees, researchers, founders, and infra leaders. 📍 On-site Venue (September 15–17): Santa Clara Convention Center 5001 Great America Pkwy, Santa Clara, CA 📍 On-site participation is by invitation only. Travel and accommodation expenses will not be covered.