Bimanual VLA on Intel

Vercel
application badge
Created by team Handoff on September 13, 2026
Intel Online

We trained LangACT — stock ACT (a 52M-parameter bimanual manipulation policy) with a language instruction written into an internal slot the architecture already has but never uses. This costs 0.2% more parameters and 0ms of inference time, since the instruction is a known string embedded once and cached. Trained from 15,000 to 200,000 steps across 5 chained Kaggle T4 sessions, it reaches 70% closed-loop success on the standard AlohaTransferCube-v0 bimanual benchmark (MuJoCo, 20 fixed seeds) — tying the no-language ACT baseline exactly. Then we ran the test that actually matters: the instruction-swap test. Feed the model the wrong instruction while running the same task. If language steers behavior, success should collapse. We ran this at three checkpoints (125k, 150k, 200k steps). At 150k and 200k, the swapped-instruction run produced the exact same per-seed outcomes as the correct instruction — every time. We diagnosed why: every training episode pairs one task with one instruction 1:1, so the visual scene alone fully determines behavior, and nothing ever rewards reading the sentence. Rather than hide this, we built the voice interface around it. Speechmatics STT transcribes the spoken command, keyword matching resolves it to a known task, and that task actually runs — Speechmatics TTS confirms the true outcome. If asked for a task the system can't do (peg insertion, measured 0/20), it declines honestly instead of failing silently. Exported to OpenVINO INT8 (NNCF, 3.9x smaller), verified with official benchmark_app numbers: 239ms (fp32 CPU) to 71ms (INT8, Intel iGPU) — a real 3.4x speedup, on our own Core i3 laptop, no NPU, disclosed plainly. Fully reproducible: public checkpoint, verified end-to-end from a fresh clone.

Category tags: