.png&w=256&q=75)
🤓 Latest Submissions
Mise en Place: Intel Bimanual VLA
Mise en Place is a dual SO-101 bimanual robot project for the Intel AI Infra Summit. The robots place a fork and spoon on a plate in parallel, then do a right-to-left spoon handoff. The system has four parts. First, SmolVLM2-256M takes an overhead camera image and a text instruction and outputs a task plan as JSON. Second, a validation gate checks the plan schema, object names, and arm assignments. Bad plans are rejected and a rule-based planner takes over. Third, a geometric controller runs inverse kinematics, parallel arm motion, collision-free handoff, and fixed grasp/release timing. Fourth, a small LoRA behavior cloning model runs on OpenVINO as a bounded correction at 50% authority. The geometric controller still guards grasps. We tested at scale: 100/100 T1 seeds, 50/50 T2 seeds, zero arm collisions in audited runs, and 10/10 hybrid INT8 closed-loop success. OpenVINO static INT8 is about 2.6x faster than FP32 on Intel iGPU. What does not work: pure behavior cloning fails closed-loop (0/10), SmolVLM failed grounding stress tests so it is proposal-only, and we had no NPU on our machine. The project is reproducible multimodal manipulation on Intel hardware, not a fully learned end-to-end policy.
16 Sep 2026
StyleCap: Grounded Video Captioning
StyleCap is a containerized video-captioning agent built for AMD Hackathon ACT II, Track 2. For each short clip it downloads the video, samples frames with ffmpeg, and runs a two-phase anti-hallucination pipeline: (1) a vision model extracts a strict JSON fact sheet from what is actually visible; (2) a text model generates captions in only the requested styles—formal, sarcastic, humorous-tech, and humorous-non-tech—from those facts alone. A batched internal judge picks the best of two candidates per style. A graceful degradation ladder and schema repair guarantee valid JSON output even under timeout pressure. The graded Docker image (ghcr.io/quantumbyte-01/amd-track2-captioner:v34) uses Fireworks serverless Kimi K2.6 for grounding and GLM 5.2 for styling and judging, with the API key baked in per Track 2 rules. Parallel workers and per-clip budgets keep 12 clips inside the 10-minute limit. For the Gemma partner prize, the identical pipeline runs end-to-end on google/gemma-3-12b-it served via vLLM on AMD Instinct MI300X (ROCm)—grounding, styling, and judging—without Fireworks Gemma deployment. Eight public validation clips complete in 86 seconds; evidence is in GEMMA_RESULTS.md and work/gemma_results.json. Demo video shows live rocm-smi GPU load and caption generation on the pod.
13 Jul 2026
