Hina Asghar@hina005

We built MISE to explore a simple question: how can a robot take a natural-language instruction, understand what is happening in the scene, coordinate two arms, and still recover when something does not go exactly as planned? MISE, or Multi-modal Instruction to Skill Execution, is a closed-loop Physical AI system for bimanual manipulation. In our demonstration, the robot receives the instruction “Set the table” and completes a multi-stage task that includes opening a drawer, placing a mug and plate, retrieving a spoon, physically handing the spoon from one arm to the other, and placing the remaining utensils. Instead of asking a learned model to directly control every robot joint, MISE separates reasoning from physical execution. A task-specific vision-language model uses the camera observation and language instruction to determine task progress and the appropriate skill. A validated dependency graph then constrains which action can legally happen next, while camera-guided motor skills perform the physical manipulation. The system continuously verifies task outcomes. If an action does not complete correctly, MISE can detect the failure and select an appropriate validated recovery strategy rather than simply continuing with a bad state. Across our final fixed-seed evaluation, the VLA-guided system completed 10 out of 10 runs, with recovery being activated in 8 of those 10 successful runs. We also exported the learned vision-language component to Intel OpenVINO and ran it on an Intel Core Ultra 7 155H system across the CPU, Intel Arc integrated GPU, and Intel AI Boost NPU. Our goal with MISE is not just to show two robot arms moving. We wanted to demonstrate the complete loop from human instruction and visual understanding to constrained execution, physical verification, recovery, and efficient edge AI deployment.

Communicating with deaf individuals can be challenging due to the language barrier. While they are proficient in sign language, not everyone is familiar with this form of communication. Additionally, when they do speak, their speech may differ from that of a typical person. Conversely, deaf individuals may face difficulties understanding spoken language, leading to emotional distance and, unfortunately, making them targets for ridicule. Consequently, these challenges hinder their ability to play a significant role in society, emphasizing the importance of fostering understanding and inclusivity for the deaf community. We aim to facilitate seamless communication between the deaf and normal individuals, empowering deaf individuals to fully utilize their capabilities in society. Leveraging the technologies provided by Clarfai , we propose a solution that provides easy understanding between both the deaf and hearing communities.

🌎 Hybrid Hackathon | 💻 Online Build: 3–8 November 2026 | On-site Phase: 8 November 2026 📍 On-site Venue: Pierpont Place 163 W Pierpont Ave, Salt Lake City, UT 84101, USA ✨ Please note that travel and accommodation are not covered.

🌎 Hybrid Hackathon | 💻 Online Build: September 10–16, 2026 | 📍 On-site: September 15–17, 2026 The official hackathon of AI Infra Summit — co-organized by Kisaco Research and lablab.ai. Build production-grade AI applications on the modern AI infrastructure stack, alongside 8,000+ attendees, researchers, founders, and infra leaders. 📍 On-site Venue (September 15–17): Santa Clara Convention Center 5001 Great America Pkwy, Santa Clara, CA 📍 On-site participation is by invitation only. Travel and accommodation expenses will not be covered.

Build a flawless RAG based app w/o any hallucinations by using the newest Vectara’s HHEM model and win up to $1500! 🕒 24 hours to complete the challenge 💡 Implement RAG based app by utilizing HHEM model 🤝 Take part alone or form a team with other participants 🚀 3 finalists will share the Vectara’s credits worth of $4000

🚀 Access Models with Clarifai Get access to advanced models released by OpenAI recently through Clarifai platform! Build with the Latest Technology Today - GPT-4(Vision), Assistants API, Text to Speech, GPT-4 Turbo, Image Generation DALL·E API!