Modalix MLSoC

Modalix MLSoC is SiMa.ai's second-generation Machine Learning System-on-Chip, announced as a System-on-Module (SoM) on March 10, 2025. It extends the original MLSoC line to multi-modal transformers, Large Language Models (LLMs), Large Multimodal Models (LMMs), and generative AI, while staying compatible with traditional CNN workloads, all within an edge-appropriate power envelope.

General
Release date	March 10, 2025 (SoM announcement)
Developer	SiMa.ai
Type	Multi-modal edge AI System-on-Module
License	Commercial hardware
Documentation	SiMa.ai Developer Center

Core Features

Machine Learning Accelerator (MLA) — supports BF16, INT8, and INT16 precision for flexible performance-per-watt tuning.
Application processor — 8x Cortex-A65 CPU cores at 1.4 GHz alongside the MLA, running Linux for full application hosting.
Memory — 32GB or 8GB of LPDDR5 at 6400 Mbps depending on configuration.
Multiple form factors — available as a System-on-Module, a PCIe companion card, and through the Modalix Development Kit for rapid prototyping.

Model Variants

Variant	TOPS	Best for
Modalix 25	25	Low-power embedded vision
Modalix 50	50	General multi-modal and CNN workloads (DevKit default)
Modalix 100	100	Higher-throughput multi-modal and LLM inference
Modalix 200	200	Maximum-performance edge deployments

Tools and Resources

Boards & DevKits — sima.ai/boards-devkits, hardware options for evaluation and deployment.
SiMa.ai Developer Center — developer.sima.ai, full documentation and getting-started guides.
SiMa.ai GitHub — github.com/SiMa-ai, example applications and model repositories.

Ecosystem and Integrations

Programmed through Palette Neat, SiMa.ai's agentic development environment, or the broader Palette SDK and ModelSDK tooling.
Used as the hackathon-provided hardware for SiMa.ai's Physical AI challenge tracks, paired with the Modalix DevKit per team.

Get started with the Modalix Development Kit product brief or the SiMa.ai Developer Center.

ElderSteady Private — fall alert with proof

Every camera product for an aging parent asks the family to watch, or to trust a company they cannot check. ElderSteady Private runs the camera's brain in the room: YOLO26-m pose on the SiMa Modalix MLA at 8 ms per picture, board-local on NVMe, no cloud. The box counts every picture it sees and destroys, and measures its own network egress on its own interface — 3.36 MB of video in, 11 KB of words out (1:303) — a number a judge can check in one command over serial. When she falls (by lean, or lying on the floor for 3 seconds), it texts the family, not 911, with a receipt instead of a clip; for 60 seconds it listens for "I'm okay" or "help" and nothing is recorded.\ The product is what comes before the fall: sit-to-stand time, seconds on the floor, company minutes, daily rhythm and night wandering — all computed from pose on the MLA, only numbers leave. Thresholds are calibrated to one room and labelled guessed; the 30-day view is fixture and says so; four controlled on-camera tests are in the repo. Built in two days by one founder with an architect session, Codex build lanes, three Claude Code lanes and a GPT-6 research lane, with Senso as shared memory. Repo: github.com/aayushdixit27/eldersteady-private

Recall

Recall turns first-person recordings, like footage from smart glasses, into a memory you can ask questions. You can ask "When was I fixing my bike?", "What was I doing before I started coding?" or "Where did I leave the wrench?", and Recall answers with a timestamp and the video clip or frame that backs it up. Everything runs locally. FFmpeg samples each video at 2 frames per second using its real timestamps. Grounding DINO Tiny finds objects and hands, and RelateAnything works out how they relate, for example holding, on, or beside. A tracker links sightings over time. From those sightings, rules build activity episodes and object states, and all of it is stored in SQLite. A FastAPI backend and a plain web UI answer questions from that data. We built it to be honest. Location answers say "last seen" and never claim to know where something is now. If a question is ambiguous or a recording failed to process, Recall says so instead of guessing. It never uses upload time as a stand-in for capture time. There is no cloud inference and no API key.

AI Conversational Fashion Agent

Modalix Edge AI Conversational Fashion Stylist delivers personalized outfit guidance through a live video feed with real-time person detection and tracking. Running on SiMa.ai Modalix and Neat, it uses YOLO26 on the MLA accelerator to detect people and an IoU tracker to assign temporary track IDs as they move through the scene. Users can visually select any detected person and request recommendations through a conversational interface. The application preserves separate chat history, styling preferences, occasion details, cultural context, and recommendations for each selected individual. Switching people loads the appropriate saved conversation and context without mixing information, while switching back restores the earlier session. A locally deployed Qwen3-VL model analyzes visible clothing, understands occasion, location, climate, culture, and dress-code requirements, and generates relevant real-time suggestions. Annotated video and metadata are streamed to Neat Insight, allowing users to view live tracking, selections, and recommendations together. The system does not perform identity recognition or infer sensitive personal attributes.

SafePath Guardian

SafePath Guardian is a camera-based safety copilot designed for autonomous mobile robots operating in busy warehouses. Running on the SiMa.ai Modalix platform, it processes live RTSP video with hardware-accelerated decoding and YOLO26n inference on the Machine Learning Accelerator. The system detects and tracks people across frames, then evaluates their location, confidence, approach motion, and proximity to configurable safety zones. Instead of returning detections alone, SafePath Guardian converts perception into actionable, explainable motion decisions: CLEAR, CAUTION, SLOW, or STOP. A state machine uses persistence and hysteresis to respond quickly to genuine hazards while avoiding unstable commands caused by brief or noisy detections. Each transition includes a human-readable reason, risk score, tracked person ID, and simulated velocity command. The application supports up to four camera streams through a shared detector and presents annotated video, tracking metadata, safety zones, and current decisions in SiMa Insight. It also records structured safety events for later analysis. SafePath Guardian demonstrates how efficient edge AI can provide responsive and transparent collision-risk awareness without relying on cloud processing.

Foreman - Natural-Language Edge AI Inspector

Foreman is a natural-language Physical AI inspection system built on SiMa.ai Modalix. It allows an operator to change a real-world inspection standard simply by speaking or typing a rule, without retraining the underlying vision models. For example, an operator can define: “The person must be holding a bottle” and “The person must not be holding a phone.” Foreman observes the live camera feed, grounds objects across the full inspection window, selects temporal evidence, and uses Qwen3-VL 4B on Modalix to understand semantic relationships such as whether a person is actually holding an object. The system follows a SEE - UNDERSTAND - ACT architecture. SEE uses real-time object detection over the camera stream. UNDERSTAND combines speech recognition, temporal evidence, and vision-language reasoning. ACT applies deterministic policy logic to produce an auditable PASS, FAIL, or UNCLEAR result. Foreman supports up to two independent rules evaluated over the same physical capture, avoiding duplicated perception work. It also provides evidence frames, detector coverage, temporal coverage, per-rule explanations, inspection history, and CSV, JSON, and ZIP exports for traceability. The key idea is: change the rule, not the model. Instead of building a separate vision pipeline for every changing factory, safety, packaging, or compliance requirement, Foreman creates a reusable inspection layer that can adapt to new standards in seconds. The prototype runs on real SiMa.ai Modalix hardware with Qwen3-VL 4B, Whisper speech recognition, and an edge object detector, demonstrating an end-to-end Physical AI workflow from real-world sensing to an actionable decision.

Baseline

This project demonstrates real-time edge AI video analytics on SiMa.ai Modalix hardware. It receives H.264 video over RTSP, decodes the stream, and runs a compiled YOLOv8n object-detection model on the Modalix MLA accelerator. The pipeline preserves a shared timeline between video and inference, ensuring detection metadata stays synchronized with the corresponding frames. Processed H.264 video and object-detection metadata are streamed to Neat Insight, where users can view live bounding boxes, COCO labels, confidence scores, and performance statistics in a browser. The application is configurable for model paths, source streams, thresholds, and Insight endpoints, and includes build, deployment, validation, and health-monitoring instructions for running the complete workflow on a Modalix DevKit.