Blinky

Blinky is an offline-first, voice-driven AI desktop tutor powered by AssemblyAI that sees your screen, talks you through it, and can do it for you. Learning complex software means bouncing between tutorials and the app. Blinky ends that tutorial hell by bringing the tutor inside the app. Voice is core: AssemblyAI Realtime STT with Universal-3 Pro streams 16kHz audio over WebSocket for sub-second transcripts with VAD and smart turn-taking. The Voice Agent calls a native control_desktop tool to open apps, press shortcuts, or run autopilot computer-use, then speaks the tool.result with word-by-word highlighted readback. Blinky is more than tutoring. It edits video with FFmpeg to trim, merge, and auto-add captions, searches the live web with self-hosted SearXNG for up-to-date answers before helping, plays the latest YouTube videos and Spotify songs on command, and summarizes unread WhatsApp messages via whatsapp-web.js headless automation. Vision uses flicker-free DirectX capture, Windows OCR plus UIA, and Microsoft OmniParser grounding, reasoned over locally by Ollama or Groq cloud, then guides you with glowing highlights drawn on a transparent Tauri 2 overlay. The Expo mobile companion remotely controls the PC over secure WebSocket, with system telemetry, file access, and camera sync. Built with Tauri 2 Rust, React 19, and Python 3.11, Blinky is local-first for privacy. Ask by voice, follow the lights, or say do it for me.