Katlego Tlhapiso@katlegotlhapiso49097

OmniCaption is a high-performance, containerized video captioning system built for Track 2 and Track 3 of the AMD Developer Hackathon (Act II). It solves automated, context-aware video cataloging by generating four distinct stylistic descriptions per clip: formal, sarcastic, humorous_tech, and humorous_non_tech. Every caption is strictly grounded in raw clip evidence, eliminating hallucinations via a modular 6-stage pipeline: Ingestion, Audio, VRAM Handoff, Vision, Style Synthesis, and Output. Innovation: Sequential VRAM Handoff Running heavy Speech-to-Text (STT) and Vision-Language Models (VLMs) concurrently causes Out-Of-Memory (OOM) crashes on 8 GB VRAM GPUs. OmniCaption resolves this with a memory-reclamation loop: 1. Loads faster-whisper on the GPU, transcribes audio, and saves text to CPU memory. 2. Unloads the STT model and runs a garbage collection sweep to free 100% of VRAM. 3. Loads the VLM or calls the vision API for the visual synthesis phase. This workflow ensures robust performance on consumer-grade hardware. AMD Hardware Acceleration OmniCaption utilizes AMD compute at every stage. Local STT runs via faster-whisper compiled against a custom CTranslate2-HIP backend, running natively on AMD ROCm. Remote VLM synthesis queries are processed using serverless endpoints hosted on Fireworks AI clusters, backed by AMD Instinct MI300X accelerators. The codebase also supports native local VLM execution via an OpenAI-compatible server powered by vLLM on ROCm. Full-Stack Control Plane The system features a decoupled control interface. A FastAPI backend handles task queues, secure media serving, and job tracking. This pairs with a dark-themed Next.js 15 dashboard featuring aurora effects, user authentication, live status tracking, and interactive diagnostic logs for quick troubleshooting.

Six Matches. One leaderboard. Build on AMD, earn points and compete for prizes.

🌎 Online Hackathon | 📅 August 3–10, 2026| 💻 Powered by native.builder The official native.builder hackathon, hosted by NativelyAI and lablab.ai. Join founders, developers, designers, and builders to create real AI-native applications powered by native.builder: from idea to deployed product, fully in the cloud. 💻 This hackathon is fully online. No travel required.

To participate in the hackathon, simply click Sign up with AMD. If you are not already a member of the AMD AI Developer Program (ADP), you will need to create an account. ADP members can access AI tutorials, experts, and community support. In addition, new members can claim a free month of DeepLearning.ai pro, $100 in AMD GPU cloud credits, and $50 in Fireworks AI API credits. All hackathon participants will receive on hackathon launch additional compute and API credits for their projects. Ready to build what's next on AMD? Sign up with AMD below to secure your spot.

Grab your ideas, spin up compute, and start building what’s next! Start with $100 in AMD Developer Cloud credits and begin building right away. Get access to hands-on tutorials, workshops, and curated learning resources to help you build faster and go further. To unlock credits and join the hackathon, sign up for the AMD AI Developer Program. 🤝 Go solo or team up — ship something real. 💰 $21,500+ total prize pool & 1 × AMD Radeon™ AI PRO R9700 GPU up for grabs. 📍 On-site Venue (May 9-10): MindsDB SF AI Collective 3154 17th St, San Francisco, California, USA 📍On-site participation is by invitation only. Travel and accommodation expenses will not be covered. 🧑‍💻 Developers, builders, and innovators - bring your boldest ideas and let’s turn them into real AI systems on AMD.