Continuum: Coherent Spatial Audio Pipeline

Continuum solves one of the hardest problems in automated spatial audio: temporal coherence across visual cuts. Traditional auto-panning algorithms react to each shot in isolation, causing audio sources to jarringly "jump" between speakers every time the camera angle changes. Continuum fixes this via a novel cross-scene memory engine powered by a multi-agent AI architecture. Our end-to-end pipeline operates in six autonomous stages. First, PySceneDetect segments the video into discrete scenes, while Demucs extracts isolated stems (vocals, drums, bass, other) from the flat stereo mix. Next, a Vision-Language Model (Kimi K2.6 via Fireworks AI) captions keyframes from each scene, providing a detailed description of the visual environment. A reasoning LLM (DeepSeek] via Fireworks AI then acts as a spatial director, assigning each stem to a channel — center, left, surround, overhead, or a full ambient bed — based on those visual cues. Crucially, before finalizing placements, our Coherence Memory Engine cross-references each assignment against a persistent placement history, locking recurring sonic elements to their established channel across cuts unless the scene visibly justifies a change. Finally, placements are compiled into an ADM metadata manifest and rendered into stereo binaural, 5.1, 5.1.4, or 7.1.4 immersive formats using the open-source EBU ADM Renderer. Continuum was originally architected for self-hosted inference on an AMD MI300X via AMD Developer Cloud; we pivoted to Fireworks-hosted inference to keep the pipeline reliable for judged evaluation. Continuum features a Next.js dashboard where creators can monitor their renders and view programmatic visualizations — like coherence timelines and polar position plots — that verify the memory engine is holding placements consistently across cuts. By combining commodity models with our coherence logic, Continuum turns flat stereo into spatially coherent audio without manual labor.