PHANTOM

PHANTOM is a hardware-transcendent local LLM runtime that enables consumer laptops and single GPUs with as little as 6 GB of VRAM to run 70B+ parameter models — including LLaMA-3 70B, Qwen2 72B, and Mixtral MoE — at conversational speeds. Unlike Ollama, llama.cpp, or vLLM, which require the entire model to fit in GPU memory, PHANTOM orchestrates VRAM, system RAM, and NVMe storage as a unified three-tier memory pool. A lightweight LSTM predictor (Wraith) runs on the CPU and streams upcoming model layers into VRAM while the GPU is still computing the current ones — eliminating transfer stalls and keeping the GPU fully utilized at all times. Seven core innovations work in concert to make this possible: Wraith Layers for predictive prefetching with sub-millisecond latency; Spectral Quantization using DCT-based FP8 compression for 4× weight reduction; Neural Cache for 8× KV-cache compression via a fused autoencoder; Phantom Pages for fast NVMe-backed weight swapping; Adaptive Compute Routing that skips ~60% of inactive neurons per token; Chronos Scheduler for sub-400ms multi-model hot-swapping; and Resonance Sampler for thermally adaptive decoding under GPU throttle. The result: a 6 GB laptop GPU can run a 70B model at ~3.5 tokens/second with a 96K token context window — a 10× ceiling lift over native hardware limits. PHANTOM exposes fully compatible OpenAI and Ollama APIs, integrates with Open WebUI, Continue.dev, and Prometheus, and supports any standard GGUF model. It runs entirely offline with zero telemetry. Built in Rust and CUDA with a Python API layer, it is MIT licensed and open source.