TokenCascade

TokenCascade is built on one idea: in an enterprise AI stack, most tasks don't need a frontier model — they need the cheapest model that gets the answer right. The agent classifies each incoming task into one of eight capability categories (factual Q&A, math reasoning, sentiment, summarization, NER, code debugging, logic puzzles, code generation) using a lightweight, instant classifier, then routes it up an escalation ladder. Categories the local model is measured to handle reliably are answered by a quantized Qwen2.5-3B running entirely inside the container on CPU — zero Fireworks tokens. Accuracy-critical categories (math, logic, code) escalate to frontier models on Fireworks AI (MiniMax-M3 for general reasoning, Kimi K2 for code), selected at runtime from the allowed model list, with terse system prompts and per-category output caps so every remote token is spent deliberately. A global time watchdog keeps the run inside the execution budget: if local inference threatens the clock, remaining tasks flush to the fast remote path, and a never-empty fallback guarantees every task ships an answer. The result: measured on the practice set, roughly 1,000 Fireworks tokens for a full task batch with high answer accuracy — the routing pattern enterprises actually want, demonstrated end to end on AMD infrastructure with all model choice, category routing, and budgets tunable by environment variable, no rebuild required.