Downshift

PROBLEM: Teams ship LLM features fast and default to the biggest model. The bill shows one total, so nobody knows which features actually need the premium model, and switching to a cheaper one feels risky without proof that quality holds, so nobody does it. Meanwhile, every PR can quietly add spend that no reviewer sees. Static analysis alone can't fix this, because real code hides models in config and wrapper helpers, and one helper often serves several features. On our demo app, the AST scanner found 7 call sites with 6 models resolved, when there were really 8 features. SOLUTION: Downshift is an open-source Python CLI and GitHub Action (pip install downshift). Downshift is the toolkit, and IBM Bob is the brain. First, an AST scanner finds every LLM call site and projects monthly spend. Then Bob, running our custom Downshift Auditor mode and skill, resolves what static analysis can't: hidden models, and shared helpers split into real features, each with a purpose, output contract and grading method. Next, Bob writes per-feature test cases, including with parallel subagents (without Bob, downshift evalgen drafts them with any LLM). Downshift then runs the baseline against cheaper models, scoring by exact match, JSON fields, or an independent judge model, and downgrades a feature only if the cheaper model keeps at least 95% of baseline quality. The report shows projected savings, what the analysis itself cost, and how fast it pays back. Bob then moves model names to config, applies the decisions and opens the PR, and from then on a GitHub Action comments "+$X/month" on any PR that changes LLM spend, using zero LLM calls. RESULTS: On the SupportDesk demo (8 features, synthetic data), 3 of 8 features were safely downgraded, cutting projected spend from $3,137 to $2,573 per month (18%), and the analysis cost $0.65 once and pays back in 51 minutes. The expensive features failed the quality bar, so Downshift kept them; it only downgrades what it can prove. On real open-source repos, Bob surfaced spend static analysis missed: for OrchestrAI (MIT), the static scan saw 1 call site and $540 per month, while Bob's audit found 5 features and $2,409 per month; for mem0 (Apache-2.0), the static scan saw 14 sites with 1 model resolved and $307 per month, while Bob found 4 real features and $2,330 per month. On mem0's reranker, Downshift refused a downgrade because the best cheaper model kept only 76% of baseline quality. SHIPPED, NOT A PROTOTYPE: v0.1.1 is on PyPI via Trusted Publishing, with 612 Python and 34 web tests, CI on Python 3.10 to 3.12, and a live demo app. HONEST LIMITS: Dollar figures are projections (measured tokens times illustrative prices times assumed volume), and the evals ran on local Qwen models.