IBM Bob 2.0 — RAG Evolution Factory

PROBLEM: Enterprise RAG systems degrade over time. As document repositories evolve, retrieval quality drops, prompts become stale, and chunking strategies stop fitting the data. Today, developers must manually detect these regressions, diagnose the root cause, experiment with fixes, measure the impact, and roll back if the change makes things worse. This is slow, error-prone, and does not scale across dozens of agents. SOLUTION: RAG Evolution Factory is an autonomous closed-loop system where IBM Bob 2.0 acts as the engineering brain. Bob orchestrates a real RAG platform through 4 MCP tools: FACTORY — create_rag_agent: provisions a real multi-tenant RAG agent. GUARDIAN — run_eval_benchmark: runs a real benchmark and computes Recall@K, MRR, NDCG, hallucination rate, latency, and cost. AUTOPSY — get_failure_report: categorizes failures into retrieval, generation, or other. CHANGELAB — update_retrieval_config: updates the agent's retrieval configuration, then re-benchmarks and keeps or rolls back. TARGET USERS: Enterprise AI teams who operate production RAG systems. They interact with the platform through a Next.js 16 dashboard, while IBM Bob 2.0 runs the closed loop autonomously via MCP. WHAT MAKES IT UNIQUE: Unlike static RAG platforms, RAG Evolution Factory creates a self-improving feedback loop. Bob measures, diagnoses, fixes, re-measures, and decides — all through real MCP tool calls. In our live validation on Bob's own 154 documents, the closed loop raised Recall@5 from 0.59 to 0.92 — a +0.33 improvement, fully measured and verified. HOW IT WORKS IN PRACTICE: Bob created its own organization, uploaded 154 real FastAPI documents (1,534 indexed chunks), and provisioned its own baseline agent. We then ran the closed loop on Bob's real documents: baseline benchmark (Recall@5 = 0.59), failure autopsy, ChangeLab fix, and final benchmark (Recall@5 = 0.92). All metrics are stored in a live PostgreSQL database and visible through the platform's API. The 4 MCP tools are real, tested (9 out of 9 tests passing), and callable by any external MCP client over HTTP.