SRE Incident Agent — Blast Radius Zero

Created by team SRE Incident Agent on November 03, 2026

The problem. An on-call SRE facing a cryptic 500-line log spends 20–40 minutes of manual grep/correlate work to know what broke and what to do first. LLM assistants can draft guesses, but executing LLM-generated code on a real system is the classic blast-radius problem: run it blind, with network access and secrets in the environment, and a hallucinated diagnostic becomes an incident itself. What we built. SRE Incident Agent closes the loop safely. Paste a log — the LLM plans a diagnostic step, the step runs as real Python in an isolated Docker container, the verified stdout and exit code feed back into the agent's context, and after ≤5 steps it returns a triage report: findings backed by numbers from executed code, three ranked root-cause hypotheses, and an ordered runbook. Nothing the LLM claims is trusted unless an executed snippet printed it. Blast radius zero. The LLM-written code is the attack surface, closed in layers: a static guard refuses subprocess, sockets, URL fetches, file writes, eval/exec and host paths before anything runs (shown as BLOCKED · contained); each snippet runs in a one-shot container with --network none, 256 MB RAM, 1 CPU, 64 pids, read-only rootfs, uid nobody, no-new-privileges — destroyed after every run; secrets never enter the container; a hard 20 s kill per step. The video shows a live containment moment: a hallucinated exfiltration snippet blocked pre-execution. On Vultr. The full backend runs on a Vultr Compute VM (Amsterdam): FastAPI, the Docker sandbox runner, Caddy/TLS. All LLM calls go to Vultr Serverless Inference — primary deepseek-v4.1-flash-1017, whole-step fallback retries on mimo-v2.6-flash. Business case. The author supports storage systems and virtualization fleets already off vendor support, where every incident is triaged manually: this cuts incident triage from ~30 min to ~60 s, with evidence attached.

Category tags: