Dobbies

Dobbies is an automated security audit platform purpose-built for AI Agents. As organizations increasingly deploy autonomous LLM agents that read databases, call APIs, execute code, and make business decisions independently, a critical gap has emerged: there is no standardized way to verify these agents are secure before production deployment. Dobbies addresses this gap with a two-phase audit pipeline. First, a two-layer static analysis scans agent code and system prompts — starting with 8+ dangerous regex patterns (shell execution, destructive operations, SQL injection) followed by an LLM-powered semantic evaluation that understands context and assigns CVSS scores per vulnerability. Second, an AI-vs-AI red-teaming phase simulates real attacks: for each detected OWASP category, an Attacker LLM probes the agent across multiple turns while an Agent LLM responds exactly as the real agent would, with an Evaluator LLM scoring every interaction. Dobbies currently covers the three most critical OWASP LLM Top 10 categories — Prompt Injection (LLM01), Sensitive Information Disclosure (LLM02), and Excessive Agency (LLM06) — with a modular architecture designed to expand to all ten. The output is a comprehensive security report with per-line vulnerability details, CVSS scores, exploit conversation transcripts, and ready-to-deploy guardrail configurations (Llama Guard rules, NeMo Guardrails config, Regex output filters). A full audit completes in under 45 seconds. Dobbies was built for the AMD Developer Hackathon: ACT II — Unicorn Track, running entirely on Fireworks AI inference optimized on AMD Instinct MI300X clusters. The tech stack includes Next.js 16, Supabase + PostgreSQL, and a custom LLM Gateway orchestrating four distinct model roles.