MergeProof

# MergeProof: Evidence-First AI Pull Request Review Software teams merge hundreds of pull requests every week, yet today's AI review tools often generate vague, ungrounded comments. They hallucinate file names, invent line numbers, and assign severity ratings without evidence. When an AI reports "there may be a security issue," developers have no reliable way to determine whether to block the PR or ignore the warning. The core problem is that AI pull request reviews lack verifiable evidence. MergeProof solves this by making evidence mandatory. Every finding must include six required fields: - **Severity:** Critical, High, Medium, Low, or Info - **Category:** Correctness, Security, Performance, or Test Coverage - **File** - **Line Number** - **Verbatim Evidence** copied directly from the diff - **Concrete Recommendation** These outputs are enforced with strict **Pydantic schemas** (`extra="forbid"`). Any hallucinated file path, invalid line number, missing field, or unexpected output fails validation, ensuring every finding is grounded in the submitted diff. ## Workflow MergeProof reviews pull requests through a five-stage pipeline. **1. Parallel Expert Review** Four specialized AI reviewers execute in parallel using `ThreadPoolExecutor`, focusing on: - Correctness - Security - Performance - Test Coverage Each reviewer uses a tightly constrained system prompt requiring structured JSON, verbatim evidence, and prohibiting fabricated paths or unsupported claims. **2. Coordination & Risk Assessment** A coordinator deduplicates findings, calculates an overall repository risk level (Critical–Low), and generates an executive summary. **3. Automated Patch Generation** Using the validated findings, MergeProof generates a unified patch addressing every reported issue. **4. Proof Through Testing** `pytest` runs before and after applying the patch. The first run confirms the reported defects exist; the second verifies the generated fixes. This converts AI suggestions into evidence-backed improvements. **5. Before-and-After Verification** The reviewers analyze the patched code again, reporting resolved findings, remaining issues, and the overall reduction in project risk. ## Hackathon Demonstration The demo project is a deliberately vulnerable Python calculator containing: - A division-by-zero crash - An `eval()` arbitrary code execution vulnerability - An O(n) implementation where an O(1) solution is trivial MergeProof detects all issues, generates fixes, validates them through automated testing, and performs a second review to measure improvement. | Metric | Before | After | |--------|--------|-------| | Passing Tests | 9 | 18 | | Failing Tests | 1 | 0 | | Overall Risk | Critical | Low | | Findings Resolved | — | 8 | | Remaining Issues | — | 1 minor documentation gap | ## Deployment MergeProof runs entirely in **DEMO_MODE** without an API key using precomputed findings clearly labeled as synthetic. With an OpenAI API key, it performs live analysis using **GPT-4o mini**, executing four reviewers in parallel and completing a review in under **15 seconds**. No authentication, database, or infrastructure is required. The complete application launches with a single command: ```bash streamlit run app.py ``` MergeProof demonstrates that AI-powered code review can be evidence-backed, schema-validated, test-verified, and directly actionable rather than relying on unverified AI suggestions.