Compass an AI-Assisted PR Context Retriev

A pull request shows what changed, but not necessarily the context needed to review it. Engineers still need to reconstruct intent, locate relevant code and tests, and decide where to focus—especially in an unfamiliar repository. Generic AI summaries can simply repeat the diff or introduce unsupported claims. Compass is a GitHub Action powered by IBM Bob that prepares a concise, source-linked starting point for human review. North, our polar-bear guide, represents orientation rather than autopilot. For eligible pull-request events, Compass collects a bounded bundle containing the PR description, changed-file patches, selected source excerpts and project documentation. Bob analyzes this context in a restricted, non-interactive session. Compass then validates Bob's structured response and source identifiers before creating or updating a bot-owned comment containing: 1. Purpose — the change's intent, labeled declared, inferred or unknown. 2. Relevant context — up to three short statements linked to collected evidence. 3. Suggested reading order — up to three source locations and why reviewers should start there. Compass does not approve, merge or edit pull requests. Engineers remain responsible for validating the evidence and making review decisions, while existing CI and review tools continue to operate alongside it. We demonstrated Compass on Shelfie, an original sample service. PR #2 changes 13 files to introduce late fees. Compass identified three useful reading locations and surfaced how a date-helper change affected existing due-soon badges—a consequence not explicitly stated in the PR description. We manually verified the relationship against the source code and architecture documentation. Across four observed live runs, two succeeded, with successful workflows completing in 36 and 46 seconds. These are observed workflow durations, not reviewer time-savings measurements. An earlier generated comment also contained an incorrect claim about zero-fee loans. We disclose this because valid structure and citations do not guarantee factual accuracy; human review remains necessary. The submitted implementation passed 257 offline source tests and 10 packaged checks. Runtime controls include disabled Bob tools, MCP and native subagents; isolated configuration; an environment allowlist; and keeping the GitHub publishing token outside Bob's process. The demonstrated workflow requests four turns and a 0.5-Bobcoin limit, although actual billing must be verified with IBM. Invalid model output is reported as unavailable rather than silently converted into a successful brief. IBM Bob serves as Compass's default analysis engine and was also used during development. The GitHub Action invokes Bob Shell with structured JSON output and an Inference-scope API key stored in GitHub Secrets. Selected code and documentation are sent to Bob for analysis. Bob returns explanations and source IDs, while Compass itself supplies provenance information such as commit metadata, timestamps and source records. Zod validates the response schema and citation references before deterministic application code publishes the result. During development, Bob IDE helped scaffold the prototype and harden the model-output contract. One recorded Bob task added stricter output validation, host-owned provenance, rejection of JSON extracted from error prose and regression tests. Its screenshot records 3.18 Bobcoins; this represents that development task, not the cost of each Compass run. Bob-assisted work was combined with manual implementation, review and testing. Our public website demonstrates the Compass experience and repository setup but does not collect model keys or provide hosted account onboarding. Future work includes stronger context selection, broader repository support and independent evaluation of reviewer effort. Compass is not an approval engine. It helps reviewers find their bearings, then leaves the call to them.