Local OCR and RAG on AMD ROCm

Local OCR and RAG on AMD ROCm

Created by team Kerdosmedia on October 14, 2026

This project contains my submissions for Mini-Challenge 2 (OCR) and Mini-Challenge 3 (RAG) of the AMD AI League. Both run in Docker images built on the mandated ROCm PyTorch base image, with the model weights shipped inside the image, so no network access is needed at evaluation time. Both solutions share the same architecture: a long-running FastAPI server, started by the container, keeps Qwen3-VL-2B-Instruct loaded on the GPU, and app.py is a thin client that the harness calls once per image or question. The model is never reloaded between calls. For OCR, a short English prompt with few-shot examples reads licence plates and road signs, with an automatic denoising and contrast step for dark images. For RAG, a single indexing pass converts every file type (PDF, Word, Excel, CSV, text, logs, Python, and images through the vision model) into text, skipping unreadable, encrypted and unknown files without stopping. Retrieval uses IDF weighting, a filename bonus and rare-identifier links for multi-hop questions. Citations and refusals are enforced by code rather than left to the model: the answer must appear in a cited file, every number or identifier in the question must appear in the sources, withdrawn revisions are excluded, and a linked file is cited only when the question does not already provide its identifier.

Category tags: