🤓 Latest Submissions
.png&w=828&q=75)
Grounded Multimodal QA on ROCm
GroundTruth RAG answers questions over a messy folder of enterprise documents and returns only the value plus the exact files it came from, running entirely on an AMD GPU with ROCm. Indexing: every file is parsed in its own isolated process with a timeout, so an unreadable file, an encrypted PDF, an unknown binary or a corrupt document is recorded and skipped instead of breaking the walk. We handle PDF, DOCX, XLSX (all sheets), CSV, logs and source code natively. Images and scanned pages are transcribed by Qwen2.5-VL-7B on the GPU. Serving: because every question runs as a new process, indexing launches a resident model server that loads the model onto the GPU once and warms it up. Each query is a lightweight client over a unix socket, answering in a few seconds. Answering: retrieval combines BM25 with automatic handling of withdrawn and superseded documents, plus an identifier hop that follows ticket and part numbers across files for multi-step questions. The model proposes an answer, then deterministic code takes over: values that do not appear in the corpus are refused, a verification pass checks the answer matches exactly what was asked, and citations are chosen by necessity rather than relevance. When the corpus does not contain the answer, the system returns an empty answer instead of guessing. Everything ships inside the image with no network access at runtime: one 7B model, about 46 GiB total.
1 Dec 2026
