Mini Challenge 2: OCR
Image: b3panda3/optiseek:latest
Optiseek extracts text from PNG/JPEG/TIFF license plates and traffic signs using Qwen2.5-VL-3B-Instruct on AMD ROCm, returning strict JSON with text and confidence.
Categories: US/Chinese plates, stop signs, speed limits, advisory speeds, and work zone signs. Preserves relevant text while filtering US state names and slogans.
Architecture: PIL adaptive preprocessing → Qwen2.5-VL (bf16) → rule-based postprocessing → JSON.
Results: 10/10 samples passed (200/200 points); 17.2s/image average; ~8 GiB peak VRAM; ~25 GiB Docker image.
Engineering: Patched Transformers caching_allocator_warmup for MI300X VF memory limits; used a 3B model for VF compatibility.
Mini Challenge 3: RAG
Image: b3panda3/optiseek-rag:latest
Optiseek RAG indexes PDF, DOCX, XLSX, CSV, TXT, LOG, PY, PNG, and JPG files. It answers questions with exact citations and abstains when evidence is missing. VLM extraction handles text in diagrams, pinout charts, and asset labels. Robustly handles unreadable files, encrypted PDFs, empty directories, and unknown formats.
Architecture: Resident Qwen2.5-7B daemon + Unix-socket client → multi-format parsers → BGE-small embeddings + hybrid semantic/keyword retrieval → LLM generation → post-hoc citation necessity verification.
Results: 8/10 questions passed (160/200 points). Supports single-hop, multimodal, and refusal tasks. Multi-hop retrieval and one error-code extraction need improvement.
Engineering: Daemon avoids model reloads; VLM unloads after indexing to free VRAM; citation verification tests necessity by removing sources and rerunning queries. Patched caching_allocator_warmup and pinned ROCm-compatible PyTorch to prevent CUDA replacement. ~50 GiB Docker image; ~14 GiB peak VRAM.
.png&w=256&q=75)
.png&w=640&q=75)