Smart_route_ai

Smart Route AI is a hybrid LLM routing agent designed to optimize inference cost without sacrificing response quality. Instead of sending every prompt to a cloud-hosted model, the system first performs lightweight local task classification and complexity estimation using deterministic heuristics. Simple tasks such as factual questions, summarization, sentiment analysis, and named entity recognition are answered locally using Qwen2.5-0.5B-Instruct, while more demanding tasks like code generation, mathematical reasoning, and logical reasoning are routed to a Fireworks-hosted LLM. The routing engine evaluates prompt type, complexity, length, and formatting requirements to make explainable routing decisions with zero remote token overhead during classification. This approach aligns with the goal of minimizing API usage while maintaining high-quality responses for complex tasks. The project includes a responsive Streamlit interface that visualizes routing decisions, an explainable complexity score, and the selected inference path. It is fully Dockerized, supports environment-based configuration, and is compatible with the AMD Hackathon evaluation harness through dynamic ALLOWED_MODELS support. By combining efficient local inference with selective remote execution, Smart Route AI demonstrates a practical and scalable approach to token-efficient hybrid AI systems.