Local AI Secretary on Snapdragon X Elite

We built a fully local AI Secretary for Snapdragon X Elite and optimized the complete inference path rather than chasing raw tokens per second. The system turns natural-language requests into deterministic structured actions such as list_files, read_file, search_files, move_file, and clarify, while keeping inference on-device. Our implementation evaluates multiple execution paths for Qwen3-0.6B: llama.cpp on CPU, llama.cpp on HTP, and Qualcomm’s native QAIRT/GenieX path on the Hexagon NPU. We created a frozen 50-case Secretary benchmark and measure end-to-end task latency, TTFT, prefill and decode throughput, correctness, memory, and energy per completed task. This lets us compare backends on the metric that matters to a real assistant: correct task completion, not synthetic throughput alone. We also designed the system around three operating modes—FAST, BALANCED, and EFFICIENT—and investigated task-aware routing so deterministic requests can bypass the LLM and harder requests can use the appropriate model/runtime. For structured reliability, we focused on constrained outputs, short tool-call generation, and deterministic scoring. The project demonstrates a practical model-to-device optimization workflow on Snapdragon X Elite: benchmark, profile, compare CPU/HTP/NPU execution, preserve correctness, and select the best path for each workload.