VisionKnob is an interactive AI-powered image understanding system that combines physical hardware controls with computer vision and multimodal AI. The idea behind the project is to make image analysis more intuitive, private, and engaging by allowing users to interact with AI through a physical knob instead of relying only on a keyboard or mouse. The system uses an Arduino UNO Q and Modulino hardware as the physical interface. Users can turn or press the knob to navigate through images, select an image, and interact with the AI assistant. A Streamlit-based interface displays the selected image and allows the user to ask questions about what the image contains. VisionKnob uses a Gemma vision model to analyze the selected image and generate natural-language responses. For example, a user can ask the system to describe an image, identify important objects, explain what is happening in a scene, or answer specific questions about visual details. One of the main goals of VisionKnob is to explore how physical computing and generative AI can work together to create more natural human-computer interactions. Instead of AI existing only as a chatbot, VisionKnob connects AI with real-world hardware controls, making the interaction more tangible and accessible. The project also focuses on local and privacy-conscious AI processing. By using models that can run locally, sensitive images do not necessarily need to be uploaded to external cloud services. This makes VisionKnob useful for applications where privacy, offline access, or low-latency responses are important. VisionKnob demonstrates how multimodal AI, embedded hardware, and modern user interfaces can be combined into a practical system for visual exploration, education, accessibility, research, and future smart-device applications.
Category tags: