Team Idea
Episodic Memory for Physical AI
We’re building a robotic agent that can act on experiences it never directly observed. A wearable camera (Mentra glasses) continuously captures a user’s first-person interactions with objects and builds an object-centric episodic memory of what they used, when, and in what context.
Later, a robot can receive natural-language requests such as, “Bring me the tool I was using when we were debugging the camera.” The system retrieves the relevant past episode, identifies the object from the wearable’s egocentric perspective, reconciles that memory with the robot’s current view of the environment, estimates its present location, and physically retrieves it.
Our goal is to explore whether continuous egocentric observation can provide persistent memory to physical agents.
Team Members
Carl Vincent Kho
Charvalton
Student
