Sensory Grounding

Sensory grounding research investigates how sensory experiences inform and constrain cognitive processes, particularly in artificial intelligence. Current efforts focus on developing methods to effectively ground language models in 3D environments, using techniques like reinforcement learning with privileged sensory information and incorporating commonsense knowledge to improve zero-shot object navigation. This work is crucial for advancing embodied AI, enabling robots and other agents to more robustly understand and interact with the physical world, and improving the reliability and interpretability of large language models.