Multimodal Grounding

Multimodal grounding studies how AI systems connect words and claims to evidence in images, video, audio, documents, and physical environments. Common approaches include predicting boxes or masks, aligning attention with visual regions, building object or scene graphs, and retrieving evidence for step-by-step reasoning; newer systems also use tools and verifiable rewards to check those links. The work supports applications such as visual question answering, video search, document analysis, GUI interaction, and robotics, where identifying the right evidence matters as much as producing a plausible answer. A central challenge is that fluent, accurate-looking answers may not depend on the relevant evidence, while precise spatial and temporal grounding and reliable use of multiple modalities remain difficult to evaluate and improve.