Visually Grounded Reasoning
Overview
Visually grounded reasoning studies how models use images, video, diagrams, and other visual inputs to support conclusions and actions with relevant evidence. Common approaches include locating regions or pixels, building scene graphs and symbolic programs, interleaving visual inspection with language reasoning, and using active perception to seek additional views or details. Benchmarks test settings such as spatial and compositional questions, charts and diagrams, video events, and embodied planning, increasingly scoring evidence alignment as well as answer accuracy. A central challenge is that better perception does not guarantee that evidence will guide later reasoning: language priors and long reasoning traces can override visual details, while extra inspection or computation can be costly or ineffective.
Latest papers
344 papers24 papers in the last four weeks, up 41% on the four weeks before. 0.1% of all new papers.