Multimodal Search Agents
Overview
Multimodal search agents use tools such as web, image, video, and document search to gather evidence and answer questions that require multiple retrieval steps. Common approaches combine visual-language models with retrieval and browsing workflows, then improve tool use through supervised trajectory training, reinforcement learning, process rewards, and memory or evidence graphs. Benchmarks such as MM-BrowseComp, InfoSeek, and MMSearch test retrieval, planning, visual grounding, and evidence integration, with newer evaluations also measuring effort and trajectory quality. Despite progress, agents still struggle to find relevant evidence efficiently, verify it across modalities, and avoid errors from conflicting or weakly grounded sources.
Latest papers
47 papers9 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.