Overview

The field studies how AI can combine evidence from different kinds of input to answer questions, recognize events, retrieve information, and reason about scenes over time. Common approaches include vision-language models and multimodal large language models, trained with contrastive alignment, cross-attention, instruction tuning, and reinforcement learning; some systems add modality-specific experts, retrieval, or tools. Benchmarks test image and video question answering, document and chart understanding, audio-visual reasoning, grounding, and performance across languages and domains. A central challenge is ensuring that answers genuinely depend on relevant evidence from each modality, while handling long inputs efficiently and making capabilities measurable with reliable tests.

Latest papers

CardsList

489 papers13 papers in the last four weeks, down 7% on the four weeks before. 0.1% of all new papers.

  1. How Well Do Vision-Language Models Understand Movies?

    Jun 6, 2025Emmanouil Zaranis, António Farinhas, Saul Santos +28Video-Language Model EvaluationMultimodal Understanding

  2. Mitigating Task Objective Conflicts for Continual Learning in Unified Multimodal Models

    Jun 4, 2025Jiaxing Zhang, Hao TangTask InterferenceUnified Multimodal Models

  3. LaViDa: A Large Diffusion Language Model for Multimodal Understanding

    May 22, 2025Shufan Li, Konstantinos Kallidromitis, Hritik Bansal +7Multimodal UnderstandingVision-Language Models

  4. HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluation

    May 16, 2025Shaina Raza, Aravind Narayanan, Vahid Reza Khazaie +6Algorithmic FairnessMultimodal Large Language Models

  5. ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models

    Feb 13, 2025Jonathan Roberts, Mohammad Reza Taesiri, Ansh Sharma +31Visually Grounded ReasoningVLM Reasoning

  6. MultiViewDx: Evidence-Linked Multi-View Clinical Diagnosis

    Oct 19, 2024Junda Wang, Zonghai Yao, Yujan Ting +5HealthcareMedical Image Analysis

  7. A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Tasks

    Aug 2, 2024Jiaqi Wang, Hanqi Jiang, Yiheng Liu +23Multimodal Large Language ModelsMultimodal Understanding