Multimodal Understanding

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, up 233% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 413

All topics

MMGist: A Comprehensive Multimodal Benchmark for 2027
Jun 21, 2026Wenzhen Yuan, Jiacheng Ruan, Wutao Xiong +3VLM Evaluation Large Vision-Language Models
1Shanghai Jiao Tong University · 2Sichuan University
BioMatrix: Towards a Comprehensive Biological Foundation Model Spanning the Modality Matrix of Sequences, Structures, and Language
Jun 20, 2026Qizhi Pei, Zhimeng Zhou, Yi Duan +9Unified Multimodal Models Multimodal Pretraining
1Gaoling School of Artificial Intelligence, Renmin University of China, 2OpenDataLab, Shanghai Artificial Intelligence Laboratory · 3Zhejiang University, 4Shanghai Innovation Institute · 2OpenDataLab, Shanghai Artificial Intelligence Laboratory, 3Zhejiang University, 4Shanghai Innovation Institute +5
CulMind: Benchmarking Multimodal Understanding and Reasoning in Chinese Cultural Heritage
Jun 19, 2026Zhangwei Cao, Shuhan Fan, Yuting Wei +5Benchmark Construction Reasoning Evaluation
University of International Relations · 2Kedge Business School · 3Peking University +3
Synergistic Dual-Branch Adaptation for Multi-modal Generalized Category Discovery
Jun 19, 2026Yuxun Qu, Minyu Zhou, Yongqiang Tang +2Cross-Modal Alignment Multimodal Classification
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 101408, China
CogniRoute: Learning to Route Social Evidence in Omni-Modal Models
Jun 18, 2026Yifan Shen, Pei Tian, Xinzhuo Li +8RL for Language Model Reasoning Mixture-of-Experts Inference
University of Illinois Urbana-Champaign · 2Columbia University · 3Google
ROSE: Benchmarking the Perception-to-Action Gap in Multimodal Models
Jun 18, 2026Yihao Wang, Zijian He, Jie Ren +1Multimodal Grounding Multimodal Reasoning
Sun Yat-sen University · Shaanxi Normal University
Timage: A Generative Text-in-Image Paradigm for Fine-Tuning Vision-Language Models
Jun 18, 2026Yifeng Wu, Huimin Huang, Ruiluo Wu +5Visual Spatial Reasoning Multimodal Large Language Models
Fudan University · Shenzhen University of Advanced Technology · Tencent Jarvis Lab +1
3D-PLOT-LLM: Part-Level Object Tokens for 3D Large Language Models
Jun 18, 2026Jintang Xue, Xinyu Wang, Yixing Wu +23D Representation Learning Multimodal Foundation Models
University of Southern California Los Angeles, California, USA · Ohio State University Columbus, Ohio, USA
SAFE-Cascade: Cost-Adaptive Vision-Language Routing for Chart Question Answering
Jun 17, 2026Ayush Dwivedi, Qixin Wang, Ashvi Soni +5Visual Question Answering Efficient VLM Inference
Walmart USA · University of Arkansas USA
Native Active Perception as Reasoning for Omni-Modal Understanding
Jun 17, 2026Zhenghao Xing, Ruiyang Xu, Yuxuan Wang +8Audio-Visual Understanding Active Perception
1The Chinese University of Hong Kong · †Work done during an internship at Qwen Team, Alibaba Group. · 2Shanghai Jiao Tong University +2
Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Games
Jun 17, 2026Shengyuan Ding, Xilin Wei, Xinyu Fang +4Multimodal Memory Sequential Decision Making
1Fudan University · 2Shanghai Innovation Institute · 3Shanghai Artificial Intelligence Laboratory +2
Seeing Before Reasoning: Decoupling Perception and Reasoning for Shortcut-Resilient Multimodal On-Policy Self-Distillation
Jun 17, 2026Sihan Wang, Xiyao Liu, Lianqing Liu +1Vision-Language Models On-Policy Self-Distillation
State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences
Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning
Jun 17, 2026Pengyu Li, Zhitao Gao, Lingling Zhang +5Cross-Modal Knowledge Distillation Efficient Multimodal Inference
Xi’an Jiaotong University · Shaanxi Province Key Laboratory of Big Data Knowledge Engineering · MOE KLINNS Lab, Xi’an Jiaotong University +1
CaVe-VLM-CoT: An Interpretable Vision-Language Model Framework
Jun 16, 2026Sneha Rao, Shaina Raza, Dhanesh RamachandramVLM Evaluation Vision-Language Grounding
Vector Institute, Toronto, ON M5G 0C6, Canada
Unified Multimodal Autoregressive Modeling with Shared Context-Visual Tokenizer is Key to Unification
Jun 16, 2026Wujian Peng, Lingchen Meng, Yuxuan Cai +7Unified Multimodal Models Autoregressive Image Generation
Institute of Trustworthy Embodied AI, Fudan University · Shanghai Innovation Institute · Qwen Team, Alibaba Inc.
MLLMs Get It Right, Then Get It Wrong: Tracing and Correcting Late-Layer Textual Bias
Jun 16, 2026Xingming Li, Ao Cheng, Qiyao Sun +4VLM Robustness Vision-Language Models
National University of Defense Technology, changsha, China · 2Chinese University of Hong Kong, Shenzhen, China · 3Intelligent Game and Decision Lab, Beijing, China
Not Truly Multilingual: Script Consistency as a Missing Dimension in VLM Evaluation
Jun 15, 2026Prabhjot Singh, Bhushan Pawar, Madhu Reddiboina +1VLM Evaluation Multilingual Language Models
RediMinds Inc., USA · The University of Texas at Austin, USA · Indian AI Research Organization (IAIRO), India
Gen-VCoT: Generative Visual Chain-of-Thought Reasoning via Diffusion-Based RGB Intermediate Representations
Jun 15, 2026Zhiqiang Zhou, Junliang Dai, Xu lingVLM Interpretability Multimodal CoT Reasoning
Hunan Chemical Industry Vocational and Technical College, Zhuzhou, Hunan 412000, China
SPICE: Synergy and Partial Information Based Curriculum Evolution
Jun 15, 2026Ankush Pratap Singh, Houwei Cao, Yong LiuCurriculum Learning Multimodal Learning
New York Institute of Technology · New York, NY, USA · New York University
LOCUS: Local Visual Cue Search for Enhancing Fine-Grained Perception in Multimodal Large Language Models
Jun 15, 2026Zhou Tao, Fang Zhang, Zewen Ding +5Multimodal Large Language Models Vision-Language Grounding
University of Science and Technology of China · State Key Laboratory of Cognitive Intelligence
MODE: Modality-Decomposed Expert-Level Mixed-Precision Quantization for MoE Multimodal LLMs
Jun 15, 2026Yuanteng Chen, Peisong Wang, Zhilei Liu +9Mixed-Precision Quantization LLM Quantization
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · 3Zhongguancun Academy
UniDDT: Unifying Multimodal Understanding and Generation with Decoupled Diffusion Transformer
Jun 15, 2026Shuai Wang, Liang Li, Yang Chen +3Diffusion Transformer Unified Multimodal Models
Nanjing University · ByteDance Seed · University of Hong Kong
VinQA: Visual Elements Interleaved Long-form Answer Generation for Real-World Multimodal Document QA
Jun 15, 2026Young Rok Jang, Hyesoo Kong, Kyunghwan An +3VLM Evaluation Multimodal Large Language Models
LG AI Research
Mind the Gap: Diagnosing Constraint Discovery Failures in Text-in-Image Editing
Jun 14, 2026Rui GuiText-Guided Image Editing Multimodal Large Language Models
Central South University
Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence
Jun 14, 2026Xuanle Zhao, Qiushi Sun, Jingyu Xiao +16Scientific Visualization Code Generation
Meituan · The University of Hong Kong · The Chinese University of Hong Kong +7
OmniOPSD: Rationale-Privileged On-Policy Self-Distillation for Affective Computing
Jun 14, 2026Zebang Cheng, Shuimu Chen, Boxue Yang +7On-Policy Self-Distillation Multimodal Large Language Models
Shenzhen University · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) · Tsinghua University +6
Mitigating Visual Hallucinations in Multimodal Systems through Retrieval-Augmented Reliability-Aware Inference
Jun 14, 2026Pratheswaran Hariharan, Haiping Xu, Donghui YanProbability Calibration Selective Prediction
Computer and Information Science University of Massachusetts, Dartmouth, MA · Mathematics and Data Science University of Massachusetts, Dartmouth, MA
Conditional Multi-Event Temporal Grounding in Long-Form Video
Jun 13, 2026Yuanhao Zou, Arthad Kulkarni, Lucas Tonanez +12Temporal Video Grounding Long-Video Understanding
University of Central Florida · 2Qualcomm AI Research
Visual-Seeker: Towards Visual-Native Multimodal Agentic Search via Active Visual Reasoning
Jun 13, 2026Zhengbo Zhang, Changtao Miao, Jinbo Su +10Multimodal IR Multimodal Grounding
School of Artificial Intelligence, UCAS · Institute of Automation, CAS · Ant Digital Technologies, Ant Group +2
DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
Jun 13, 2026David Huang, Lianlei ShanEfficient Multimodal Inference Multimodal Reasoning
University of Toronto · Tsinghua University

Jun 21, 2026, cs.CVJ/K move · Enter open · S save

MMGist: A Comprehensive Multimodal Benchmark for 2027

Wenzhen Yuan, Jiacheng Ruan, Wutao Xiong, Chengping Zhao+2

1Shanghai Jiao Tong University · 2Sichuan University

VLM Evaluation Large Vision-Language Models Multimodal Foundation Models Multimodal Learning Multimodal Understanding

Open paper AskPDF arXiv

We conduct a systematic study of 18 widely used vision-language benchmarks and identify three major issues: 1) many items do not rely on visual cues and therefore fail to effectively measure multimodal understanding; 2) many items are already close to performance saturation for current LVLMs, which limits their discriminative power; 3) a small number of anomalous items affect the reliability of evaluation results. To this end, we propose MMGist, a curated benchmark that covers seven capability dimensions and contains 7,262 items. MMGist is constructed through a three-stage pipeline, which sequentially combines text-ablation filtering, cross-model saturation filtering, and anomaly detection filtering. We conduct extensive experiments on 27 leading LVLMs and compare MMGist with the raw pool of 23,250 items. The results show that MMGist preserves model rankings with high fidelity, with Spearman

ρ= 0.98

, while reducing evaluation items by 69% and improving cross-model discrimination by 78%. Further results indicate that Visual Logic remains a systematic weakness of current LVLMs, while knowledge-intensive dimensions such as Expert Knowledge dimensions remain important factors for distinguishing closed-source models from open-source models. These findings suggest that high-quality evaluation should prioritize visual dependency, discriminative power, and reliability, rather than simply pursuing benchmark scale.

Momentum

Related topics

Latest papers 413