Multimodal Large Language Models

Also known as MLLM

Follow this topic Ask about this topic

Momentum

67 papers in the last four weeks, up 91% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 676

All topics

DynFrame: Adaptive Reasoning-Driven Multimodal Framework with Dynamic Frame Augmentation for Complex Video Understanding
May 26, 2026Peng Zhang, Guanghao Zhang, Wanggui He +10Multimodal Large Language Models Video Reasoning
Zhejiang University · Alibaba Group
Not All Modalities Are Equal: Instruction-Aware Gating for Multimodal Videos
May 25, 2026Bonan Ding, Umair Nawaz, Ufaq Khan +5Cross-Modal Learning Multimodal Large Language Models
Mohamed bin Zayed University of Artificial Intelligence · Tianjin University · Chongqing University +1
Prism: A Plug-in Reproducible Infrastructure for Scalable Multimodal Continual Instruction Tuning
May 25, 2026Jun-Tao Tang, Yu-Cheng Shi, Zhen-Hao Xie +1Continual Learning for LLMs Multimodal Large Language Models
National Key Laboratory for Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China
Visual-Redundancy-Controlled Parallel Decoding for Diffusion-Based Multimodal Large Language Models
May 25, 2026Yulin Yuan, Hongshuo Zhao, Xiangming MengLanguage Model Decoding Multimodal Large Language Models
Zhejiang University · ZJUI-UIUC Institute
AnE: Pushing the Reasoning Frontier of Multimodal LLMs via Anchor Evolution
May 25, 2026Zehao Wang, Yihan Zeng, Zidong Gong +5RL for Language Model Reasoning Multimodal Large Language Models
Harbin Institute of Technology · Huawei Noah’s Ark Lab · Independent Researcher
StructBreak: Structural Cognitive Overload-Induced Safety Failures in MLLMs
May 25, 2026Yang Luo, Xinran Liu, Tiantian Ji +3Adversarial Attacks on VLMs Language Model Safety Evaluation
Key Laboratory of Trustworthy Distributed Computing and Service (MoE), Beijing University of Posts and Telecommunications · Institute of Computing Technology, Chinese Academy of Sciences
MetaphorVU: Towards Metaphorical Video Understanding
May 25, 2026Zhuoqun Li, Boxi Cao, Guiping Jiang +13VLM Evaluation Figurative Language Understanding
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Kuaishou Technology +1
BigMac: Breaking the Pareto Frontier of Compute and Memory in Multimodal LLM Training
May 25, 2026Zili Zhang, Chengxu Yang, Shenglong Zhang +8Deep Learning Optimization Multimodal Large Language Models
1Peking University · †Work done during internships at Xiaohongshu, Inc. · 2Independent Researcher +2
Dual-Pathway Geometry-Aware MLLM for Spatial Intelligence
May 25, 2026Yufei Zheng, Xuhan Zhu, Zide Liu +9Visual Spatial Reasoning Metric Depth Estimation
University of Science and Technology of China · Li Auto Inc. · Shanghai Jiao Tong University
Divide-and-Conquer Inference for Large-Scale Visual Recognition with Multimodal Large Language Models
May 24, 2026Zhipeng Ye, Jiaqi Huang, Feng Jiang +5Efficient Multimodal Inference Multimodal Large Language Models
Taizhou Institute of Science and Technology, Nanjing University of Science and Technology, Taizhou, 225300, Jiangsu, China · Department of Intelligence Science, Xi’an Jiaotong-Liverpool University, Suzhou, 215123, Jiangsu, China · Department of Statistical Sciences, University of Toronto, Toronto, M5S 1A1, Ontario, Canada +1
EgoProx: Evaluating MLLMs on Egocentric 3D Proximity Reasoning Across a Cognitive Hierarchy
May 23, 2026Jinzhao Li, Yinuo Chen, Dongxu Piao +9Spatial Reasoning Benchmarks LLM Evaluation
College of AI, Tsinghua University · 2ByteDance · 3State Key Laboratory of General Artificial Intelligence, BIGAI
Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignment
May 22, 2026Haoyuan Wang, Xiaohao Liu, Jiajie Su +2Multimodal Robustness Adversarial Robustness
Zhejiang University · National University of Singapore · Jiangxi Normal University
ChartFI: Benchmarking Faithfulness and Insightfulness of Chart Descriptions from Multimodal Large Language Models
May 22, 2026Fen Wang, Zekai Shao, Qiman Kang +5Data Visualization Multimodal Large Language Models
School of Data Science, Fudan University · Zhengzhou Zhongke Institute of Integrated Circuit and System Application · School of Computer Science, Fudan University
From Recognition to Reasoning: Benchmarking and Enhancing MLLMs on Real-World Receipt Document Understanding
May 21, 2026Yandi Wang, Libin Zhan, Ziwei Huang +5Document Understanding Document Information Extraction
Zhejiang University, China
Enhancing Multimodal Large Language Models for Safety-Critical Driving Video Analysis
May 21, 2026Tomaso Trinci, Henrique Piñeiro Monteagudo, Leonardo TaccariMultimodal Large Language Models Autonomous Driving
Verizon Connect, Florence, Italy.
Do Factual Recall Mechanisms Carry over from Text to Speech in Multimodal Language Models?
May 21, 2026Luca Modica, Filip Landin, Mehrdad Farahani +3Multimodal Large Language Models Factual Knowledge in Language Models
Zenseact · Chalmers University of Technology · University of Gothenburg +2
Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attention
May 21, 2026Changyuan Tian, Zhicong Lu, Huaxing Liu +7Multimodal Large Language Models Multimodal Reasoning
AMAP, Alibaba Group · University of Chinese Academy of Sciences · Tsinghua University +1
LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning
May 21, 2026Yifan Dai, Zhenhua Wu, Bohan Zeng +18Cross-Modal Learning Audio-Visual Understanding
School of AI, Shanghai Jiao Tong University · 2Kling Team, Kuaishou Technology · 3Peking University +5
MLLMs Know When Before Speaking: Revealing and Recovering Temporal Grounding via Attention Cues
May 21, 2026Dazhao Du, Liao Duan, Jian Liu +5Temporal Video Grounding Multimodal Large Language Models
Hong Kong University of Science and Technology · Tencent · Xi’an Jiaotong University
Seizure-Semiology-Suite (S3): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology Understanding
May 21, 2026Lina Zhang, Tonmoy Monsoor, Peizheng Li +23Video Understanding Multimodal Large Language Models
University of California, Los Angeles · Mercedes-Benz AG · University of Pittsburgh +4
Multimodal LLMs under Pairwise Modalities
May 20, 2026Yan Li, Yunlong Deng, Yuewen Sun +3Cross-Modal Learning Representation Identifiability
Mohamed bin Zayed University of Artificial Intelligence · Carnegie Mellon University
Frequency-Domain Regularized Adversarial Alignment for Transferable Attacks against Closed-Source MLLMs
May 20, 2026Leitao Yuan, Qinghua Mao, Daizong Liu +5Adversarial Attacks on VLMs Multimodal Large Language Models
Shanghai Artificial Intelligence Laboratory · Zhejiang University · Shanghai Jiao Tong University +3
Miller-Index-Based Latent Crystallographic Fracture Plane Reasoning and generation with Vision-Language Models
May 19, 2026Qinwu Xu, Xiaofu Ma, Yifan JiangMultimodal Large Language Models Multimodal Reasoning
Independent research
Stage-adaptive Token Selection for Efficient Omni-modal LLMs
May 19, 2026Zijie Xin, Jie Yang, Ruixiang Zhao +4Efficient Multimodal Inference Multimodal Large Language Models
Renmin University of China · WeChat Vision, Tencent Inc.
A Nash Equilibrium Framework For Training-Free Multimodal Step Verification
May 19, 2026Rohit Sinha, Kunal Tilaganji, Tanuja Ganu +3Multi-View Consistency Game Theory
1Microsoft Research India · 2Indian Institute of Technology Hyderabad
AffectVerse: Emotional World Models for Multimodal Affective Computing
May 19, 2026Bo Zhao, Fanghua Ye, Yixin Ji +3Cross-Modal Learning Multimodal Large Language Models
1Great Bay University · 2Tencent · 3Tsinghua University +1
Are Tools Always Beneficial? Learning to Invoke Tools Adaptively for Dual-Mode Multimodal LLM Reasoning
May 19, 2026Qinghe Ma, Zhen Zhao, Yiming Wu +3Multimodal Large Language Models Multimodal Reasoning
The State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China · Shanghai Artificial Intelligence Laboratory +2
LMM-Track4D: Eliciting 4D Dynamic Reasoning in LMMs via Trajectory-Grounded Dialogue
May 19, 2026Chaoyue Li, Yongxue Xu, Jie Feng +1Spatiotemporal Reasoning Multimodal Large Language Models
Huazhong University of Science and Technology · Sun Yat-sen University · Beihang University +1
MetaRA: Metamorphic Robustness Assessment for Multimodal Large Language Model-based Visual Question Answering Systems
May 19, 2026Quanxing Xu, Yuhao Tian, Ling Zhou +4Visual Question Answering Automated Software Testing
School of Computer Science and Engineering, Macau University of Science and Technology, Macao SAR 999078, China · Hubei Key Laboratory of Transportation Internet of Things, School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan 430070, China · State Key Laboratory of Maritime Technology and Safety, Wuhan University of Technology, Wuhan 430063, China +3
Modality-Decoupled Online Recursive Editing
May 19, 2026Siyuan Li, Youyuan Zhang, Fangming Liu +1Continual Learning for LLMs Multimodal Large Language Models
Harbin Institute of Technology, Shenzhen, China. · Peng Cheng Laboratory, China. · Huazhong University of Science and Technology, China.

May 26, 2026, cs.CVJ/K move · Enter open · S save

DynFrame: Adaptive Reasoning-Driven Multimodal Framework with Dynamic Frame Augmentation for Complex Video Understanding

Peng Zhang, Guanghao Zhang, Wanggui He, Longxiang Zhang+9

Zhejiang University · Alibaba Group

Multimodal Large Language Models Video Reasoning Adaptive Sampling Multimodal Foundation Models RL for VLMs Temporal Video Understanding+1

Open paper AskPDF arXiv

Recent video multimodal large language models (MLLMs) increasingly couple step-by-step reasoning with on-demand visual evidence retrieval, allowing models to revisit relevant video segments during inference. However, two structural gaps remain in existing thinking-with-video systems. (i) Sampling density is not a learnable decision: existing methods may let the model decide where to look, but the per-window frame rate is largely fixed. As a result, fine-grained evidence is often recovered through repeated retrieval calls, which increases inference context length and training difficulty. (ii) Retrieval and answer generation are usually optimized with a single trajectory-level advantage, so the "where to look" tokens and the "how to answer" tokens receive the same credit even when one is correct and the other is not. To address these gaps, we present DynFrame, a framework that emits the temporal window and the sampling density as native tokens within a single autoregressive pass. This learnable span-density retrieval enables acquiring multi-granularity evidence with a single retrieval step. Based on the above tokenized retrieval interface, we further introduce Segment-Decoupled GRPO (SD-GRPO), which splits each rollout at the retrieval boundary and assigns role-specific token-level advantages, separately crediting the sampling decision and the answer. Trained on the curated DM-CoT-74k and DM-RL-45k, DynFrame-4B is competitive with strong 7B-8B baselines across six benchmarks (NExT-GQA, Charades-STA, ActivityNet-MR, Video-MME, MLVU, LVBench), and DynFrame-8B sets new state-of-the-art on most metrics. Code is available at https://github.com/zhangguanghao523/DynFrame.

Momentum

Related topics

Latest papers 676