Multimodal Large Language Models

Also known as MLLM

Follow this topic Ask about this topic

Momentum

67 papers in the last four weeks, up 91% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 676

All topics

Robust-U1: Can MLLMs Self-Recover Corrupted Visual Content for Robust Understanding?
Jun 6, 2026Jiaqi Tang, Jianmin Chen, Youyang Zhai +6VLM Robustness Image Corruption Robustness
The Hong Kong University of Science and Technology, Hong Kong · Northwestern Polytechnical University, Xi’an, China · Northeastern University, Shenyang, China +2
Seeing Without Exposing: Adaptive Privacy Control for Open-World, Context-Hungry MLLMs
Jun 5, 2026Siyuan Xu, Yibing Liu, Peilin Chen +3Privacy-Preserving Language Models Multimodal Large Language Models
City University of Hong Kong · Hon Hai Research Institute · Lingnan University
Textual Supervision Enhances Geospatial Representations in Vision-Language Models
Jun 5, 2026Marcelo Sartori Locatelli, Fernando Tonucci, Jea Kwon +5Vision-Language Models Geospatial Representation Learning
Max Planck Institute for Security and Privacy (MPI-SP), Bochum, Germany · Universidade Federal de Minas Gerais, Belo Horizonte, Brazil · Changwon National University, Changwon, South Korea +1
When Recovery Matters: The Blind Spot of Surrogate Privacy in MLLM Editing
Jun 5, 2026Siyuan Xu, Yibing Liu, Peilin Chen +3Image Editing Evaluation Text-Guided Image Editing
1City University of Hong Kong, Hong Kong. · 2Hon Hai Research Institute, Taipei. · 3Lingnan University, Hong Kong.
Evaluating Stochastic Collapse and Implicit Bias in Multimodal Large Language Models
Jun 4, 2026Huiyuan Zheng, Houtao Zhang, Boyang Wang +2Multimodal Large Language Models Language Model Bias Evaluation
Fudan University · Beihang University
Mechanistic Insights into Functional Sparsity in Multimodal LLMs via CoRe Heads
Jun 4, 2026Ruoxi Sun, Quantong Qiu, Juntao Li +3Multimodal Large Language Models VLM Interpretability
Soochow University · Peking University
Learning Geometric Representations from Videos for Spatial Intelligent Multimodal Large Language Models
Jun 4, 2026Haibo Wang, Lifu Huang3D Spatial Reasoning Multimodal Large Language Models
University of California, Davis
Beyond Generative Decoding: Discriminative Hidden-State Readout from a Native Omni-Modal LLM for Multimodal Sentiment Analysis
Jun 4, 2026Bin Wen, Tien-Ping TanEfficient Multimodal Inference Sentiment Analysis
School of Computer Sciences, Universiti Sains Malaysia, Penang, Malaysia.
LongSpace: Exploring Long-Horizon Spatial Memory from Perception to Recall in Video
Jun 4, 2026Shiqiang Lang, Jing Liu, Haoyang He +6Visual Spatial Reasoning Multimodal Large Language Models
Beijing University of Posts and Telecommunications · Zhongguancun Academy · Institute of Automation, Chinese Academy of Sciences +2
WorldBench: A Challenging and Visually Diverse Multimodal Reasoning Benchmark
Jun 4, 2026Yida Yin, Harish Krishnakumar, Chung Peng Lee +9Multimodal Large Language Models Multimodal Reasoning
Princeton University · NYU · University of Waterloo +1
Brick-Composer: Using MLLMs for Assembly with Diverse Bricks
Jun 3, 2026Jiateng Liu, Bingxuan Li, Zhenhailong Wang +8Visual Spatial Reasoning Benchmark Construction
UIUC1 · Stevens Institute of Technology2 · Northwestern University3
UniCAD: A Unified Benchmark and Universal Model for Multi-Modal Multi-Task CAD
Jun 3, 2026Jingyuan Chen, Sheng Jin, Haopeng Sun +23D Reconstruction Multimodal Large Language Models
SenseTime Research and Tetras.AI
Query-based Cross-Modal Projector Bolstering Mamba Multimodal LLM
Jun 3, 2026SooHwan Eom, Jay Shim, Gwanhyeong Koo +4Mamba Efficient VLM Inference
Korea Advanced Institute of Science and Technology / Korea, Republic of · University of Illinois in Urbana-Champaign / United States of America · Korea University / Korea, Republic of
A Systematic Evaluation of Positional Bias in Multi-Video Summarization with MLLMs
Jun 3, 2026Huangchen Xu, Yuan Wu, Yi ChangMultimodal Large Language Models Multimodal Model Evaluation
School of Artificial Intelligence, Jilin University · 2Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, Jilin University · 3International Center of Future Science, Jilin University
Hyper-ICL: Attention Calibration with Hyperbolic Anchor Distillation for Multimodal In-Context Learning
Jun 3, 2026Niloufar Alipour Talemi, Hossein Kashiani, Fatemeh AfghahEfficient Multimodal Inference Multimodal ICL
Holcombe Department of Electrical and Computer Engineering, Clemson University, Clemson, United States.
From Symbolic to Geometric: Enabling Spatial Reasoning in Large Language Models
Jun 3, 2026Chen Chu, Bita Azarijoo, Li Xiong +2Spatial Reasoning Benchmarks Multimodal Large Language Models
University of Southern California Los Angeles, CA, USA · Emory University Atlanta, GA, USA · Novateur Research Solutions Ashburn, Virginia, USA
MorphoQuant: Modality-Aware Quantization for Omni-modal Large Language Models
Jun 3, 2026Yue Wu, Changyuan Wang, Zixuan Wang +2LLM Quantization 4-Bit Quantization
FindIt: A Format-Informed Visual Detection Benchmark for Generalist Multimodal LLMs
Jun 2, 2026Eshika Khandelwal, Jingjing Pan, Mingfang Zhang +3Multimodal Grounding Video Object Detection
Tuebingen AI Center, University of Tuebingen · Woven by Toyota, Inc., Tokyo, Japan · Toyota Motor Europe, Brussels, Belgium
Benchmarking Visual State Tracking in Multimodal Video Understanding
Jun 2, 2026Sihyun Yu, Nanye Ma, Pinzhi Huang +8Multimodal Large Language Models Video Reasoning
New York University · KAIST
MAGNIFIED: RL Fine-tuning of Multimodal Large Language Models for Motion Planning
Jun 2, 2026Letian Chen, Yiren Lu, Justin Fu +5VLMs for Autonomous Driving Multimodal Large Language Models
Waymo LLC.
Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models
Jun 2, 2026Hashmat Shadab Malik, Muzammal Naseer, Salman KhanMultilingual Language Model Evaluation Adversarial Attacks on VLMs
Mohamed Bin Zayed University of AI, UAE · Khalifa University, UAE · Australian National University, Australia
Investigating Adversarial Robustness of Multi-modal Large Language Models
Jun 2, 2026Hashmat Shadab Malik, Muzammal Naseer, Salman KhanAdversarial Training VLM Robustness
Mohamed Bin Zayed University of AI, UAE · Khalifa University, UAE · Australian National University, Australia
DMT-CBT: Longitudinal Therapeutic State Modeling for CBT Counseling
Jun 2, 2026Chang Liu, Shuyi Zhang, Changsheng Ma +3Mental Health Counseling Multimodal Large Language Models
School of Information Science and Engineering, Lanzhou University
CORE: Conflict-Oriented Reasoning for General Multimodal Manipulation Detection
Jun 2, 2026Jinjie Shen, Yaxiong Wang, Yujiao Wu +5Fake News Detection Multimodal Large Language Models
School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China · Wuhan University, Wuhan, China · Lab for Intelligence and visiON (LION) +2
AVI-Bench: Toward Human-like Audio-Visual Intelligence of Omni-MLLMs
Jun 1, 2026Yaoting Wang, Ziyi Zhang, Wenming Tu +10Audio-Visual Understanding Multimodal Robustness
Institute of Big Data, College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China · Huazhong University of Science and Technology · Shanghai Jiaotong University +5
ToolFG: Towards Well-Grounded Fine-Grained Image Classification
Jun 1, 2026Yu Xue, Haoxuan Qu, Zhuoling Li +4Tool Use in VLMs Multimodal Large Language Models
1Lancaster University · 2Peking University
Mechanistic Diagnostics of Spatial Lexical Bias in Multimodal Large Language Model Spatial Reasoning
Jun 1, 2026Chuang Ma, Qianying Liu, Tomoyuki Obuchi +6VLM Robustness Visual Spatial Reasoning
Kyoto University · NII LLMC · RIKEN AIP +4
Improving Visual Token Reduction via Rectifying Distortions for Efficient Multimodal LLM Inference
Jun 1, 2026Hyeonwoo Cho, Donghyeon Baek, Yewon Kim +1Efficient Multimodal Inference Multimodal Large Language Models
Yonsei University · Korea Institute of Science and Technology (KIST)
SPACE: Source-free Proxy Anchor Concept Erasure for MLLMs
Jun 1, 2026Zhijing Zhang, Jiaqi Ding, Qianshan Wei +5VLM Unlearning Multimodal Large Language Models
School of Computer Science and Engineering, Southeast University · Institute of Automation, Chinese Academy of Sciences
Attention-guided Fine-tuning of Multimodal Large Language Models Improves Chain-of-Thought Reasoning
Jun 1, 2026Sanchit Sinha, Guangzhi Xiong, Bohan Liu +2Fine-Tuning Multimodal Large Language Models
University of Virginia

Jun 6, 2026, cs.CVJ/K move · Enter open · S save

Robust-U1: Can MLLMs Self-Recover Corrupted Visual Content for Robust Understanding?

Jiaqi Tang, Jianmin Chen, Youyang Zhai, Wei Wei+5

The Hong Kong University of Science and Technology, Hong Kong · Northwestern Polytechnical University, Xi’an, China · Northeastern University, Shenyang, China+2

VLM Robustness Image Corruption Robustness Multimodal Large Language Models

Open paper AskPDF arXiv

Multimodal Large Language Models (MLLMs) have demonstrated remarkable success in visual understanding, yet their performance degrades significantly under real-world visual corruptions. While existing robustness enhancement approaches exist, they are limited: black-box feature alignment lacks interpretability, and white-box text-based reasoning cannot restore lost pixel-level details. This work investigates a fundamental research question: Can MLLMs recover corrupted visual content by themselves? To address this, we propose Robust-U1, a novel framework that equips MLLMs with explicit visual self-recovery capability for robust understanding. The approach comprises three core stages: supervised fine-tuning for initial reconstruction, reinforcement learning with dual rewards (pixel-level SSIM and semantic-level CLIP similarity) for aligning high visual quality, and multimodal reasoning that jointly considers both the corrupted input and the recovered image. Extensive experiments demonstrate that Robust-U1 achieves state-of-the-art robustness on the real-world corruption benchmark and maintains superior performance under adversarial corruptions on general VQA benchmarks. Analysis confirms that high-quality visual recovery directly enhances reasoning performance, establishing self-recovery as a critical mechanism for robust visual understanding. The source code is available at https://github.com/jqtangust/Robust-U1.

Momentum

Related topics

Latest papers 676