Multimodal Large Language Models

Follow this topic Ask about this topic

Momentum

90 papers in the last four weeks, up 61% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 877

All topics

OmniThoughtVis: A Scalable Distillation Pipeline for Deployable Multimodal Reasoning Models
May 12, 2026Yuanhao Yue, Chengyu Wang, Yuanjie Lyu +2Multimodal Reasoning Benchmarks Multimodal Large Language Models
Alibaba Group, Hangzhou, China
Logit-Attention Divergence: Mitigating Position Bias in Multi-Image Retrieval via Attention-Guided Calibration
May 12, 2026Mingtao Xian, Yifeng Yang, Qinying Gu +2Multimodal Large Language Models Cross-Modal Attention
Zhiyuan College, Shanghai Jiao Tong University, Shanghai, China · Shanghai Jiao Tong University, Shanghai, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China +1
When Looking Is Not Enough: Visual Attention Structure Reveals Hallucination in MLLMs
May 12, 2026Fanpu Cao, Xin Zou, Xuming Hu +1Large Language Model Hallucination Multimodal Large Language Models
Thrust of Artificial Intelligence, HKUST (Guangzhou), China · Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China
LDDR: Linear-DPP-Based Dynamic-Resolution Frame Sampling for Video MLLMs
May 12, 2026Jingfeng Chen, Jiawen Qian, Wendi Deng +5Video Understanding Frame Rate
1Carnegie Mellon University · 2Individual Researcher · 3National University Singapore +2
Leveraging Multimodal Large Language Models for All-in-One Image Restoration via a Mixture of Frequency Experts
May 12, 2026Eunho Lee, Rei Kawakami, Youngbae HwangImage Restoration Query-Specific Degradation-Aware Representation
Chungbuk National University, Cheongju 28644, Republic of Korea · Institute of Science Tokyo, Tokyo 152-8550, Japan
Instruct-ICL: Instruction-Guided In-Context Learning for Post-Disaster Damage Assessment
May 12, 2026Armin Zarbaft, Ehsan Karimi, Nhut Le +1Post-Disaster Damage Assessment Multimodal Large Language Models
Lehigh Uniersity Pennsylvania, USA
LatentRouter: Can We Choose the Right Multimodal Model Before Seeing Its Answer?
May 11, 2026Xueqi Cheng, Yushun DongLarge Language Model Routing Multimodal Large Language Models
Department of Computer Science Florida State University University
ReVision: Scaling Computer-Use Agents via Temporal Visual Redundancy Reduction
May 11, 2026Amirhossein Abaskohi, Yuhang He, Peter West +3Computer-Use Agents Multimodal Agents
University of British Columbia · Microsoft Research
MPerS: Dynamic MLLM MixExperts Perception-Guided Remote Sensing Scene Segmentation
May 11, 2026Ziyi Wang, Xianping Ma, Ziyao Wang +2Remote Sensing Image Segmentation Scene Reasoning
The Chinese University of Hong Kong (Shenzhen) · Southwest Jiaotong University
Dynamic Cross-Modal Prompt Generation for Multimodal Continual Instruction Tuning
May 11, 2026Tao Hu, Da-Wei ZhouMultimodal Continual Instruction Tuning Instruction Tuning
School of Artificial Intelligence, Nanjing University · State Key Laboratory for Novel Software Technology, Nanjing University
Geospatial-Temporal Sensemaking of Remote Sensing Activity Detections with Multimodal Large Language Model
May 11, 2026David F. Ramirez, Tim Overman, Kristen Jaskie +1Sentinel-2 Textvqa
SenSIP Center, School of ECEE, Arizona State University, Tempe, Arizona, USA · Prime Solutions Group Inc, Goodyear, Arizona, USA
Auditing Multimodal LLM Raters: Central Tendency Bias in Clinical Ordinal Scoring
May 11, 2026Jiaqing Zhang, Sandeep Elluri, Bhanu Cherukuvada +7Rater Llm-As-A-Judge
Department of Electrical & Computer Engineering · Department of Computer and Information Science and Engineering · Department of Clinical and Health Psychology +1
Not Blind but Silenced: Rebalancing Vision and Language via Adversarial Counter-Commonsense Equilibrium
May 11, 2026Qingxin Xiao, Peilin Zhao, Yangyang Zhao +2Contrastive Decoding Large Language Model Hallucination
South China University of Technology · Institute for Super Robotics (Huangpu) · Shanghai Jiao Tong University +1
GuardAD: Safeguarding Autonomous Driving MLLMs via Markovian Safety Logic
May 11, 2026Tianyuan Zhang, Peng Yue, Zihao Peng +8Autonomous Driving Probabilistic Safety
Beihang University · Zhongguancun Laboratory · Peking University
V-ABS: Action-Observer Driven Beam Search for Dynamic Visual Reasoning
May 11, 2026Zhiwei Ning, Xuanang Gao, Jiaxi Cao +6Multimodal Large Language Models Beam Search
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University · Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University · SenseTime Research +1
MicroWorld: Empowering Multimodal Large Language Models to Bridge the Microscopic Domain Gap with Multimodal Attribute Graph
May 11, 2026Manyu Li, Ruian He, Chenxi Ma +2Multimodal Large Language Models Knowledge Graphs
Shanghai Key Laboratory of Intelligent Information Processing School of Computer Science, Fudan University Shanghai, China
ViSRA: A Video-based Spatial Reasoning Agent for Multi-modal Large Language Models
May 11, 2026Tingshu Mou, Jiabo He, Renying Wang +53D Spatial Reasoning Multimodal Large Language Models
Fudan University · BCAI · Bosch Center for Artificial Intelligence (BCAI) +1
Adversarial Attacks Against MLLMs via Progressive Resolution Processing and Adaptive Feature Alignment
May 11, 2026Haobo Wang, Xiaorong Ma, Weiqi Luo +2Multimodal Large Language Models Latent Representation Alignment
Sun Yat-sen University, China · Nanyang Technological University, Singapore · Shenzhen MSU-BIT University, China
The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space
May 11, 2026Xia Hu, Zhenrui Yue, Brian Potetz +4Visual Reasoning Multimodal Large Language Models
Google DeepMind · Stanford University · Google Research
SpaceMind++: Toward Allocentric Cognitive Maps for Spatially Grounded Video MLLMs
May 10, 2026Bo Gu, Zhikang Zhang, Zizhuang Wei +33D Spatial Reasoning Video Multimodal Large Language Models
1Fudan University · 2Huawei · 3Shenzhen Loop Area Institute
Fre-Res: Frequency-Residual Video Token Compression for Efficient Video MLLMs
May 10, 2026Yigui Feng, Qinglin Wang, Yang Liu +1Token Compression Video Compression
The College of Computer Science, National University of Defense Technology, Changsha, Hunan, China · College of Computer Science, National University of Defense Technology, Deya Road 109, Changsha, Hunan 410073, China · The Shien-Ming Wu School of Intelligent Engineering, South China University of Technology, Guangzhou, Guangdong, China +1
Meow-Omni 1: A Multimodal Large Language Model for Feline Ethology
May 9, 2026Jucheng Hu, Zhangquan Chen, Yulin Chen +9Multimodal Large Language Models Emotion Recognition
University College London · Tsinghua University · Nanjing University +1
LLaVA-UHD v4: What Makes Efficient Visual Encoding in MLLMs?
May 9, 2026Kechen Fang, Yihua Qin, Chongyi Wang +3Vision Encoders Multimodal Large Language Models
Tsinghua University · ModelBest
Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Models
May 9, 2026Tri Cao, Khoi Le, Thong Nguyen +7Video Multimodal Large Language Models Spatio-Temporal Reasoning
National University of Singapore · VinUniversity · Nanyang Technological University
How Many Visual Tokens Do Multimodal Language Models Need? Scaling Visual Token Pruning with F^3A
May 9, 2026YiJie Huang, Yiqun Zhang, Zhuoyue Jia +7Visual Token Pruning Long Visual-Token Sequences
School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China · Shanghai Artificial Intelligence Laboratory · School of Computer and Communication Engineering, Northeastern University, Qinhuangdao 066004, China
PPU-Bench:Real World Benchmark for Personalized Partial Unlearning in Vision Language Models
May 9, 2026Jiahui Guang, Zexun Zhan, Zhenlin Xu +5Exact Unlearning Multimodal Large Language Models
Harbin Institute of Technology, Shenzhen, China · Sichuan University, Chengdu, China · Harbin Institute of Technology, Weihai, China +3
SYNCR: A Cross-Video Reasoning Benchmark with Synthetic Grounding
May 8, 2026Sara Ghazanfari, Siddharth Garg, Prashanth Krishnamurthy +1Video Multimodal Large Language Models Long-Video Benchmarks
New York University
SalesSim: Benchmarking and Aligning Multimodal Language Models as Retail User Simulators
May 8, 2026Yada Pruksachatkun, Elaine Wan, Lyanna Chen +2User Simulation Multimodal Large Language Models
Salesforce Research · University of California Los Angeles
WeatherSyn: An Instruction Tuning MLLM For Weather Forecasting Report Generation
May 8, 2026Zinan Zheng, Yang Liu, Nuo Chen +3Weather Prediction Multimodal Continual Instruction Tuning
Hong Kong University of Science and Technology (Guangzhou) · The Chinese University of Hong Kong · HY Founda- tion Model Team, Tencent +2
AudioFace: Language-Assisted Speech-Driven Facial Animation with Multimodal Language Models
May 8, 2026Kai Zheng, Zejian Kang, Rui Mao +4Facial Expression Recognition Articulatory
Westlake University · Zhejiang University · Tiangong University +2
BalCapRL: A Balanced Framework for RL-Based MLLM Image Captioning
May 8, 2026Shaokai Ye, Vasileios Saveris, Yihao Qian +3Image Difference Captioning Video Captioning
Apple
StrLoRA: Towards Streaming Continual Visual Instruction Tuning for MLLMs
May 8, 2026Chang Che, Ziqi Wang, Hui Ma +2Multimodal Continual Instruction Tuning Replay-Based Continual Learning
Hefei University of Technology · Tsinghua University
Hard to Read, Easy to Jailbreak: How Visual Degradation Bypasses MLLM Safety Alignment
May 8, 2026Zhixue Song, Boyan Han, Yiwei Wang +1Recent Vision-Language Models Multimodal Large Language Models
1AGI Lab, Westlake University, China · University of California, Merced, USA
Qwen3-VL-Seg: Unlocking Open-World Referring Segmentation with Vision-Language Grounding
May 8, 2026Yuan Yao, Qiushi Yang, Humen Zhong +5Open-Vocabulary Segmentation Vision-Language Model Grounding
Tongyi Lab, Alibaba Group
Retrieve, Integrate, and Synthesize: Spatial-Semantic Grounded Latent Visual Reasoning
May 8, 2026Jin Cui, Xinyue Long, Xunyong Zhang +5Latent Visual Reasoning Recent Vision-Language Models
1State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University · School of Software Engineering, State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University
Bringing Multimodal Large Language Models to Infrared-Visible Image Fusion Quality Assessment
May 7, 2026Yuchen Guo, Junli Gong, Yao Lu +3Infrared-Visible Image Fusion Multimodal Fusion
Northwestern University · Northeastern University · University of Washington +2
MIST: Multimodal Interactive Speech-based Tool-calling Conversational Assistants for Smart Homes
May 7, 2026Maximillian Chen, Xuanming Zhang, Michael Peng +3Voice Agents Smart Homes
Columbia University · Seoul National University
Towards Annotation-Free Validation of MLLMs: A Vision-Language Logical Consistency Metric
May 7, 2026Ying Gu, Mei Chee Leong, Hui Li Tan +3Multimodal Large Language Models Textvqa
Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (A*STAR), Singapore
CrossCult-KIBench: A Benchmark for Cross-Cultural Knowledge Insertion in MLLMs
May 7, 2026Zhen Zeng, Leijiang Gu, Feng Li +2Multilingual Benchmark Cross-Cultural Variation
1Hefei University of Technology · 2Key Laboratory of Ethnic Language Intelligent Analysis and Security Governance of MOE, Minzu University of China · School of Information Engineering, Minzu University of China
OmicsLM: A Multimodal Large Language Model for Multi-Sample Omics Reasoning
May 7, 2026Maciej Sypetkowski, Joanna Krawczyk, Łukasz Smoliński +4Multi-Omic Integration Transcriptomics
Ingenix.AI, Warsaw, Poland
ICU-Bench:Benchmarking Continual Unlearning in Multimodal Large Language Models
May 7, 2026Yuhang Wang, Wenjie Mei, Junkai Zhang +3Multimodal Large Language Models Multimodal Learning
School of Computer Science and Technology Xidian University
Null Space Constrained Contrastive Visual Forgetting for MLLM Unlearning
May 7, 2026Yuhang Wang, Zhenxing Niu, Haoxuan Ji +3Large Language Model Unlearning Multimodal Large Language Models
School of Computer Science and Technology, Xidian University · Xi’an Jiaotong University
Training-Free Dense Hand Contact Estimation with Multi-Modal Large Language Models
May 7, 2026Daniel Sungho Jung, Kyoung Mu Lee3D Hand Dense Prediction
1IPAI, 2Dept. of ECE & ASRI, Seoul National University, Korea
Conceal, Reconstruct, Jailbreak: Exploiting the Reconstruction-Concealment Tradeoff in MLLMs
May 7, 2026Md Farhamdur Reza, Richeng Jin, Tianfu Wu +1Large Language Model Jailbreaks Jailbreak Attacks
1NC State University · 2Zhejiang University
The Cost of Context: Mitigating Textual Bias in Multimodal Retrieval-Augmented Generation
May 7, 2026Hoin Jung, Xiaoqian WangMultimodal Retrieval Augmented Generation Multimodal Large Language Models
Elmore Family School of Electrical and Computer Engineering Purdue University West Lafayette, IN 47907
Causal Probing for Internal Visual Representations in Multimodal Large Language Models
May 7, 2026Zehao Deng, Tianjie Ju, Zheng Wu +5Multimodal Large Language Models Visual Representations
School of Computer Science, Shanghai Jiao Tong University · Ant Group
AstroAlertBench: Evaluating the Accuracy, Reasoning, and Honesty of Multimodal LLMs in Astronomical Classification
May 7, 2026Claire Chen, Jiabao Sean Xiao, Shuze Daniel Liu +7Astronomical Images Multimodal Benchmarks
1California Institute of Technology · 2Massachusetts Institute of Technology · 3Purdue University
Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Models
May 6, 2026Huatian Zhang, Zhendong Mao, Lei Zhang +1Large Language Model Uncertainty Multimodal Large Language Models
University of Science and Technology of China
JASTIN: Aligning LLMs for Zero-Shot Audio and Speech Evaluation via Natural Language Instructions
May 6, 2026Leying Zhang, Bowen Shi, Haibin Wu +2Modern Generative Audio Models Instruction
Auditory Cognition and Computational Acoustics Lab, School of Computer Science& MoE Key Laboratory of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, 200240 P. R. China · independent researchers, United States
DiffCap-Bench: A Comprehensive, Challenging, Robust Benchmark for Image Difference Captioning
May 6, 2026Yuancheng Wei, Haojie Zhang, Linli Yao +7Image Difference Captioning Multi-Image Editing Benchmarks
South China University of Technology · Peking University · The University of Hong Kong +2
Enhancing Visual Question Answering with Multimodal LLMs via Chain-of-Question Guided Retrieval-Augmented Generation
May 5, 2026Quanxing Xu, Ling Zhou, Xian Zhong +3Knowledge-Based Visual Question Answering Multimodal Large Language Models
School of Computer Science and Engineering, Macau University of Science and Technology, Macao SAR 999078, China · Hubei Key Laboratory of Transportation Internet of Things, School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan, Hubei 430070, China · State Key Laboratory of Maritime Technology and Safety, Wuhan University of Technology, Wuhan 430063, China +3
Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipe
May 5, 2026Wenjin Hou, Shangpin Peng, Weinong Wang +13Uni-Opd Post-Training
1Zhejiang University · 2Shenzhen Loop Area Institute · 3LLM Department, Tencent
Can Multimodal Large Language Models Understand Pathologic Movements? A Pilot Study on Seizure Semiology
May 5, 2026Lina Zhang, Tonmoy Monsoor, Mehmet Efe Lorasdagi +8Seizure Detection Multimodal Large Language Models
University of California, Los Angeles · Mercedes-Benz AG · Zhejiang University
Bolek: A Multimodal Language Model for Molecular Reasoning
May 4, 2026Frederic Grabowski, Jacek Szczerbiński, Maciej Jaśkowski +4Moleculenet Molecular Property Prediction
1Ingenix.ai, Warsaw, Poland · Laboratory of Bioinformatics and Computational Genomics, Warsaw University of Technology, Warsaw, Poland
Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMs
May 4, 2026Xin Zhang, Qiqi Tao, Jiawei Du +2Latent Visual Reasoning Efficient Latent Reasoning
Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore · Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore · Singapore University of Technology and Design +1
From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs
May 4, 2026Le Zhang, Jihan Yang, Soundarya Krishnan +11Stable Spatial Understanding Spatial Reasoning
1Mila - Qu´ebec AI Institute, UdeM · 2NYU · 3Apple +1
Chart-FR1: Visual Focus-Driven Fine-Grained Reasoning on Dense Charts
May 3, 2026Hongkun Pan, Yuwei Wu, Wanyi Hong +8Chart Multimodal Reasoning
Zhejiang University · State Key Lab of CAD&CG, Zhejiang University · HiThink Research
Mitigating Multimodal LLMs Hallucinations via Relevance Propagation at Inference Time
May 3, 2026Itai Allouche, Joseph KeshetMultimodal Large Language Models Large Language Model Hallucination
Department of Electrical and Computer Engineering, Technion, Haifa, Israel
VisInject: Disruption != Injection -- A Dual-Dimension Evaluation of Universal Adversarial Attacks on Vision-Language Models
May 2, 2026Pang Liu, Yingjie LaoAdversarial Examples Prompt-Injection Detectors
Department of Electrical and Computer Engineering, Tufts University
Injecting Distributional Awareness into MLLMs via Reinforcement Learning for Deep Imbalanced Regression
May 2, 2026Yao Du, Shanshan Song, Xiaomeng LiMultimodal Large Language Models Distributional Information
The Hong Kong University of Science and Technology, Hong Kong SAR, China.

May 12, 2026, cs.CLJ/K move · Enter open · S save

OmniThoughtVis: A Scalable Distillation Pipeline for Deployable Multimodal Reasoning Models

Yuanhao Yue, Chengyu Wang, Yuanjie Lyu, Lei Shen+1

Alibaba Group, Hangzhou, China

Multimodal Reasoning Benchmarks Multimodal Large Language Models LLM Reasoning Strategies Large Reasoning Models Dataset Distillation Recent Vision-Language Models+2

Open paper AskPDF arXiv

Recent multimodal large language models (MLLMs) have shown strong chain-of-thought (CoT) reasoning ability on vision-language tasks, but their direct deployment in real-world systems is often limited by latency and resource constraints. In practice, smaller MLLMs are preferred for online serving, yet their reasoning performance is bottlenecked by the lack of large-scale, high-quality multimodal CoT supervision. In this paper, we present OmniThoughtVis, a scalable data curation and distillation pipeline for transferring multimodal reasoning capabilities from high-capacity teacher models to smaller, deployment-oriented MLLMs. Starting from a diverse open-source seed pool, our pipeline generates structured CoT traces and performs joint annotation of reasoning difficulty, answer quality, and semantic task tags. To maintain data quality at scale, we combine rule-based filtering, difficulty-aware selection, and tag-based diversity sampling, resulting in a curated corpus of 1.8M samples that supports controllable subset construction for downstream training. We use OmniThoughtVis to distill Qwen3-VL models from 2B to 8B parameters and evaluate them on nine multimodal reasoning benchmarks. The resulting distilled models show consistent gains across model scales, including improvements of up to +16.8 points on MathVerse and +5.6 points on MMMU-Pro for the 4B model. Notably, the distilled 4B model matches or surpasses the undistilled 8B baseline on several tasks, highlighting the practical value of scalable reasoning distillation for deployment-oriented MLLMs.

Momentum

Related topics

Latest papers 877