Multimodal Large Language Models

Also known as MLLM

Follow this topic Ask about this topic

Momentum

67 papers in the last four weeks, up 91% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 676

All topics

MILM: Large Language Models for Multimodal Irregular Time Series with Informative Sampling
May 13, 2026Hsing-Huan Chung, Shijun Li, Yoav Wald +3Irregular Time-Series Modeling Healthcare
University of Texas at Austin · Technion-IIT · Johns Hopkins University
Multilingual OCR-Aware Fine-Tuning and Prompt-Guided Chain-of-Thought Reasoning for Multimodal Large Language Models
May 13, 2026Qinwu Xu, Yifan Jiang, Haoyu RenMultimodal Grounding CoT Reasoning
Meta AI · UT Austin · Current independent researcher
Robust Checkpoint Selection for Multimodal LLMs via Agentic Evaluation and Stability-Aware Ranking
May 13, 2026Qinwu Xu, Zhuoheng Li, Jessie SalasMultimodal Large Language Models Multimodal Model Evaluation
Meta AI · Independent Researcher, previously Meta AI
Learning to See What You Need: Gaze Attention for Multimodal Large Language Models
May 13, 2026Junha Song, Byeongho Heo, Geonmo Gu +3Efficient Multimodal Inference Visual Attention
1KAIST · 2NAVER AI Lab
DiM\textsuperscript{3}: Bridging Multilingual and Multimodal Models via Direction- and Magnitude-Aware Merging
May 13, 2026Zijing Wang, Mingyang Wang, Ercong Nie +6Multilingual Language Models Multimodal Large Language Models
Northeastern University, China · CIS, LMU Munich, Germany · Munich Center for Machine Learning (MCML), Germany +1
Seg-Agent: Test-Time Multimodal Reasoning for Training-Free Language-Guided Segmentation
May 13, 2026Chao Hao, Jun Xu, Ji Du +6Text-Guided Image Segmentation Promptable Image Segmentation
School of Computing and Information Technology, Great Bay University, Dongguan, Guangdong, 523000, China · Hangzhou International Innovation Institute, Beihang University, Hangzhou, 311115, China · Department of Computing, The Hong Kong Polytechnic University, Hong Kong
Visual Aesthetic Benchmark: Can Frontier Models Judge Beauty?
May 12, 2026Yichen Feng, Yuetai Li, Chunjiang Liu +14Multimodal Large Language Models Image Quality Assessment
Beyond Localization: A Comprehensive Benchmark of Perspective-Conditioned Spatial Reasoning in MLLMs from Omnidirectional Images
May 12, 2026Yuangong Chen, Wai Keung Wong, Jiaxing Li +2Spatial Reasoning Benchmarks Visual Spatial Reasoning
The Hong Kong Polytechnic University Hong Kong, China · Guangzhou University Guangzhou, China · Queen Mary, University of London London, United Kingdom +1
Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models
May 12, 2026Yanting Miao, Yutao Sun, Dexin Wang +8Multimodal Large Language Models Latent Visual Reasoning
1Qwen Large Model Application Team, Alibaba · 3Zhejiang University · University of Waterloo +1
Towards Automated Air Traffic Safety Assessment Around Non-Towered Airports Using Large Language Models
May 12, 2026Torsten Darrell, Mahyar Ghazanfari, Jordan Kam +3Air Traffic Management Multimodal Large Language Models
Department of Mechanical and Aerospace Engineering, George Washington University · Aerospace Program, University of California, Berkeley · Department of Electrical Engineering and Computer Science, University of California, Berkeley +1
OmniRefine: Alignment-Aware Cooperative Compression for Efficient Omnimodal Large Language Models
May 12, 2026Yuchen Deng, Zidang Cai, Hai-Tao Zheng +3Audio-Visual Understanding Efficient Multimodal Inference
Tsinghua Shenzhen International Graduate School, Tsinghua University · Pengcheng Laboratory
Boosting Omni-Modal Language Models: Staged Post-Training with Visually Debiased Evaluation
May 12, 2026Che Liu, Lichao Ma, Xiangyu Tony Zhang +4Cross-Modal Learning Multimodal Large Language Models
StepFun-Audio Team
Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training
May 12, 2026Qihuang Zhong, Liang Ding, Wenjie Xuan +3Multimodal Large Language Models Multimodal Reasoning
School of Computer Science, National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University, China · The University of Sydney, Australia · Nanyang Technological University, Singapore
UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs
May 12, 2026Houcheng Jiang, Jiajun Fu, Junfeng Fang +4Efficient Multimodal Inference Multimodal Large Language Models
University of Science and Technology of China · Zhongguancun Academy · National University of Singapore +1
SafeSteer: A Decoding-level Defense Mechanism for Multimodal Large Language Models
May 12, 2026Xinyi Zeng, Xue Yang, Jingyuan Zhang +5Adversarial Attacks on VLMs Multimodal Robustness
Tsinghua University, Beijing, China · Shanghai Jiao Tong University · Kuaishou Technology, Beijing, China +4
Measuring What Matters Beyond Text: Evaluating Multimodal Summaries by Quality, Alignment, and Diversity
May 12, 2026Abid Ali, Diego Molla-Aliod, Usman NaseemLLM Evaluation Multimodal Large Language Models
School of Computing, Macquarie University, Sydney, Australia
Logit-Attention Divergence: Mitigating Position Bias in Multi-Image Retrieval via Attention-Guided Calibration
May 12, 2026Mingtao Xian, Yifeng Yang, Qinying Gu +2Multimodal Large Language Models Cross-Modal Retrieval
Zhiyuan College, Shanghai Jiao Tong University, Shanghai, China · Shanghai Jiao Tong University, Shanghai, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China +1
When Looking Is Not Enough: Visual Attention Structure Reveals Hallucination in MLLMs
May 12, 2026Fanpu Cao, Xin Zou, Xuming Hu +1Hallucination Detection in VLMs Hallucination in Language Models
Thrust of Artificial Intelligence, HKUST (Guangzhou), China · Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China
LDDR: Linear-DPP-Based Dynamic-Resolution Frame Sampling for Video MLLMs
May 12, 2026Jingfeng Chen, Jiawen Qian, Wendi Deng +5Efficient VLM Inference Multimodal Large Language Models
1Carnegie Mellon University · 2Individual Researcher · 3National University Singapore +2
LatentRouter: Can We Choose the Right Multimodal Model Before Seeing Its Answer?
May 11, 2026Xueqi Cheng, Yushun DongEfficient Multimodal Inference Multimodal Large Language Models
Department of Computer Science Florida State University University
Personal Visual Context Learning in Large Multimodal Models
May 11, 2026Zihui Xue, Ami Baid, Sangho Kim +2Vision-Language Models Multimodal Large Language Models
The University of Texas at Austin
Count Anything at Any Granularity
May 11, 2026Chang Liu, Haoning Wu, Weidi XieObject Counting Multimodal Large Language Models
School of Artificial Intelligence, Shanghai Jiao Tong University, China · CMIC, Shanghai Jiao Tong University, China
Probing Cross-modal Information Hubs in Audio-Visual LLMs
May 11, 2026Jihoo Jung, Chaeyoung Jung, Ji-Hoon Kim +1Cross-Modal Learning Multimodal Large Language Models
Department of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea · The Graduate School of Advanced Imaging Science, Multimedia & Film, Chung-Ang University, Seoul, Republic of Korea
Dynamic Cross-Modal Prompt Generation for Multimodal Continual Instruction Tuning
May 11, 2026Tao Hu, Da-Wei ZhouCross-Modal Learning Continual Learning for LLMs
School of Artificial Intelligence, Nanjing University · State Key Laboratory for Novel Software Technology, Nanjing University
Geospatial-Temporal Sensemaking of Remote Sensing Activity Detections with Multimodal Large Language Model
May 11, 2026David F. Ramirez, Tim Overman, Kristen Jaskie +1Remote Sensing Image Understanding Visual Question Answering
SenSIP Center, School of ECEE, Arizona State University, Tempe, Arizona, USA · Prime Solutions Group Inc, Goodyear, Arizona, USA
Auditing Multimodal LLM Raters: Central Tendency Bias in Clinical Ordinal Scoring
May 11, 2026Jiaqing Zhang, Sandeep Elluri, Bhanu Cherukuvada +7LLM Auditing Multimodal Large Language Models
Department of Electrical & Computer Engineering · Department of Computer and Information Science and Engineering · Department of Clinical and Health Psychology +1
Phoenix-VL 1.5 Medium Technical Report
May 11, 2026Team Phoenix, :, Arka Ray +30Multilingual Language Models Domain Adaptation
HTX × Mistral AI ∗
SciVQR: A Multidisciplinary Multimodal Benchmark for Advanced Scientific Reasoning Evaluation
May 11, 2026Longteng Guo, Xuanxu Lin, Dongze Hao +5Multimodal Large Language Models Scientific QA
Institute of Automation, Chinese Academy of Sciences (CASIA) · School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS) · OPPO AI Center
MicroWorld: Empowering Multimodal Large Language Models to Bridge the Microscopic Domain Gap with Multimodal Attribute Graph
May 11, 2026Manyu Li, Ruian He, Chenxi Ma +2Multimodal RAG Visual Question Answering
Shanghai Key Laboratory of Intelligent Information Processing School of Computer Science, Fudan University Shanghai, China
ViSRA: A Video-based Spatial Reasoning Agent for Multi-modal Large Language Models
May 11, 2026Tingshu Mou, Jiabo He, Renying Wang +5Visual Spatial Reasoning 3D Spatial Reasoning
Fudan University · BCAI · Bosch Center for Artificial Intelligence (BCAI) +1

May 13, 2026, cs.LGJ/K move · Enter open · S save

MILM: Large Language Models for Multimodal Irregular Time Series with Informative Sampling

Hsing-Huan Chung, Shijun Li, Yoav Wald, Xing Han+2

University of Texas at Austin · Technion-IIT · Johns Hopkins University

Irregular Time-Series Modeling Healthcare Time Series Classification Multimodal Large Language Models Clinical Outcome Prediction

Open paper AskPDF arXiv

Multimodal irregular time series (MITS) consist of asynchronous and irregularly sampled observations from heterogeneous numerical and textual channels. In healthcare, for example, patients' electronic health records (EHR) include irregular lab measurements and clinical notes. The irregular timing and channel patterns of observations carry predictive signal alongside the numerical values and textual content. LLMs are natural candidates for processing such heterogeneous data, given their extensive pretrained knowledge spanning textual and numerical domains. We introduce MILM (Multimodal Irregular time series Language Model), which represents MITS as time-ordered triplets in Extensible Markup Language (XML) format and fine-tunes an LLM through a two-stage strategy for MITS classification. The first stage trains on value-redacted MITS to predict from sampling patterns alone, and the second stage trains on full MITS to jointly model sampling patterns and observed values. Our two-stage model (MILM-2S) and its single-stage counterpart (MILM-Direct) achieve the best and second-best average performance on multiple EHR datasets. Further value redaction evaluations confirm that sampling patterns carry predictive signal and that MILM-2S learns to exploit them. In the value pending evaluation we introduce, where some values are unavailable at prediction time, MILM-2S outperforms MILM-Direct by a larger margin compared to standard evaluation. For MILM-2S, preserving the time and channel of value-pending observations as additional sampling information further improves in-hospital mortality prediction.

Momentum

Related topics

Latest papers 676