Multimodal Large Language Models

Also known as MLLM

Follow this topic Ask about this topic

Momentum

67 papers in the last four weeks, up 91% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 676

All topics

ViPo-MLLM: Visual-Pose Multimodal LLM for Gloss-Free Sign Language Translation
Jul 4, 2026Ahmed Abul Hasanaath, Bicheng Xu, Mir Rayat Imtiaz Hossain +2Sign Language Translation Multimodal Large Language Models
King Fahd University of Petroleum and Minerals, Dhahran, KSA · University of British Columbia, Vancouver, Canada
Responsibility Distribution Estimation in Ego-View Accident Videos with Multimodal Large Language Models
Jul 3, 2026Ryosei Tamura, Andrew ShinMultimodal Large Language Models Egocentric Video Understanding
Keio University
OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Models
Jul 3, 2026Shijie Cao, Qingyu Zhang, Boxi Yu +6Efficient Multimodal Inference Multimodal Large Language Models
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Limerick +1
GuideMe: Multi-Domain Task Guidance and Intervention in Streaming Video
Jul 3, 2026Fang Liu, Jinpeng Chen, Ke Xu +7VLM Evaluation Streaming Video Understanding
City University of Hong Kong · Huawei Research · University of Science and Technology of China +2
CL-Anomaly: Layer-Adaptive Mixture-of-Experts with Multimodal Large Language Model for Continual Learning in Anomaly Detection
Jul 3, 2026Wen Dong, Zhao Wang, Shuangqing Zhang +5Mixture of Experts Multimodal Large Language Models
Nanjing University, Nanjing, China · China Mobile Zijin Innovation Institute, Nanjing, China · Nanjing University of Science and Technology, Nanjing, China
Learning Taxonomic Trees with Hierarchical Representation Regularization for Large Multimodal Models
Jul 3, 2026Hulingxiao He, Zhi Tan, Yuxin PengHierarchical Representation Learning Multimodal Large Language Models
Wangxuan Institute of Computer Technology, Peking University
ProLaViT: Learning Progressive Latent Visual Thoughts in Structured Latent Space
Jul 3, 2026Peiming Li, Yifan Wang, Xiaotian Zhang +4Multimodal Large Language Models Latent Visual Reasoning
Basic Algorithm Center, PCG, Tencent · Zhejiang University · Peking University
Gemma 4 Technical Report
Jul 2, 2026Gemma Team, Sherif El Abd, Vaibhav Aggarwal +298Efficient Multimodal Inference Unified Multimodal Models
K9-Bench: Evaluating Multimodal LLMs on Canine-Centric Videos
Jul 2, 2026Khush Attarde, Yusuf Ali, Megha Thukral +3Video Understanding Multimodal Large Language Models
Ogmen Robotics Inc., Massachusetts, USA · Georgia Institute of Technology, Atlanta, USA
EAGLE-360: Embodied Active Global-to-Local Exploration in 360 $^\circ$
Jul 2, 2026Jingtao Xu, Zizhuo Lin, Jianwen Sun +2Multimodal Large Language Models
1Zhejiang University · 2Central China Normal University
InduceKV: Fixed-Footprint Continual Adaptation of Multimodal LLMs via Inducing KV Memories
Jul 2, 2026Qianyu Chen, Ziteng Feng, Canran Xiao +1Continual Learning for LLMs Memory-Efficient Fine-Tuning
1Nanyang Technological University · University of Science and Technology of China · 3Shenzhen Campus of Sun Yat-sen University
Multimodal Knowledge Edit-Scoped Generalization for Online Recursive MLLM Editing
Jul 2, 2026Siyuan Li, Youyuan Zhang, Ruitong Liu +2Knowledge Editing Multimodal Large Language Models
Harbin Institute of Technology, Shenzhen, China · Peng Cheng Laboratory · Peking University +1
Token-level Response-visual Attention Guidance for Multimodal LLMs Knowledge Distillation
Jul 1, 2026Jaehyun Jang, Eunseop Yoon, Hee Suk Yoon +3Cross-Modal Knowledge Distillation VLM Distillation
Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea · University of Illinois Urbana-Champaign, Champaign, USA
Multimodal Continuous Reasoning via Asymmetric Mutual Variational Learning
Jul 1, 2026Shijie Li, Yilin Gao, Siyuan Yang +7Multimodal Large Language Models Latent Visual Reasoning
Shanghai Jiao Tong University · Ant Group
Wake up for Touch! Mask-isolated Tactile Alignment Learning in MLLMs
Jul 1, 2026Yoonhyung Park, Minji Kim, Sungwon Moon +1Visuo-Tactile Representation Learning Multimodal Large Language Models
Division of Artificial Intelligence & Software, Ewha Womans University
Decompose, Compare, and Decide: Multimodal LLMs are Implicit Few-Shot Learners
Jun 30, 2026Yunhan Wang, Eshika Khandelwal, Edson Araujo +3Multimodal Large Language Models Few-Shot Prompting
Tuebingen AI Center, University of Tuebingen, Germany
CoMet: Context and Multiplicity Decomposition for Multimodal Uncertainty Estimation
Jun 30, 2026Sanghyuk Chun, William Yang, Amaya Dharmasiri +1Uncertainty Quantification Multimodal Large Language Models
Princeton University
ERA: Entropy-Guided Visual Token Pruning with Rectified Attention for Efficient MLLMs
Jun 30, 2026Yuhao Wang, Mu Qiao, Haiwen Diao +5Efficient VLM Inference Multimodal Large Language Models
School of Future Technology, Dalian University of Technology, Dalian 116024, China · S-Lab, Nanyang Technological University, Singapore 639798 · OPPO Research Institute, Shenzhen 518000, China +1
Attend, Transform, or Silence: Operator-Level Visual Skipping for Efficient Multimodal LLM Inference
Jun 30, 2026Zhaoyang Luo, Runmin Dong, Miao Yang +4Efficient VLM Inference Multimodal Large Language Models
Tsinghua Shenzhen International Graduate School, Shenzhen, China · Sun Yat-sen University, Zhuhai, China · Tsinghua University, Beijing, China +1
Harnessing Textual Refusal Directions for Multimodal Safety
Jun 30, 2026Moreno D'Incà, Nicu Sebe, Massimiliano ManciniLanguage Model Safety Evaluation Multimodal Large Language Models
University of Trento
UniCoder: Unified Visual-to-Code Generation via Symbolic Rewards and Reference-Guided Code Optimization
Jun 30, 2026Yaozhi Zheng, Yilei Jiang, Manyuan Zhang +5RL for Code Generation Reinforcement Learning
The Chinese University of Hong Kong · Shanghai Artificial Intelligence Laboratory
MS-Resampler: Multi-Scope Visual Resampling for Efficient Multimodal LLMs
Jun 30, 2026Zhongyang Li, Yaqian Li, Faming Fang +6Multimodal Large Language Models Multimodal Fusion
Li Auto Inc. · East China Normal University.
Learning to Deny: Action Denial in Multimodal Large Language Models
Jun 30, 2026Raiyaan Abdullah, Shehreen Azad, Yogesh Singh RawatMultimodal Large Language Models Causal Reasoning in Videos
Institute of Artificial Intelligence, University of Central Florida, USA
OmniCoT: A Benchmark for Global and Multi-Step Panoramic Reasoning
Jun 29, 2026Haocong He, Chenfei Liao, Zichen Wen +13Visual Spatial Reasoning Multimodal Large Language Models
SJTU · HKUST(GZ) · Shanghai AI Lab +6
Latent Noise Mask for Reducing Visual Redundancy in Multimodal Large Language Models
Jun 29, 2026Kai Jiang, Ruishu Zhu, Siqi Huang +2Visual Question Answering Multimodal Large Language Models
School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University · Institute of Artificial Intelligence, China Telecom (TeleAI) · 3Fudan University +1
MuseBench: Benchmarking Intent-Level Audiovisual Arts Understanding in MLLMs
Jun 29, 2026Yuxuan Fan, Gyusik Seo, Jing Hao +3Audio-Visual Understanding Multimodal Large Language Models
1NTU Singapore · 2The University of Hong Kong · 3Johns Hopkins University +2
Consistency as Inductive Bias: Learning Cross-View Invariance for Robust Multimodal Reasoning
Jun 29, 2026Xin Zou, Haolin Deng, Yibo Yan +6Multi-View Consistency Multimodal Robustness
HKUST (GZ) · HKUST · UIC +1
Making Multimodal LLMs Reliable Chart Data Extractors: A Benchmark and Training Framework
Jun 29, 2026Yuchen He, Peizhi Ying, Liqi Cheng +4Document Information Extraction Multimodal Large Language Models
State Key Lab of CAD&CG, Zhejiang University · Zhejiang University Hangzhou, Zhejiang, China · Guangdong University of Technology Guangzhou, Guangdong, China
Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigation
Jun 29, 2026Xin Zou, Haolin Deng, Yibo Yan +5Multimodal Large Language Models Multimodal Hallucination
HKUST (GZ) · HKUST · OPPO AI Center
How LLMs See Creativity: Zero-Shot Scoring of Visual Creativity with Interpretable Reasoning
Jun 29, 2026William Orwig, Roger E. BeatyCreativity Assessment LLM-as-a-Judge
Harvard University, Department of Psychology, Cambridge, MA, USA · Pennsylvania State University, Department of Psychology, University Park, PA, USA

Jul 4, 2026, cs.CVJ/K move · Enter open · S save

ViPo-MLLM: Visual-Pose Multimodal LLM for Gloss-Free Sign Language Translation

Ahmed Abul Hasanaath, Bicheng Xu, Mir Rayat Imtiaz Hossain, Leonid Sigal+1

King Fahd University of Petroleum and Minerals, Dhahran, KSA · University of British Columbia, Vancouver, Canada

Sign Language Translation Multimodal Large Language Models

Open paper AskPDF arXiv

Gloss-free Sign Language Translation (SLT) translates sign language videos into spoken-language sentences without gloss annotations, avoiding costly labeling but requiring fine-grained modeling of hands, body, and facial cues. Existing methods often use single-modality or weakly fused features, limiting performance. We propose ViPo-MLLM, a framework that integrates spatio-temporal RGB and human pose features. Dedicated encoders model intra-modal dynamics and cross-modal attention captures long-range dependencies. The fused representation is conditioned with a structured prompt and processed by an LLM trained with contrastive and language modeling objectives. The proposed model was evaluated on the PHOENIX14T and CSL-Daily datasets and achieved new state-of-the-art results on both datasets. Moreover, the ViPo-MLLM model attained competitive performance compared to gloss-based recognition approaches, confirming the effectiveness of the proposed pose cues and cross-modal attention mechanisms.

Momentum

Related topics

Latest papers 676