Modality Imbalance

Follow this topic Ask about this topic

Latest papers 31

5 papers in the last four weeks, up 67% on the four weeks before. 0.0% of all new papers.

All topics

Balancing Multimodal Learning via Functional Progress
Oct 2, 2026Zhongjing Gu, Fengqiang Wan, Yiming Cui +2Modality Imbalance Multimodal Optimization
Confidence Falls Short: Asymmetric Certainty Gains from Optimization Hinder Multimodal Classification
Sep 23, 2026Longfei Huang, Xiangyu Wu, Yang YangModality Imbalance Multimodal Optimization
Nanjing University of Science and Technology · Alibaba Group
All modalities are equal, but video is more equal: Closing the Cross-Attention Gap in Joint Video Generation
Sep 23, 2026Ohad Rahamim, Dvir Samuel, Idan Schwartz +1Video Diffusion Models Modality Imbalance
Bar-Ilan University · NVIDIA
Reassessing Global Gradient-Norm Imbalance in BLIP Fine-Tuning Across Physical Domains
Sep 20, 2026Kiran Naseer, Samreen Azhar, Dwarikanath MahapatraImage Captioning VLM Adaptation
Department of Computer Science, University of Gujrat, Pakistan · Department of Computer Science, Khalifa University, Abu Dhabi, UAE
GeoBalance: Geometry-Aware Monitoring and Reconstruction with Asymmetric Optimization for Balanced Multimodal Learning
Sep 20, 2026Zechang Xiong, Da Li, Rong Yin +7Modality Imbalance Geometric Representation Learning
Beihang University, Beijing, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · Amap, Alibaba Group, Beijing, China +3
The Illusion of Balanced Multimodal Sentiment Analysis: Beyond the Limits of Optimization-Based Methods
Sep 10, 2026Ioanna Kaffeza, Efthymios Georgiou, Alexandros PotamianosModality Imbalance Multimodal Optimization
PERSEE Center, Mines Paris-PSL University, Sophia-Antipolis, France · National Technical University of Athens, Athens, Greece · University of Bern, Bern, Switzerland +2
MM-Spectrum: Multimodal Multi-spectral Molecular Structural Elucidation with a Stable MoE Framework
Aug 27, 2026Hai-tao Yu, Nan Min, Zheng Fang +4Multimodal Fusion Modality Imbalance
The Hong Kong University of Science and Technology (Guangzhou) · Southeast University · The Hong Kong University of Science and Technology +1
CAT-GS: Balanced Multimodal Learning via Calibrated Gating and Fusion Surgery
Aug 24, 2026Mahir Shahriar Tamim, Sharjil Khan, Md. Samiul Alim +3Cross-Modal Learning Modality Imbalance
Department of Electrical and Computer Engineering, North South University, Dhaka, Bangladesh
C $^3$ PO: Evaluating Cross-Modal Composition and Counterfactual Performance in Omnimodal Models
Aug 5, 2026Swapnanil Mukherjee, Agyeya Negi, Tanuja Ganu +1Modality Imbalance Multimodal Reasoning
Microsoft Research India · IIIT Hyderabad, India
OPD-V: Visual On-Policy Self-Distillation with Modality Balance
Aug 5, 2026Aniri, Jinhe Bi, Peng Liao +5Modality Imbalance On-Policy Self-Distillation
2Ludwig Maximilian University of Munich · 3Munich Center for Machine Learning · 1National University of Singapore +1
Beyond Modality Fusion: Deep Ensembles for Multimodal Classification
Jul 6, 2026Ilya Burenko, Dmitry VetrovModality Imbalance Multimodal Classification
ScaDS.AI, Technische Universität Dresden · Dresden, Germany · Constructor University +1
Pareto LoRA: Mitigating Modality Imbalance in Unified Multimodal Models via Pareto-Optimal Gradient Integration
Jun 15, 2026Xiwen Wei, Mark Nutter, Madhusudhanan Srinivasan +1Unified Multimodal Models Modality Imbalance
The University of Texas at Austin · Advanced Micro Devices, Inc.
Beyond Symmetric Alignment: Spectral Diagnostics of Modality Imbalance in Vision-Language Models in the Medical Domain
Jun 3, 2026Alessandro Gambetti, Qiwei Han, Cláudia Soares +1VLM Evaluation Modality Imbalance
NOVA School of Science and Technology · Lisbon, Portugal · Nova School of Business and Economics +2
A Conflict-Aware Penalty and Statistical Loss Framework for Balancing Modalities and Enhancing Stability in Multimodal Sentiment Analysis
May 27, 2026Jianheng Dai, Jiazhang Liang, Sijie MaiModality Imbalance Gradient Interference
School of Computer Science, South China Normal University, Guangzhou, Guangdong, China
Balancing Multimodal Learning through Label Space Reshaping
May 22, 2026Xiaoyu Ma, Weijie Zhang, Yuanhao Gao +3Cross-Modal Learning Cross-Modal Representation Learning
PREPRINT 2026.05
Tackling Multimodal Learning Challenges with Mixture-of-Expert: A Survey
May 22, 2026Liangwei Nathan Zheng, Wei Emma Zhang, Olaf Maennel +2Efficient Multimodal Inference Cross-Modal Representation Learning
Adelaide University
Leveraging Self-Paced Curriculum Learning for Enhanced Modality Balance in Multimodal Conversational Emotion Recognition
May 20, 2026Phuong-Anh Nguyen, The-Son Le, Duc-Trong Le +1Modality Imbalance Emotion Recognition in Conversations
Faculty of Information Technology, VNU University of Engineering and Technology, 100000, Hanoi, Vietnam.
Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration
May 5, 2026Xun Jiang, Yufan Gu, Disen Hu +5Multimodal Robustness Modality Imbalance
School of Computer Science and Technology, Tongji University · School of Computer Science and Engineering, University of Electronic Science and Technology of China · Independent Researcher +1
Joint-Centric Dual Contrastive Alignment with Structure-Preserving and Information-Balanced Regularization
Apr 17, 2026Habibeh Naderi, Behrouz Haji Soleimani, Stan MatwinCross-Modal Learning Modality Imbalance
Dalhousie University, Halifax NS, Canada
Enhance-then-Balance Modality Collaboration for Robust Multimodal Sentiment Analysis
Apr 14, 2026Kang He, Yuzhe Ding, Xinrong Wang +3Multimodal Sentiment Analysis Modality Imbalance
Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University. · Shanghai Innovation Institute
PDMP: Rethinking Balanced Multimodal Learning via Performance-Dominant Modality Prioritization
Apr 7, 2026Shicai Wei, Chunbo Luo, Qiang Zhu +1Multimodal Learning Multimodal Optimization
Laboratory of Intelligent Collaborative Computing, University of Electronic Science and Technology of China. · School of Information and Communication Engineering, University of Electronic Science and Technology of China, Sichuan, China. · Peng Cheng Laboratory, Shenzhen, China.
Mind the Gap: A Framework for Assessing Pitfalls in Multimodal Active Learning
Mar 31, 2026Dustin Eisenhardt, Yunhee Jeong, Florian BuettnerActive Learning Modality Imbalance
German Cancer Research Center (DKFZ), Heidelberg, Germany · German Cancer Consortium (DKTK), partner site Frankfurt/Mainz, a partnership between DKFZ and UCT Frankfurt-Marburg, Frankfurt am Main, Germany · Institute of Informatics, Goethe University Frankfurt, Frankfurt am Main, Germany +3
CCF: Complementary Collaborative Fusion for Domain Generalized Multi-Modal 3D Object Detection
Mar 24, 2026Yuchen Wu, Kun Wang, Yining Pan +13D Object Detection Camera-LiDAR Fusion
Singapore University of Technology and Design
BALM: A Model-Agnostic Framework for Balanced Multimodal Learning under Imbalanced Missing Rates
Mar 20, 2026Phuong-Anh Nguyen, Tien Anh Pham, Duc-Trong Le +1Modality Imbalance Multimodal Learning
VNU University of Engineering and Technology, Hanoi, Vietnam
Unbiased Dynamic Multimodal Fusion
Mar 20, 2026Shicai Wei, Kaijie Zhang, Luyi Chen +2Multimodal Fusion Uncertainty-Aware Multimodal Fusion
University of Electronic Science and Technology of China
Parameter-Efficient Modality-Balanced Symmetric Fusion for Multimodal Remote Sensing Semantic Segmentation
Mar 18, 2026Haocheng Li, Juepeng Zheng, Shuangxi Miao +4Remote Sensing Image Segmentation Multimodal Fusion
the College of Land Science and Technology, China Agricultural University, and also with the Key Laboratory of Remote Sensing for Agri-Hazards, Ministry of Agriculture and Rural Affairs, Beijing 100083, China · the School of Artificial Intelligence, Sun Yat-Sen University, Zhuhai, China · Henan Polytechnic University, and also with the Key Laboratory of Spatio-Temporal Information and Ecological Restoration of Mines, Ministry of Natural Resources of the People’s Republic of China, Jiaozuo 454003, China +4
Dr. SHAP-AV: Decoding Relative Modality Contributions via Shapley Attribution in Audio-Visual Speech Recognition
Mar 12, 2026Umberto Cappellazzo, Stavros Petridis, Maja PanticAudio-Visual Speech Recognition Modality Imbalance
Imperial College London, UK · NatWest AI Research, UK
MissBench: Benchmarking Multimodal Affective Analysis under Imbalanced Missing Modalities
Mar 10, 2026Tien Anh Pham, Phuong-Anh Nguyen, Duc-Trong Le +1Multimodal Model Evaluation Missing-Modality Learning
VNU University of Engineering and Technology, Hanoi, Vietnam
AMB-DSGDN: Adaptive Modality-Balanced Dynamic Semantic Graph Differential Network for Multimodal Emotion Recognition
Mar 7, 2026Yunsheng Wang, Yuntao Shou, Yilong Tan +3Emotion Recognition in Conversations Multimodal Emotion Recognition
College of Computer and Mathematics, Central South University of Forestry and Technology · Department of Computer Science, State University of New York
SGMA: Semantic-Guided Modality-Aware Segmentation for Remote Sensing with Incomplete Multimodal Data
Mar 3, 2026Lekang Wen, Liang Liao, Jing Xiao +1Remote Sensing Image Segmentation Semantic Segmentation
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University · Hangzhou Institute of Technology, Xidian University · School of Artificial Intelligence, Wuhan University

Oct 2, 2026J/K move · Enter open · S save

Balancing Multimodal Learning via Functional Progress

Zhongjing Gu, Fengqiang Wan, Yiming Cui, Yufa Feng+1

Modality Imbalance Multimodal Optimization

Open paper AskPDF arXiv

Multimodal learning often suffers from modality imbalance, where the joint optimization process is dominated by a single modality. Existing methods typically estimate modality imbalance from score disparities derived from prediction uncertainty or optimization statistics. However, due to distinct prediction uncertainty and learning dynamics across modalities, direct comparison of such scores may misinterpret intrinsic modality differences as progress gaps, leading to biased imbalance estimation. In this paper, we propose Function-Space Guided Multimodal Optimization (FGMO), which leverages a function-space progress signal to assess modality-wise optimization progress and coordinate optimization across modalities to alleviate modality imbalance. Specifically, we introduce Functional Progress Estimation (FPE) to measure each modality's update-induced function-space response and calibrate it against a loss-aligned unimodal reference, producing a comparable progress signal. Based on this signal, Functional Response Control (FRC) redistributes modality-level function-space budgets and realizes the target responses through tensor-wise learning-rate adjustment. Theoretical analysis establishes a one-step target-contraction property of FRC under bounded controller-state mismatch, and extensive experiments demonstrate the effectiveness of FGMO across multiple multimodal benchmarks.