Multimodal Optimization

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, with none the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 21

All topics

Confidence Falls Short: Asymmetric Certainty Gains from Optimization Hinder Multimodal Classification
Sep 23, 2026Longfei Huang, Xiangyu Wu, Yang YangModality Imbalance Multimodal Optimization
Nanjing University of Science and Technology · Alibaba Group
VCMM: Variance-Calibrated Momentum for Multimodal Learning
Sep 23, 2026Zhongjing Gu, Chenyang Huang, Yufa Feng +3Multimodal Optimization Adaptive Gradient Methods
GeoBalance: Geometry-Aware Monitoring and Reconstruction with Asymmetric Optimization for Balanced Multimodal Learning
Sep 20, 2026Zechang Xiong, Da Li, Rong Yin +7Modality Imbalance Geometric Representation Learning
Beihang University, Beijing, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · Amap, Alibaba Group, Beijing, China +3
From Models to Systems: A Comprehensive Survey of Efficient Multimodal Learning
Sep 16, 2026Pan Wang, Siwei Song, Hui Ji +12Efficient Multimodal Inference Multimodal Optimization
University of Pittsburgh · New York University · Carnegie Mellon University +6
PACE: Progressive Angular-to-Norm Contrastive Embedding
Sep 14, 2026Yanping Li, Wei Zhou, Yawen Liu +7Cross-Modal Representation Learning Multimodal Embedding
The Hong Kong University of Science and Technology · Alibaba Group · Huazhong University of Science and Technology +1
Tracing and Coordinating Cross-Layer Influence for Multimodal Model Merging
Sep 14, 2026Pengyang Zhou, Xiaobin Tu, Zhengxi Liu +7Multimodal Optimization Language Model Merging
Alibaba Cloud
The Illusion of Balanced Multimodal Sentiment Analysis: Beyond the Limits of Optimization-Based Methods
Sep 11, 2026Ioanna Kaffeza, Efthymios Georgiou, Alexandros PotamianosModality Imbalance Multimodal Optimization
PERSEE Center, Mines Paris-PSL University, Sophia-Antipolis, France · National Technical University of Athens, Athens, Greece · University of Bern, Bern, Switzerland +2
Beyond Independent Optimization: Compression, MoE Routing, and Quantization Interactions in Multimodal Edge Intelligence
Jul 23, 2026Jay Gor, Karm Dave, Akshita Abrol +3Efficient Multimodal Inference VLM Quantization
Nirma University · Singapore Institute of Technology, Singapore · Department of CE-AI and Big Data, Marwadi University
OPOD: On-Policy Omni Distillation
Jul 23, 2026Tong Zhao, Yuyang Hu, Yutao Zhu +5Cross-Modal Knowledge Distillation Unified Multimodal Models
1Renmin University of China · 2Tencent
S-CARD-CMSA: A Score-Aware Candidate Archive with Density-Filtered Reporting for Multimodal Optimization
Jul 15, 2026Dikshit ChauhanEvolutionary Optimization Multimodal Optimization
Department of Electrical and Computer Engineering, National University of Singapore
Bridging Interleaved Multi-Modal Reasoning as a Unified Decision Process
Jul 4, 2026Zican Hu, Xuyang Hu, Yiming Liu +10Unified Multimodal Models Multimodal Reasoning
Nanjing University · Shanghai AI Laboratory · Tsinghua University +5
\chisao{}: A GPU-Native Parallel Optimizer for Multimodal Black-Box Functions via Convergence-Anticonvergence Oscillation
Jun 24, 2026Ira WolfsonGPU Acceleration Black-Box Optimization
Department of Electronics and Electrical Engineering Braude College of Engineering, Karmiel, Israel
MSC-CMA-ES: Structure-Aware Restarts for CMA-ES via Cyclic Nearest-Better Basin Discovery
Jun 14, 2026Dimitar Nedanovski, Svetoslav Nenov, Dimitar PilevEvolutionary Optimization Covariance Matrix Adaptation Evolution Strategy
Faculty of Mathematics and Informatics Sofia University St. Kliment Ohridski James Baucher Blvd., 1164 Sofia, Bulgaria · Department of Mathematics University of Chemical Technology and Metallurgy Sofia, Bulgaria · Department of Informatics University of Chemical Technology and Metallurgy Sofia, Bulgaria
GLENS: Global Search via Learning from Solver Iterates with Diffusion Models
May 29, 2026Anjian Li, Bartolomeo Stellato, Ryne BeesonLearning to Optimize Diffusion Sampling
Department of Electrical and Computer Engineering, Princeton University, Princeton, 08544, NJ, USA. · Department of Operations Research and Financial Engineering, Princeton University, Princeton, 08544, NJ, USA. · Department of Mechanical and Aerospace Engineering, Princeton University, Princeton, 08544, NJ, USA.
DIVA: Harnessing the Representation Divergence in Unified Multimodal Models for Mutual Reinforcement
May 25, 2026Renjie Lu, Xulong Zhang, Xiaoyang Qu +2Multimodal Disentangled Representation Learning Unified Multimodal Models
Ping An Technology (Shenzhen) Co., Ltd., Shenzhen, China · University of Science and Technology of China, Hefei, China.
Mapping the Fitness Landscape: A Structure-Guided Approach to Multi-Modal Optimization
May 18, 2026Meng Xiang, Pei YanEvolutionary Optimization Multi-Objective Evolutionary Optimization
University of Aizu Aizuwakamatsu, Japan
Second-Order Multi-Level Variance Correction for Modality Competition in Multimodal Models
May 15, 2026Yishun Lu, Wes ArmourSecond-Order Optimization Multimodal Optimization
University of Oxford, Oxford, United Kingdom
MM-OptBench: A Solver-Grounded Benchmark for Multimodal Optimization Modeling
May 12, 2026Zhong Li, Qi Huang, Yuxuan Zhu +6Multimodal Optimization Synthetic Benchmark Generation
Great Bay University · Leiden University · Rensselaer Polytechnic Institute +1
Stein Variational Black-Box Combinatorial Optimization
Apr 17, 2026Thomas Landais, Olivier Goudet, Adrien Goëffon +2Stochastic Optimization Black-Box Optimization
LERIA, Université d’Angers, 2 Boulevard Lavoisier, Angers 49045, France
Less Data, Faster Convergence: Goal-Driven Data Optimization for Multimodal Instruction Tuning
Mar 12, 2026Rujie Wu, Haozhe Zhao, Hai Ci +1Video Understanding Instruction-Tuning Data Selection
School of Computer Science, Peking University, Beijing, China · Institute for Artificial Intelligence, Peking University, Beijing, China · State Key Laboratory of General Artificial Intelligence, Peking University, Beijing, China +2
CAT-GS: Balanced Multimodal Learning via Calibrated Gating and Fusion Surgery
Date pendingMahir Shahriar Tamim, Sharjil Khan, Md. Samiul Alim +3Cross-Modal Learning Modality Imbalance
Department of Electrical and Computer Engineering, North South University, Dhaka, Bangladesh

Sep 23, 2026, cs.LGJ/K move · Enter open · S save

Confidence Falls Short: Asymmetric Certainty Gains from Optimization Hinder Multimodal Classification

Longfei Huang, Xiangyu Wu, Yang Yang

Nanjing University of Science and Technology · Alibaba Group

Modality Imbalance Multimodal Optimization Multimodal Classification Multimodal Fusion

Open paper AskPDF arXiv

Multimodal learning (MML) falls into the optimization dilemma due to the modality imbalance phenomenon, leading to suboptimal overall performance in practice. While many attempts primarily focus on balancing the optimization dynamics across modalities to address this issue, we identify a subtle yet critical flaw: optimization yields asymmetric gains in predictive certainty, with the strong modality more confident than the weak one, driving imbalanced modality contributions. In this paper, our analysis reveals that this flaw stems from unimodal characteristics rather than multimodal learning, and this confidence discrepancy can be corrected by positive cross-modal intervention. Based on this insight, we propose multimodal Max Confidence Regularization (MaxCR) to dynamically intervene in modality semantic confidence. Specifically, the semantic confidence of each modality is tracked using a nonlinear sparsity measure. We then design max suppression and max excitation based on this measure to regularize strong and weak modalities, respectively. They penalize and encourage the top-1 confidence, thereby constraining multimodal prediction. To this end, strong and weak modalities are expected to make calibrated confidence, thereby improving the overall performance. Empirical experiments on widely used datasets reveal the superiority of our method through comparison with various state-of-the-art (SOTA) multimodal learning baselines.

Momentum

Related topics

Latest papers 21