stat.MLMay 21, 2025

Robust Multimodal Learning via Entropy-Gated Contrastive Fusion

Authors: Leon Chlon, Maggie Chlon, MarcAntonio M. Awada

Abstract

Real-world multimodal systems routinely face missing-input scenarios, and in reality, robots lose audio in a factory or a clinical record omits lab tests at inference time. Standard fusion layers either preserve robustness or calibration but never both. We introduce Adaptive Entropy-Gated Contrastive Fusion (AECF), a single light-weight layer that (i) adapts its entropy coefficient per instance, (ii) enforces monotone calibration across all modality subsets, and (iii) drives a curriculum mask directly from training-time entropy. On AV-MNIST and MS-COCO, AECF improves masked-input mAP by +18 pp at a 50% drop rate while reducing ECE by up to 200%, yet adds 1% run-time. All back-bones remain frozen, making AECF an easy drop-in layer for robust, calibrated multimodal inference.

Figures & tables

Explore similar work

CardsList
  1. Conformal Fusion Under Missing Modalities

    Aug 7, 2026Alireza MoayedikiaMissing-Modality LearningGroup-Conditional Conformal Prediction

  2. Efficient Multimodal Inference through Adaptive Acquisition and Sequential Fusion

    Oct 5, 2026Payal Mohapatra, Haodong Yang, Yueyuan Sui +3Efficient Multimodal InferenceMultimodal Classification

  3. MER-DG: Modality-Entropy Regularization for Multimodal Domain Generalization

    May 3, 2026Yavuz Yarici, Ghassan AlRegibMultimodal Domain GeneralizationDomain Generalization