Multimodal Learning

Follow this topic Ask about this topic

Recent momentum

-72%

9 papers in the last 28 days · 0.2% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

5 new papers

A weekly snapshot of new work published in Multimodal Learning.

Period ending 2026-09-07

2 new papers

A weekly snapshot of new work published in Multimodal Learning.

160 papers

Latest in Multimodal Learning

Open your feed →

TwinICL: Diagnosing Multimodal In-Context Learning through Paired Counterfactuals
Sep 14, 2026Zihan Xue, Po-Yi Lu, Serhii Honcharenko +5Multimodal Learning In-Context Learning
When Does Text Inform? Benchmarking Information-Theoretic Metrics for Multimodal Time-Series Forecasting
Sep 12, 2026Emma Andrews, Gianmarco MengaldoTime Series Forecasting Multimodal Learning
Exploring Diffusion Transformers for Cross-Modal Augmentation in Multimodal Brain State Decoding
Sep 12, 2026Ziwei Wang, Xingyi He, Hongbin Wang +3Cross-Modal Attention Multimodal Learning
Studying Image Tokenizers as Visual Languages in Unified Multimodal Models
Sep 8, 2026Siting Li, Zhengyang Wang, Simon Shaolei Du +2Multimodal Learning Tokenizer
Noise Adaptive Streaming Audio-Visual Speech Token Enhancement for Robust Full-Duplex Spoken Dialogue Models
Sep 8, 2026Bella Godiva, Yeonju Kim, Yong Man RoSpeech Enhancement Auditory Attention
When Semantically Consistent Encoding Meets View-Label Heterogeneity Modeling: A Unified Framework for Incomplete Multi-View Multi-Label Learning
Sep 7, 2026Chengliang Liu, Bo Li, Bob Zhang +3Multi-Label Classification Multi-View
Contribution-Aware Bandwidth Allocation for Multimodal Split Learning
Sep 1, 2026Iason Ofeidis, Leandros TassiulasMultimodal Learning Wireless Resource Allocation
Joint Training Is Not Enough: Conditioned Cross-Granularity Training for Multimodal Document Understanding
Sep 1, 2026Chengguang Gan, Yunhao Liang, Hanjun Wei +2Multimodal Query Multimodal Learning
Cross-Regional Grapevine Cold Hardiness Prediction via Learned Multimodal Latent Representations
Aug 31, 2026William Solow, Paola Pesantez-Cabrera, Markus Keller +3Transfer Learning Latent Representation
Multimodal Shared Latent Representation of Narration, Microscope and iOCT Images for Phase Recognition in Vitreoretinal Surgery
Aug 31, 2026Onur Izmitlioglu, Shervin Dehghani, Tarek Ghannoum +2Optical Coherence Tomography Microscopy
When Is a Task Vector Enough? An Empirical Theory of Implicit Multimodal ICL
Aug 13, 2026Jiaqian LiMultimodal Learning In-Context Learning
MAG: MAnifold Guided Semi-Supervised Multi-modal In-Context Learning
Aug 13, 2026Zirui Cheng, Xun Xu, Tiankai Chen +7Large Multimodal Models Multimodal Learning
LEMUR: Latent Entropy-aware Multimodal Unlearning via Visual-anchored Reasoning Redirection
Aug 12, 2026Xinhao Zhong, Yuxia Qiao, Junhao Li +3Large Language Model Unlearning Multimodal Reasoning
PRMU: A Corpus-Free Benchmark for Person-Centric Knowledge Unlearning in Multimodal Large Language Models
Aug 11, 2026Huafeng Chen, Yueming Lyu, Ziyuan Chen +4Large Language Model Unlearning Large Multimodal Models
FiGuRO: Intrinsic Dimension Estimation for Multi-Modal Data
Aug 11, 2026Viktoria Schuster, Sana Tonekaboni, Caroline UhlerIntrinsic Dimensionality Multimodal Learning
Rationale-Guided Learning for Multimodal Emotion Recognition
Aug 11, 2026Sujung Oh, Jung Uk Kim, Sangmin LeeMultimodal Emotion Recognition Multimodal Reasoning
UniMod: Enhancing Multi-Modal Medical Diagnosis through Cross-Modality and Within-Modality Alignment
Aug 10, 2026Zijian Gu, Weikai Lin, Shuang Zhou +2Multimodal Clinical Data Multimodal Learning
Multimodal Item Parameter Estimation using Simulated Response Probabilitie
Aug 10, 2026Christopher Ormerod, YoungKoung KimItem Response Theory Multiple-Choice Questions
Hyperbolic Multimodal Continual Learning
Aug 10, 2026Jiahong Liu, Ming Shen, Xiaohao Liu +4Hyperbolic Learning Efficient Continual Learning
GRACE: LLM-Grounded Semantic Metric Spaces for Scalable Mixed-Data Clustering
Aug 8, 2026Zihua Yang, Zhencheng Xie, Junyang Chen +4Clustering Cluster
A Hierarchical Energy-Based Model for Multimodal Cognition
Aug 8, 2026Subir VarmaPredictive Coding Multimodal Learning
GAUGE: Granularity-Adaptive Counterfactual Gating of Evidence for Incomplete Multimodal Classification
Aug 6, 2026Yunping Shi, En Yu, Kairui Guo +1Missing Modalities Multimodal Learning
iStructTab: Structured Feature Sequencing for Multimodal Learning of Image and Tabular Data
Aug 5, 2026Al Zadid Sultan Bin Habib, Md Younus Ahamed, Prashnna Gyawali +2Multimodal Learning Tabular Learning
Beyond Simply Environment Scaling: Designing Effective Environment Distributions for Multimodal Agent Learning
Aug 4, 2026Kejian Zhu, Zhuoran Jin, Dongqi Huang +4Multimodal Agents Multi-Agent Reinforcement Learning
Modality Agreement- and Conflict-Aware Prototype Hypergraph Learning for Multimodal Intent Understanding
Aug 4, 2026Mohnish Raj, Suraj Kumar, Soumi Chattopadhayay +2Multimodal Learning Text Modality
CIGTSurv: Clinical Information Guided Tri-modal Survival Prediction with Local Prototype Association and Global Feature Alignment
Aug 4, 2026Jing Dai, Qibin Zhang, Weiwei Zhou +4Survival Predictions The Cancer Genome Atlas Datasets
Understanding and Overcoming Cross-modal Fusion Bias in Multimodal Anomaly Detection From A Fisher Information Perspective
Aug 2, 2026Kaifang Long, Lianbo Ma, Liming Liu +1Cross-Modal Fusion Multimodal Learning
CALM-AH: An ABAW11-Calibrated Multimodal Ensemble with Reliability-Gated Multi-Expert Consensus for Video-Level Ambivalence and Hesitancy Recognition
Jul 31, 2026Wenzhuo Sun, Mingjian Liang, Richard Attfield +3Multimodal Evaluation Ambiguity
GALA: Generative Aligned Learning for Adaptive Multimodal Representation in the Taobao Shangou Recommender System
Jul 31, 2026Jiping Liu, Zhongmin Zhang, Zisen Sang +7Multimodal Learning Generalizable Ranking Purpose
MMFGU: Multimodal Federated Graph Unlearning
Jul 30, 2026Haodong Lu, Zekai Chen, Weiwei Ji +5Federated Graph Learning Multimodal Learning
FedOGL: Combating Catastrophic Forgetting in Federated Open-World Multimodal Graph Learning
Jul 30, 2026Zekai Chen, Haodong Lu, Shihao Li +5Federated Graph Learning Multimodal Graph
Understanding Submodular Information Measure Based Objectives for Representation Learning: A Variance and Separation Perspective
Jul 30, 2026Rishabh Iyer, Truong Pham, Anay MajeeMutual Information Submodular Maximization
Temporally Centered SIGReg Improves Multi-Task LeWorldModel Learning: From Analysis to Method
Jul 29, 2026Chang Liu, Fei Suo, Yanzhou Jin +3Multi--Task Learning Regularization Methods
SCALPEL: Semantic Cross-modal Alignment via LLM-Powered Encoder Learning for Medical Vision-Language Representation
Jul 29, 2026Yunzhan Fu, Enyu Bao, Xiangyu Shen +4Medical Vision-Language Models Multimodal Clinical Data
Regularizing modality contribution drift in multimodal continual learning
Jul 29, 2026Zhen Zhang, Jielei Chu, Bin Liu +1Multimodal Learning Efficient Continual Learning
From Cellular Responses to Pharmacological Domains: Multimodal Zero-Shot Drug Representation Learning
Jul 28, 2026Jintao Huang, Lu Leng, Ziyuan YangDrug-Drug Interactions Multimodal Learning
CLBench-V: Evaluating Multimodal Context Learning from Grounding to Knowledge Acquisition
Jul 28, 2026Lai Wei, Chengqi Li, Jiapeng Li +3Multimodal Learning In-Context Learning
Co-Learning for Missing Arbitrary Modalities in Multi-modal Classification
Jul 27, 2026Francisco Mena, Dino Ienco, Roberto Interdonato +2Missing Modalities Multimodal Learning
FedTaste: Topology-Aware Structural Transfer for Multimodal Federated Learning with Missing Modalities
Jul 25, 2026Haochen Liang, Jie Zhang, Hideya OchiaiFederated Fine-Tuning Missing Modalities
Toward Generalizable Cognitive Impairment Detection with Speech-Based Multimodal Large Language Models
Jul 23, 2026Yingchao Huang, Xin Wang, Yuhan Su +1Alzheimer Multimodal Learning
Adaptive Confidence-weighted Expansion for Trustworthy Multi-Omics Multimodal Fusion
Jul 22, 2026Mohammad Raahemi, Ali Sekhavati, Alireza Maleki +1Multi-Modal Fusion Multimodal Learning
Robust Multi-View Classification under Noisy Supervision via Global Anchor Consensus
Jul 20, 2026Yuliang Yang, Hongzhe Zhang, Huiru WangNoisy Labels Multi-View
Coordinated Disentanglement with Iterative Mode Discovery Under Hidden Correlations
Jul 19, 2026Rong Hu, Ling ChenDisentanglement Multimodal Learning
MultiLoReFT: Decoupling Shared and Modality-Specific Subspaces in Multimodal Learning via Low-Rank Representation Fine-Tuning
Jul 18, 2026Sana Tonekaboni, Viktoria Schuster, Caroline UhlerMultimodal Learning Text Modality
Capacity and Redundancy Trade-offs in Multi-Task Learning
Jul 17, 2026Asif KhanMulti--Task Learning Redundancy
AlphaWiSE: Adaptive Weight Interpolation for Continual Multimodal Representation Learning
Jul 16, 2026Sarthak Jain, Qiran Hu, Zhen Zhu +1Cross-Modal Efficient Continual Learning
LATTICE: Graph Self-Supervised Learning for Multimodal Spatial Omics Integration
Jul 15, 2026Jagan Mohan Reddy Dwarampudi, Veena Kochat, Suresh Satpati +2Multi-Omic Framework Transcriptomics
Multi-Agent Collaborative Reasoning with Tool-Augmented Evidence for Urban Region Profiling
Jul 15, 2026Xixuan Hao, Yutian Jiang, Jiabo Liu +4Multi-Agent Reasoning Street-View Imagery
A Calibrated Multimodal Ensemble for Ambivalence/Hesitancy Recognition: System Description and Private-Test Submission Strategy
Jul 13, 2026Josep Cabacas-Maso, Ismael Benito-Altamirano, Carles VenturaAffective Computing Multimodal Evaluation
MMRM: A Multiplex Multimodal Representation Model for Product Ranking in E-commerce Search
Jul 13, 2026Zhen-Lin Chen, Maosen Sheng, Peng Lin +4Generalizable Ranking Purpose Multimodal Learning
On the modality gap and the contrastive loss in multi-modal representation learning
Jul 12, 2026Fabian Mager, Hiba Nassar, Lars Kai HansenMultimodal Contrastive Learning Modality Gap
ShapKO: Shapley-Adaptive Modality Knockout for Robust Multimodal Learning
Jul 10, 2026Nusrat Binta Nizam, Fengbei Liu, Sunwoo Kwak +3Multimodal Learning Multimodal Clinical Data
One-Shot Multimodal Learning from Demonstration with Force-Constrained Elastic Maps
Jul 10, 2026Brendan Hertel, Jonathan Spanos, Navya Garg +1Force Sensing Real-World Manipulation Tasks
Unpaired Joint Distribution Modeling via Multi-Scale Image Representations
Jul 9, 2026Yihang Zou, Hui Zhang, Zuowei Shen +1Multiscale Representation Learning
The Importance of Encoder Choice:A Tabular-Image Study
Jul 8, 2026Ilia Koloiarov, Diego Coello de Portugal Mecke, Vijaya Krishna Yalavarthi +2Tabular Learning Multimodal Learning
General Incomplete Multimodal Learning via Dynamic Quality Perception
Jul 8, 2026Xiangyu Meng, Shicai WeiMissing Modalities Multimodal Learning
Integrating Neural Encoders in Bayesian Generalized Linear Mixed Models for Multimodal Data
Jul 6, 2026Yuankang Zhao, Youngsoo Baek, Felipe A. Medeiros +2Generalized Linear Model Multimodal Learning
Hidden Forgetting in Continual Multimodal Learning: When Accuracy Survives but Grounding Fails
Jul 2, 2026Qianyu Chen, Canran Xiao, Runxuan TangMultimodal Learning Catastrophic Forgetting
Deep Multitask Learning for Mixed-Type Outcomes with Shared Sparsity
Jul 1, 2026Huichao Li, Tong Wang, Sanguo Zhang +1Multi--Task Learning Predictors
Multimodal Continuous Reasoning via Asymmetric Mutual Variational Learning
Jul 1, 2026Shijie Li, Yilin Gao, Siyuan Yang +7Large Multimodal Models Multimodal Learning

Sep 14, 2026 · cs.CVJ/K move · Enter open · S save

TwinICL: Diagnosing Multimodal In-Context Learning through Paired Counterfactuals

Zihan Xue, Po-Yi Lu, Serhii Honcharenko, Zih-Ching Chen+4

In-context learning (ICL) enables models to infer tasks from demonstrations, but existing benchmarks generally lack matched text and image versions needed to compare ICL performance across modalities. We introduce TwinICL, a procedurally generated benchmark providing such pairs for controlled comparison. Across six open-weight models and 38 tasks, multimodal ICL consistently underperforms text-only ICL, with gaps varying by task family. To test whether this gap can be recovered, we target visual access, task framing, and reasoning through three interventions. Their combination recovers strong multimodal ICL performance on a diagnostic subset, despite limited or inconsistent individual effects. To distinguish difficulties in executing tasks from those in inferring them, we evaluate models with explicit task instructions, revealing a modality gap even when the task is known. We then examine how adding demonstration inputs and outputs reshapes this gap, highlighting demonstrations' dual role as additional context to process and evidence about the task. The dataset is available at https://github.com/lab-flair/TwinICL.

Multimodal Learning In-Context Learning

Open paper Ask about it PDF