Multimodal Disentangled Representation Learning

Follow this topic Ask about this topic

This field studies how to represent information from sources such as images and text, MRI and PET, or audio and video without losing either cross-modal meaning or each source’s distinctive signals. Common approaches use shared and private latent spaces, variational autoencoders, contrastive objectives, orthogonality or mutual-information constraints, and attention or gated fusion. Researchers evaluate these methods on tasks including sentiment analysis, medical prediction and image translation, retrieval, and robustness to missing modalities or new domains. A central tension is that stronger alignment can suppress useful modality-specific information, while the benefits of disentanglement vary across tasks, architectures, and evaluation measures.

Latest papers

54 papers10 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

Learning Invariant Modality Representation for Robust Multimodal Learning from a Causal Inference Perspective
Apr 20, 2026Sijie Mai, Shiqin HanDisentangled Representation Learning Causal Representation Learning
School of Computer Science, South China Normal University
Learning What's Real: Disentangling Signal and Measurement Artifacts in Multi-Sensor Data, with Applications to Astrophysics
Apr 10, 2026Pablo Mercader-Perez, Carolina Cuesta-Lazaro, Daniel Muthukrishna +5Astrophysics Multimodal Disentangled Representation Learning
Massachusetts Institute of Technology · Flatiron Institute, Simons Foundation · Institute for Advanced Studies +5
M-IDoL: Information Decomposition for Modality-Specific and Diverse Representation Learning in Medical Foundation Model
Apr 10, 2026Yihang Liu, Longzhen Yang, Jiaxiong Yang +3Medical Imaging Foundation Models Medical Image Representation Learning
School of Computer Science and Technique, Tongji University, Shanghai 201804, China. · School of Communications and Electronic Engineering, East China Normal University, Shanghai 200241, China.
Bridging MRI and PET physiology: Untangling complementarity through orthogonal representations
Apr 8, 2026Sonja Adomeit, Kartikay Tehlan, Lukas Förner +7Medical Imaging Multimodal Fusion
Dept. of diagnostic and interventional Radiology and Neuroradiology, University Hospital Augsburg, Germany · Digital Medicine, University Hospital Augsburg, Germany · Chair for Computer Aided Medical Procedures and Augmented Reality, Technical University of Munich, Germany +4
Hierarchical Contrastive Learning for Multimodal Data
Apr 7, 2026Huichao Li, Junhan Yu, Doudou ZhouRepresentation Learning Multimodal Contrastive Learning
Department of Mathematical Sciences, University of Chinese Academy of Sciences · Department of Statistics and Data Science, National University of Singapore
Modality-Aware and Anatomical Vector-Quantized Autoencoding for Multimodal Brain MRI
Apr 6, 2026Mingjie Li, Edward Kim, Yue Zhao +2Vector-Quantized Variational Autoencoder Medical Image Representation Learning
Stanford University
Multimodal Structure Learning: Disentangling Shared and Specific Topology via Cross-Modal Graphical Lasso
Apr 5, 2026Fei Wang, Yutong Zhang, Xiong WangMultimodal Learning Multimodal Graph Learning
Department of Applied Mathematics and Statistics, Stony Brook University · School of Mathematics, Sichuan University · School of Computer Science and Technology, USTC
Disentangled Dual-Branch Graph Learning for Conversational Emotion Recognition
Apr 3, 2026Chengling Guo, Yuntao Shou, Tao Meng +3Emotion Recognition in Conversations Multimodal Emotion Recognition
College of Computer and Mathematics, Central South University of Forestry and Technology · Changsha Hospital for Maternal and Child Health Care · Department of Computer Science, State University of New York
CMCC-ReID: Cross-Modality Clothing-Change Person Re-Identification
Apr 3, 2026Haoxuan Xu, Hanzi Wang, Guanglin NiuVisible-Infrared Person Re-Identification Person Re-Identification
School of Artificial Intelligence, Beihang University, Beijing, China · School of Informatics, Xiamen University, Xiamen, Fujian, China
DMMRL: Disentangled Multi-Modal Representation Learning via Variational Autoencoders for Molecular Property Prediction
Mar 22, 2026Long Xu, Junping Guo, Jianbo Zhao +2Molecular Representation Learning Multimodal Disentangled Representation Learning
Guangxi Key Lab of Human-machine Interaction and Intelligent Decision, Nanning Normal University, Nanning, China · College of Big Data and Software Engineering, Zhejiang Wanli University, Ningbo, China
Disentangle-then-Align: Non-Iterative Hybrid Multimodal Image Registration via Cross-Scale Feature Disentanglement
Mar 20, 2026Chunlei Zhang, Jiahao Xia, Yun Xiao +2Cross-Modal Image Registration Deformable Image Registration
Faculty of Engineering and IT, University of Technology Sydney, Australia · School of Artificial Intelligence, Anhui University, Hefei, China · School of Computer Science and Technology, Anhui University, Hefei, China
Cheers: Decoupling Patch Details from Semantic Representations Enables Unified Multimodal Comprehension and Generation
Mar 13, 2026Yichen Zhang, Da Peng, Zonghao Guo +19Unified Multimodal Models Multimodal Generation
Tsinghua University · Xi’an Jiaotong University · University of Chinese Academy of Sciences
Multiscale Structure-Guided Latent Diffusion for Multimodal MRI Translation
Mar 13, 2026Jianqiang Lin, Zhiqiang Shen, Peng Cao +3Medical Image Synthesis Latent Diffusion Models
Northeastern University, Shenyang, China · Key Laboratory of Intelligent Computing in Medical Image, Shenyang, China · Computer Science and Engineering, Northeastern University, Shenyang, China +5
IDRL: An Individual-Aware Multimodal Depression-Related Representation Learning Framework for Depression Diagnosis
Mar 12, 2026Chongxiao Wang, Junjie Liang, Peng Cao +2Depression Detection Multimodal Fusion
School of Computer Science and Engineering, Northeastern University, Shenyang, China · Key Laboratory of Intelligent Computing in Medical Image of Ministry of Education, Northeastern University, Shenyang, China · National Frontiers Science Center for Industrial Intelligence and Systems Optimization, Shenyang, China +1
Visually-Guided Controllable Medical Image Generation via Fine-Grained Semantic Disentanglement
Mar 11, 2026Xin Huang, Junjie Liang, Qingshan Hou +4Medical Image Synthesis Controllable Image Generation
Computer Science and Engineering, Northeastern University, Shenyang, China · Key Laboratory of Intelligent Computing in Medical Image of Ministry of Education, Northeastern University, Shenyang, China · National Frontiers Science Center for Industrial Intelligence and Systems Optimization, Shenyang, China +2
RTFDNet: Fusion-Decoupling for Robust RGB-T Segmentation
Mar 10, 2026Kunyu Tan, Mingjian LiangSemantic Segmentation Multimodal Robustness
Kunyu Tan and Mingjian Liang are independent researchers.
Aligning the True Semantics: Constrained Decoupling and Distribution Sampling for Cross-Modal Alignment
Mar 5, 2026Xiang Ma, Lexin Fang, Litian Xu +1Cross-Modal Alignment Vision-Language Alignment
Shandong University, Jinan, China · The University of Exeter, Exeter, United Kingdom
TSEmbed: Unlocking Task Scaling in Universal Multimodal Embeddings
Mar 5, 2026Yebo Wu, Feng Liu, Ziwei Xie +4Multimodal Embedding Multimodal Disentangled Representation Learning
State Key Laboratory of IOTSC, University of Macau · OPPO Research Institute
Bridging the gap between Performance and Interpretability: An Explainable Disentangled Multimodal Framework for Cancer Survival Prediction
Mar 2, 2026Aniek Eijpe, Soufyan Lakbir, Melis Erdal Cesur +4Cancer Survival Prediction Multimodal Survival Analysis
AI Technology for Life, Department of Information and Computing Sciences, Department of Biology, Utrecht University, Utrecht, The Netherlands · Department of Metabolic Diseases, Wilhelmina Children’s Hospital, University Medical Center Utrecht, Utrecht, The Netherlands · Regenerative Medicine Center Utrecht, Utrecht, The Netherlands +3
SFL-Net: Source-Factorized Latent Representation Learning for Multi-Contrast MRI to Tau-PET Synthesis
Feb 26, 2026Agamdeep S. Chopra, Caitlin Neher, Tianyi Ren +3Disentangled Representation Learning Multimodal Disentangled Representation Learning
Department of Mechanical Engineering, University of Washington, Seattle, WA, USA · Department of Radiology, University of Washington, Seattle, WA, USA
CG-DMER: Hybrid Contrastive-Generative Framework for Disentangled Multimodal ECG Representation Learning
Feb 24, 2026Ziwei Niu, Hao Sun, Shujun Bian +4Electrocardiogram Representation Learning Multimodal Disentangled Representation Learning
Department of Biomedical Engineering, National University of Singapore, Singapore, Singapore · College of Computer Science and Technology, Zhejiang University, Hangzhou, China · College of information science and engineering, Ritsumeikan university, ibaraki Osaka, Japan +1
CLCR: Cross-Level Semantic Collaborative Representation for Multimodal Learning
Feb 23, 2026Chunlei Meng, Guanhong Huang, Rong Fu +3Cross-Modal Representation Learning Multimodal Learning
Fudan University · Shantou University · University of Macau +1
Tri-Subspaces Disentanglement for Multimodal Sentiment Analysis
Feb 23, 2026Chunlei Meng, Jiabin Luo, Zhenglin Yan +4Multimodal Sentiment Analysis Multimodal Disentangled Representation Learning
Fudan University · Peking University · University of Macau
Orthogonalized Multimodal Contrastive Learning with Asymmetric Masking for Structured Representations
Feb 16, 2026Carolin Cissee, Raneen Younis, Zahra AhmadiMultimodal Embedding Multimodal Contrastive Learning
Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, Hannover, Germany · Lower Saxony Center for AI and Causal Methods in Medicine (CAIMed), Hannover, Germany

Apr 20, 2026J/K move · Enter open · S save

Learning Invariant Modality Representation for Robust Multimodal Learning from a Causal Inference Perspective

Sijie Mai, Shiqin Han

School of Computer Science, South China Normal University

Disentangled Representation Learning Causal Representation Learning Multimodal Robustness Multimodal Disentangled Representation Learning Sentiment Analysis Multimodal Emotion Recognition+2

Open paper AskPDF arXiv

Multimodal affective computing aims to predict humans' sentiment, emotion, intention, and opinion using language, acoustic, and visual modalities. However, current models often learn spurious correlations that harm generalization under distribution shifts or noisy modalities. To address this, we propose a causal modality-invariant representation (CmIR) learning framework for robust multimodal learning. At its core, we introduce a theoretically grounded disentanglement method that separates each modality into causal invariant representation' and environment-specific spurious representation' from a causal inference perspective. CmIR ensures that the learned invariant representations retain stable predictive relationships with labels across different environments while preserving sufficient information from the raw inputs via invariance constraint, mutual information constraint, and reconstruction constraint. Experiments across multiple multimodal benchmarks demonstrate that CmIR achieves state-of-the-art performance. CmIR particularly excels on out-of-distribution data and noisy data, confirming its robustness and generalizability.