Multimodal SSL

SSL: Self-Supervised Learning

Latest papers

27 papers10 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Multimodal Information Interactions Bottleneck for Multimodal Self-Supervised Learning
Nov 5, 2026Liangjian Wen, Jiangtao Zheng, Jiang Duan +8Multimodal SSL Multimodal Learning
School of Computing and Artificial Intelligence, Southwestern University of Finance and Economics, Chengdu, China · Artificial Intelligence and Digital Finance Key Laboratory of Sichuan Province · Chengdu Everimaging Science and Technology Co., Ltd. +8
Prism: Bidirectional Self-Teaching for Unified Multimodal Models
Nov 4, 2026Peng Zhang, Qian Chen, Xianyin Zhang +6Unified Multimodal Models Multimodal Generation
Xi’an Jiaotong University · Qwen DianJin Team, Alibaba Cloud Computing
DenseTwins: Dense Modality Decoupling for Multimodal Self-Supervision
Nov 3, 2026Carlo Sgaravatti, Daniele Kopyshevskiy, Davide Secco +1Multimodal SSL Cross-Modal Representation Learning
Politecnico di Milano · Sorbonne University · Universit`a Bocconi
ILR-SMO: Iterative Latent Refinement for Robust Spatial Multi-Omics Integration
Nov 2, 2026Anqi Yu, Xudong Xu, Jianzhi Lu +5Multi-Omics Data Integration Reliability-Aware Multimodal Fusion
College of Computer Science and Artificial Intelligence, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai 200438, China
SomaAlign: Reliability-Aware Multimodal Representation Learning for Soft Robotic Grasping
Nov 2, 2026Haoyuan Wang, Shuai Peng, Yongkai Liao +4Reliability-Aware Multimodal Fusion Robotic Grasping
the Hubei Key Laboratory of Brain-inspired Intelligent Systems and the Key Laboratory of Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology (HUST), Wuhan 430074, Hubei, China. · Department of Innovation Center, China Ship Development and Design Center, Wuhan 430064, Hubei, China.
Latent Spectral Prompting for Multimodal Hyperspectral Object Tracking
Nov 2, 2026Wangshu Cai, Jun Zhou, Lei Wang +3Multimodal Visual Object Tracking Cross-Modal Representation Transfer
Griffith University, Australia · East China Normal University, China · Bangladesh Agricultural University, Bangladesh
Learning to Group and Order: Cross-Instance Self-Supervised RL for Vision-Centric MLLMs
Oct 31, 2026Songming Yang, Hong-Tao Yu, Xiu-Shen WeiRL for VLMs Self-Supervised RL
School of Computer Science and Engineering, Southeast University · Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Southeast University
LoRe-MIM: Lost-or-Replaced Masked Image Modeling for Self-Supervised Representation Learning in Whole-Body PET/CT
Oct 29, 2026Amartya Bhattacharya, Bashirul Azam Biswas, Biratal Raj Wagle +4Medical Image Representation Learning Multimodal SSL
Department of Biomedical Data Science, Geisel School of Medicine at Dartmouth, Hanover, NH 03755, USA · Department of Urology, Yale University, New Haven, CT, USA · Radiology, Dartmouth Hitchcock Medical Center, Lebanon, NH 03766 , USA +1
Quantum-Inspired Convolutional Networks for High-Resolution Pseudo-Label Generation from Multimodal Earth Observation Data
Oct 29, 2026Abhishek Singh, Giovanni Cuozzo, Alexander Jacob +1Pseudo-Labeling Weakly Supervised Semantic Segmentation
Institute for Earth Observation, Eurac Research, Bolzano, Italy
SpectralTF-MAE: A Pre-trained Spectral Foundation Model for Tokamak Fusion Plasma Diagnostics
Oct 28, 2026Xiao Wang, Zhijie Lu, Duoqing Yang +7Representation Learning Self-Supervised Pre-Training
School of Computer Science and Technology, Anhui University, Hefei 230601, China · Institute of Plasma Physics, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China · New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China
A-RSI: Adaptive Task Refresh for Zero-Data Multimodal Self-Improvement
Oct 28, 2026Yu Liu, Youxing Li, Yuedong Tan +17Language Model Self-Improvement Self-Training for Language Models
SleepPairNet: Learning Multimodal Pediatric Sleep Representations with Random-Pair Cross-Attention
Oct 19, 2026Saurav Raj Pandey, Harlin LeeMultimodal Pretraining Cross-Modal Representation Learning
Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA · School of Data and Information Sciences, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
MM-LeJEPA: Correlated Gaussian Regularization for Multimodal Joint-Embedding Predictive Architectures
Oct 17, 2026Haowen Bai, Rui Gong$, Meichen Liu +3Multimodal SSL Joint-Embedding Predictive Architecture
Nanyang Technological University · Hyundai Motor Group Innovation Center Singapore (HMGICS)
HRIL: Learning Multimodal Synergy via Higher-Order Tensor Modeling
Oct 8, 2026Qun Dai, Liangjian Wen, Jiang Duan +7Multimodal SSL Multimodal Learning
School of Computing and Artificial Intelligence, Southwestern University of Finance and Economics · Artificial Intelligence and Digital Finance Key Laboratory of Sichuan Province · Chengdu Everimaging Science and Technology Co., Ltd +5
LeAVJEPA: A Minimalist Architecture for Audio-Visual Self-Supervised Learning
Oct 5, 2026Benjamin Robson, Santeri Mentu, Wenshuai Zhao +1Audio-Visual Representation Learning Joint-Embedding Predictive Architecture
ELLIS Institute Finland Aalto University
On the Geometry of Multimodal Saturation: Riemannian VICReg
Oct 5, 2026Nessim Ben Abbes, Duc Han Le, Sabri Mtibaa +1Self-Supervised Learning Geometric Representation Learning
Protectline · Télécom Paris
SynCo: Learning Cross-Modal Synergy by Contrasting Interaction Residuals
Sep 26, 2026Yavuz Yarici, Ghassan AlRegibCross-Modal Representation Learning Multimodal Contrastive Learning
OLIVES, CSIP, School of ECE Georgia Institute of Technology Atlanta, GA, USA
Synergistic Information Disentanglement for Omni-modal Slide Representation Learning in Computational Pathology
Sep 2, 2026Mingxin Liu, Chengfei Cai, Anwen Lu +5Multimodal SSL Medical Image Representation Learning
Jiangsu Key Laboratory of Intelligent Medical Image Computing, School of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing, China · Jiangsu Key Laboratory of Intelligent Drug Screening and Repositioning, College of Information Engineering, Taizhou University, Taizhou, China · College of Bioinformatics Science and Technology, Harbin Medical University, Harbin, China
MUST-PET: MUltimodal Self-supervised learning across Tracers for whole-body PET/CT-based lesion segmentation
Aug 20, 2026Bashirul Azam Biswas, Amartya Bhattacharya, Biratal Raj Wagle +3Multimodal SSL PET/CT Tumor Segmentation
Department of Biomedical Data Science Geisel School of Medicine at Dartmouth Hanover, NH 03755, USA · Radiology Dartmouth Hitchcock Medical Center Lebanon, NH 03766, USA · Radiation Oncology Dartmouth Hitchcock Medical Center Lebanon, NH 03766, USA
OctoSense: Self-Supervised Learning for Multimodal Robot Perception
Jun 25, 2026Anthony Bisulco, Jeremy Wang, Kostas Daniilidis +2Autonomous Driving Datasets Masked Autoencoders
1General, Robotics, Automation, Sensing and Perception (GRASP) Laboratory, University of Pennsylvania · Computer Science Department, Brown University
Cross-Modal-Domain Generalization Through Semantically Aligned Discrete Representations
May 12, 2026Souptik Sen, Raneen Younis, Zahra AhmadiMultimodal Learning Representation Learning
Peter L. Reichertz Institute for Medical Informatics, Hannover Medical School, Germany · Lower Saxony Center for AI and Causal Methods in Medicine (CAIMed), Hannover, Germany
SSL-R1: Self-Supervised Visual Reinforcement Post-Training for Multimodal Large Language Models
Apr 22, 2026Jiahao Xie, Alessio Tonioni, Nathalie Rauschmayr +2Multimodal Large Language Models Reinforcement Learning with Verifiable Rewards
Max Planck Institute for Informatics, SIC · VIA Research Center · Google
Le MuMo JEPA: Multi-Modal Self-Supervised Representation Learning with Learnable Fusion Tokens
Mar 25, 2026Ciem Cornelissen, Sam Leroux, Pieter SimoensMultimodal SSL Multimodal Learning
IDLab, Department of Information Technology, Ghent University - imec, Belgium
Structure is Supervision: Multiview Masked Autoencoders for Radiology
Nov 27, 2025Sonia Laguna, Andrea Agostini, Alain Ryser +9Medical Image Representation Learning Masked Autoencoders
Department of Computer Science, ETH Zurich · Department of Computer Science, UC Irvine · Department of Quantitative Biomedicine, University of Zurich
PrismSSL: One Interface, Many Modalities; A Single-Interface Library for Multimodal Self-Supervised Learning
Nov 21, 2025Melika Shirian, Kianoosh Vadaei, Kian Majlessi +3Multimodal SSL Self-Supervised Learning
dept. Computer Engineering University of Isfahan Isfahan, Iran · dept. Computer Engineering University of Texas at Dallas Isfahan, Iran · dept. Artificial Intelligence University of Isfahan Isfahan, Iran
NeuroBridge: Bio-Inspired Self-Supervised EEG-to-Image Decoding via Cognitive Priors and Bidirectional Semantic Alignment
Nov 10, 2025Wenjiang Zhang, Sifeng Wang, Yuwei Su +3EEG-Based Visual Decoding Cross-Subject EEG Decoding
School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China · Queen Mary School Hainan, Beijing University of Posts and Telecommunications, Hainan 570100, China
FairSSL: Fair Multimodal Self-Supervised Learning
Aug 22, 2025Jiaee Cheong, Abtin Mogharabin, Paul Liang +2Cross-Modal Learning Representation Learning
Harvard University · University of Cambridge · Middle East Technical University +1

Nov 5, 2026J/K move · Enter open · S save

Multimodal Information Interactions Bottleneck for Multimodal Self-Supervised Learning

Liangjian Wen, Jiangtao Zheng, Jiang Duan, Qun Dai+7

School of Computing and Artificial Intelligence, Southwestern University of Finance and Economics, Chengdu, China · Artificial Intelligence and Digital Finance Key Laboratory of Sichuan Province · Chengdu Everimaging Science and Technology Co., Ltd.+8

Multimodal SSL Multimodal Learning Multimodal Fusion

Open paper AskPDF arXiv

Multimodal self-supervised learning is challenged by heterogeneity, unequal informativeness, and complex information interactions across multimodal data sources such as images, text, audio, and tabular data. While contrastive methods effectively align shared semantics, they typically rely on strong multi-view redundancy assumptions, overlooking modality-specific and synergistic information, thus leading to suboptimal representations. We propose the Multimodal Information Interactions Bottleneck (MIIB), a principled framework grounded in Partial Information Decomposition that explicitly encodes redundant, unique, and synergistic information within multimodal interactions. MIIB employs a dual-path model architecture for modality-specific encoding and multimodal fusion, optimized using surrogate tractable entropy-based objectives that enforce minimal sufficient representations without requiring explicit mutual information estimation. By leveraging variational bounds and score-based entropy estimation, MIIB efficiently captures complex cross-modal interactions while suppressing superfluous information. Experiments across diverse benchmarks demonstrate consistent improvements over state-of-the-art multimodal self-supervised methods. Code is available at https://github.com/wanglingzaishi/MIIB.