Cross-Modal Representation Learning

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 136

All topics

SAREO-FM: Decoupled Semantic Supervision for SAR-EO Foundation Models
Oct 7, 2026Jeonghyeok Do, Munchurl KimRemote Sensing Image Understanding Multimodal Pretraining
Korea Advanced Institute of Science and Technology (KAIST)
UniAfford: Token-Routed Multitask Learning for Generalizable 2D-3D Affordance Perception
Sep 29, 2026Yuhao Liu, Yiming Zhong, Hanqing Wang +9Cross-Modal Representation Learning Unified Multimodal Models
ShanghaiTech University · Shandong University · HKUST(GZ) +3
Do Emotion Concepts Generalize Across Sources, Modalities, and Architectures in Vision-Language Models?
Sep 28, 2026Bohao Xing, Xin Liu, Kaishen Yuan +5Cross-Modal Learning Vision-Language Models
Lappeenranta-Lahti University of Technology LUT · Shanghai Jiao Tong University · The Hong Kong University of Science and Technology (Guangzhou) +3
TRACE: Expert-Aligned ECG Representation Learning with Rigorous Benchmarking and Real-World Validation in Acute Cardiac Care
Sep 28, 2026Lovely Yeswanth Panchumarthi, Andrew Lu, Saurabh Kataria +11Representation Learning Cross-Modal Representation Learning
Department of Computer Science, Emory University, GA, USA · School of Nursing, Emory University, GA, USA · Institute for Medical Engineering and Science, Massachusetts Institute of Technology, MA, USA +3
PhyMo: A Physical-Field Modality for Multimodal AI4Physics
Sep 23, 2026Henan Sun, Haitao Hu, Jin Liu +4AI for Science Cross-Modal Representation Learning
The Hong Kong University of Science and Technology (Guangzhou) · Huawei Noah’s Ark Lab · Tencent HY +1
RPA: Residual Patch-Token Adapter for Image Retrieval from EEG and MEG
Sep 19, 2026Yuhui Jin, Yonghao Song, Bingchuan LiuCross-Subject EEG Decoding Cross-Modal Representation Learning
Vortek Lab Inc. · Department of Computer Science and Technology, Tsinghua University · Biology and Biological Engineering, California Institute of Technology
Hub-Spectral Activation of Latent Multimodal Knowledge
Sep 15, 2026Ying Guo, Haidong Chen, Linrui Xu +7Cross-Modal Representation Learning Cross-Modal Alignment
Beijing Key Laboratory of Key Technologies for AI+ Domain Applications, North China University of Technology, Beijing 100144, China · School of Geosciences and Info-Physics, Central South University, Changsha 410083, China · NExT++ Research Centre, National University of Singapore, Singapore +2
Hyper-RED: Scalable Event Pre-training via Semantic Hypergraph Distillation
Sep 15, 2026Meisen Wang, Zhiqiang Tian, Wei Bao +3Event-Based Vision Cross-Modal Knowledge Distillation
School of Software Engineering, Xi’an Jiaotong University, Xi’an 710049, China · National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an 710049, China · BNRist, THUIBCS, BLBCI, School of Software, Tsinghua University, Beijing 100084, China +2
FLAT: Resampling Image and Text into 1D Flexible-Length Aligned Transmodal Tokens for Retrieval and Generation
Sep 15, 2026Guangyu Sun, Shlok Kumar Mishra, Wentao Bao +8Cross-Modal Representation Learning Cross-Modal Alignment
Meta AI
PACE: Progressive Angular-to-Norm Contrastive Embedding
Sep 14, 2026Yanping Li, Wei Zhou, Yawen Liu +7Cross-Modal Representation Learning Multimodal Embedding
The Hong Kong University of Science and Technology · Alibaba Group · Huazhong University of Science and Technology +1
FRIST: FMRI Representation Informed Shared-space Training Improves EEG-only Individual-Finger BCI Decoding
Sep 14, 2026Jintao Zhang, Yidan Ding, Joshua Kosnoff +3MI Classification Cross-Modal Representation Learning
Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA. · Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA. · Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA.
Multimodal Floorplan Encoding: Learning Dense Modality-Invariant Representations
Sep 14, 2026Xavier Anadón, Rémi Pautrat, Rui WangRepresentation Learning Cross-Modal Representation Learning
University of Zaragoza, Spain · Microsoft, Switzerland
MMGait: Benchmarking and Unifying Gait Recognition across Heterogeneous Modalities
Sep 11, 2026Saihui Hou, Chenye Wang, Qingyuan Cai +2Cross-Modal Representation Learning Gait Recognition
School of Artificial Intelligence, Beijing Normal University, Beijing, China · The Hong Kong University of Science and Technology, Hong Kong, China · JD.com, Beijing, China +1
Leveraging Cardiac Imaging to Improve ECG-Based Detection of Chagas Disease in Resource-Constrained Settings
Sep 8, 2026Laura Alvarez-Florez, Daniel Uyterlinde, Samuel Ruipérez-Campillo +3Cross-Modal Representation Learning Medical Diagnosis
Department of Biomedical Engineering and Physics, Amsterdam University Medical Center, The Netherlands · Quantitative Healthcare Analysis Group, University of Amsterdam, The2026 Netherlands · Department of Clinical and Experimental Cardiology, Amsterdam University Medical Center, The Netherlands +2
Proprioception-Anchored Cross-Modal Pretraining for Zero-Shot Sim-to-Real Contact-Rich Assembly
Sep 7, 2026Yuhan Wang, Yurou Chen, Hongye Jiang +2RL for Robotics Cross-Modal Representation Learning
Shanghai Jiao Tong University · University of Sussex
CrossFeat: Bridging Imaging Modalities in Feature Descriptor Space
Aug 31, 2026Paul Schneider, Nazim HaouchineCross-Modal Representation Learning Multimodal Disentangled Representation Learning
Harvard Medical School, Brigham and Women’s Hospital, Boston, MA, USA · Technical University of Munich, Munich, Germany
Paths: Prompt-aware Spatio-temporal Transformer with Hierarchical Multi-modal Fusion for RGB-Event Video Person Re-Identification
Aug 13, 2026Yakun Huo, Yingquan Wang, Yangyang Liu +4Cross-Modal Representation Learning Transformer
Dalian University of Technology Dalian, China
CardioState-JEPA: Delay-Aware Cross-Modal Learning of a Shared Cardiac Representation
Aug 13, 2026Hamza Shafiq, Hung Manh Pham, Bin Zhu +3Cross-Modal Representation Learning Joint-Embedding Predictive Architecture
Eindhoven University of Technology, Netherlands · Singapore Management University, Singapore
Unlocking the Power of Medical Tabular Data via Semantic-Aware Multimodal Pre-training
Aug 11, 2026Yingsheng Liu, Haiming Li, Jingmin Zhu +6Cross-Modal Representation Learning Multimodal Pretraining
Faculty of Information Technology, Monash University, Melbourne, Australia · Monash University, Melbourne, Victoria, Australia · The University of Queensland, Brisbane, Queensland, Australia
TeaMatch: Teachable Cross-Modal Representation Learning for 2D-3D Matching
Aug 10, 2026Chongjian Wang, Junjie GaoCross-Modal Representation Learning Image Matching
Shandong Women’s University · Jinan, China · Shandong University of Science and Technology +1
Hyperbolic Multimodal Continual Learning
Aug 10, 2026Jiahong Liu, Ming Shen, Xiaohao Liu +4Cross-Modal Representation Learning Invariant Representation Learning
Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China · School of Computing, National University of Singapore, Singapore · Department of Computer Science, Yale University, New Haven, CT, United States +1
GeoUniPR: A Geometry-Consistent Unified Framework for Cross-Modal Place Recognition
Aug 10, 2026Wonbong Kim, Jiatong Xiao, Rui Li +5Cross-Modal Representation Learning LiDAR Place Recognition
School of Computer Science and Technology, Tongji University, Shanghai, China · Shanghai Research Institute for Intelligent Autonomous System, Tongji University, Shanghai, China · MOE Key Lab of Embedded System and Service Computing, Tongji University, Shanghai, China +1
FSTC-Encoder: Feature--Spatial--Temporal Correlation Learning for Generalizable RF Sensing
Aug 9, 2026Jing Wang, Zhu Wang, Changlong Cheng +2Cross-Modal Representation Learning Multimodal Robustness
Northwestern Polytechnical University
Learning Deep Modality-Shared Self-Expressiveness for Image Clustering with Textual Information
Aug 9, 2026Xianghan Meng, Wei He, Zhiyuan Huang +1Cross-Modal Representation Learning Clustering
School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing 100876, P.R. China
Consistency-Driven Co-Evolution for Self-Supervised Cross-Representation Learning
Aug 5, 2026Xuehang Guo, Pengyuan Li, Tom Hope +3Multi-View Consistency Representation Learning
William & Mary · IBM · Allen Institute for AI +2
Decoupling Perception from Description: Computation-Grounded Representation Alignment between Multivariate Time Series and Language
Aug 5, 2026Xinran Feng, Yi Xie, Chao Zhang +4Cross-Modal Representation Learning Multimodal Pretraining
Thrust of Financial Technology, The Hong Kong University of Science and Technology (Guangzhou) · Technical University of Munich · Heilbronn Data Science Center, Munich Data Science Institute +1
CRIL-U-Net: Compact Ratio-Interaction Learning for Focal Cortical Dysplasia Segmentation from T1w and FLAIR MRI
Aug 4, 2026Soumen Ghosh, Amit Soni Arya, Tilottama Goswami +3Cross-Modality Medical Image Segmentation Cross-Modal Representation Learning
Syntax Meets Semantics: Understanding Scientific Formulae
Aug 3, 2026Yuni Susanti, Moritz SchubotzSemantic Correspondence Cross-Modal Representation Learning
FIZ Karlsruhe Berlin, Germany
Learning Molecular Representations from Cellular Phenotypes with Structure Preservation
Aug 3, 2026Xuan Lin, Jingyu Sheng, Tengfei Ma +2Disentangled Representation Learning Cross-Modal Representation Learning
School of Computer Science, Xiangtan University, Xiangtan, Hunan, China · College of Computer Science and Electronic Engineering, Hunan University, Changsha, Hunan, China · School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, China +1
Generic Vision and Cross-Attention for Reaction Yield Prediction
Aug 1, 2026Qiwei Han, Chi ZhouCross-Modal Representation Learning Cross-Modal Attention
Department of Chemistry, Duke University, Durham, NC 27708, USA · Department of Computer Science, Georgia Institute of Technology, Atlanta, GA 30332, USA

Oct 7, 2026, cs.CVJ/K move · Enter open · S save

SAREO-FM: Decoupled Semantic Supervision for SAR-EO Foundation Models

Jeonghyeok Do, Munchurl Kim

Korea Advanced Institute of Science and Technology (KAIST)

Remote Sensing Image Understanding Multimodal Pretraining Multimodal Foundation Models Cross-Modal Representation Learning Masked Image Modeling

Open paper AskPDF arXiv

Synthetic aperture radar (SAR) and electro-optical (EO) imagery provide complementary observations: SAR enables day-and-night, weather-resilient sensing, whereas EO provides rich appearance and fine-grained semantic cues. We introduce SAREO-FM, which avoids forcing a single token stream to serve two distinct roles: modality tokens preserve how each sensor observes the scene through masked reconstruction, while learnable semantic queries capture what the scene contains under guidance from a pretrained vision foundation model (VFM). By jointly encoding these queries with SAR and EO tokens, the queries acquire modality-grounded semantic context, while the modality-token outputs remain the explicit targets of masked reconstruction. This design assigns semantic and reconstruction supervision to separate token streams while preserving their interaction within the shared encoder. Pretrained on the million-scale SAR-1M corpus, SAREO-FM achieves strong unimodal transfer for both SAR-only and EO-only inputs, while delivering substantial gains from joint SAR--EO observations on tasks that benefit from complementary sensing.

Momentum

Related topics

Latest papers 136