Multimodal Pretraining

Ask about this topic

Momentum

15 papers in the last four weeks, up 67% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 121

All topics
CardsList

  1. KAST-BAR: Knowledge-Anchored Semantically-Dynamic Topology Brain Autoregressive Modeling for Universal Neural Interpretation

    May 13, 2026Haoning Wang, Wenchao Yang, Shuai Shen +1EEG DecodingMultimodal Pretraining

    School of Automation Science and Electrical Engineering, Beihang University, Beijing, China. · School of Biological Science and Medical Engineering, Beihang University, Beijing, China. · State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China. +1

  2. BrainAnytime: Anatomy-Aware Cross-Modal Pretraining for Brain Image Analysis with Arbitrary Modality Availability

    May 13, 2026Guangqian Yang, Tong Ding, Wenlong Hou +4Missing-Modality LearningMasked Autoencoders

    Department of Biomedical Engineering, The Hong Kong Polytechnic University, Hong Kong SAR, China · Department of Technology Management for Innovation, The University of Tokyo, Japan · Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong SAR, China

  3. Phoenix-VL 1.5 Medium Technical Report

    May 11, 2026Team Phoenix, :, Arka Ray +30Multilingual Language ModelsDomain Adaptation

    HTX × Mistral AI ∗

  4. MultiMedVision: Multi-Modal Medical Vision Framework

    May 9, 2026Frank Li, Bardia Khosravi, Mohammadreza Chavoshi +5Medical Imaging Foundation ModelsVision Transformer

    Emory University · Yale University

  5. Anisotropic Modality Align

    May 8, 2026Xiaomin Yu, Yijiang Li, Yuhui Zhang +8Cross-Modal LearningCross-Modal Alignment

    HKUST(GZ) · NUS · Stanford +2

  6. Pan-FM: A Pan-Organ Foundation Model with Saliency-Guided Masking for Missing Robustness

    May 8, 2026Qiangqiang Wu, Grace McIlvain, Zhou Yu +1Missing-Modality LearningMedical Imaging

    Laboratory of AI and Biomedical Science (LABS) · 2Columbia University

  7. TRAJGANR: Trajectory-Centric Urban Multimodal Learning via Geospatially Aligned Neural Representations

    May 7, 2026Maria Despoina Siampou, Gengchen Mai, Ni Lao +4Geospatial Foundation ModelsCross-Modal Alignment

    Google Research, Mountain View, CA · Dept. of Computer Science, University of Southern California, Los Angeles, CA · SEAI Lab, Dept. of Geography and the Environment, The University of Texas at Austin, Austin, TX +1

  8. Knowledge Transfer Scaling Laws for 3D Medical Imaging

    May 7, 2026Ho Hin Lee, Dongna Du, Chu Wang +4Medical Imaging Foundation Models3D Medical Imaging

    Vanderbilt University · Zhejiang University · McGill University +1

  9. Audio-Visual Intelligence in Large Foundation Models

    May 5, 2026You Qin, Kai Liu, Shengqiong Wu +12Audio-Visual UnderstandingAudio-Video Generation

    National University of Singapore · University of Oxford · University of Toronto +5

  10. Valley3: Scaling Omni Foundation Models for E-commerce

    May 2, 2026Zeyu Chen, Guanghao Zhou, Min Yang +6Unified Multimodal ModelsMultimodal Pretraining

    Valley Team, ByteDance Group

  11. GaMMA: Towards Joint Global-Temporal Music Understanding in Large Multimodal Models

    May 1, 2026Zuyao You, Zhesong Yu, Mingyu Liu +3Multimodal PretrainingMultimodal Large Language Models

    1Fudan University · 2ByteDance

  12. A Multimodal Pre-trained Network for Integrated EEG-Video Seizure Detection

    Apr 29, 2026Tong Lu, Ke Xu, Zimo Zhang +12ElectroencephalographyEEG Seizure Detection

    Beijing Institute for Brain Research, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, 102206, China · Chinese Institute for Brain Research, Beijing, Beijing, 102206, China · Department of Neurosurgery, SanBo Brain Hospital, Capital Medical University, Beijing, 100018, China +4

  13. Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation

    Apr 27, 2026Zhiheng Liu, Weiming Ren, Xiaoke Huang +12Unified Multimodal ModelsMultimodal Pretraining

    Meta AI · University of Waterloo · The University of Hong Kong

  14. OceanPile: A Large-Scale Multimodal Ocean Corpus for Foundation Models

    Apr 25, 2026Yida Xue, Ningyu Zhang, Tingwei Wu +5AI for ScienceMultimodal Pretraining

    College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China. · School of Software Technology, Zhejiang University, Ningbo 315048, China. · State Key Laboratory of Ocean Sensing, Hangzhou 311200, China. +2

  15. CheXmix: Unified Generative Pretraining for Vision Language Models in Medical Imaging

    Apr 24, 2026Ashwin Kumar, Robbie Holland, Corey Barrett +8Radiology Report GenerationChest X-Ray Classification

    Stanford AIMI, Stanford University · Oracle Health AI · Department of Radiology, Stanford University

  16. Context Unrolling in Omni Models

    Apr 23, 2026Ceyuan Yang, Zhijie Lin, Yang Zhao +16Cross-Modal LearningUnified Multimodal Models

  17. Multimodal Transformer for Sample-Aware Prediction of Metal-Organic Framework Properties

    Apr 21, 2026Seunghee Han, Jaewoong Lee, Jihan KimMultimodal PretrainingMaterials Property Prediction

    Department of Chemical and Biomolecular Engineering, Korea Advanced Institute of Science and Technology, Daejeon 34141, Republic of Korea. · Department of Materials Science and Engineering, Korea Advanced Institute of Science and Technology, Daejeon 34141, Republic of Korea.

  18. Better with Less: Tackling Heterogeneous Multi-Modal Image Joint Pretraining via Conditioned and Degraded Masked Autoencoder

    Apr 18, 2026Bowen Peng, Yongxiang Liu, Jie Zhou +4Cross-Modal LearningMasked Autoencoders

    College of Electronic Science and Technology, National University of Defense Technology (NUDT), Changsha 410073, China · Beijing Institute of Remote Sensing Information, Beijing 100192, China

  19. PivotMerge: Bridging Heterogeneous Multimodal Pre-training via Post-Alignment Model Merging

    Apr 18, 2026Zibo Shao, Baochen Xiong, Xiaoshan Yang +4Cross-Modal LearningCross-Modal Alignment

    State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · Pengcheng Laboratory, Shenzhen 518066, China · School of Artificial Intelligence, University of Chinese Academy Sciences, Beijing 100049, China +1

  20. MedP-CLIP: Medical CLIP with Region-Aware Prompt Integration

    Apr 13, 2026Jiahui Peng, He Yao, Jingwen Li +9Promptable Image SegmentationMultimodal Pretraining

    aSichuan University, Chengdu, 610065, China · bXinjiang University, Urumqi, 830046, China · cAlibaba Group, Hangzhou, 310023, China +3

  21. Symbiotic-MoE: Unlocking the Synergy between Generation and Understanding

    Apr 9, 2026Xiangyue Liu, Zijian Zhang, Miles Yang +3Cross-Modal Representation LearningUnified Multimodal Models

    HKUST · Tencent Hunyuan

  22. Overcoming the Modality Gap in Context-Aided Forecasting

    Mar 12, 2026Vincent Zhihao Zheng, Étienne Marcotte, Arjun Ashok +4Synthetic Data AugmentationMultimodal Pretraining

    ServiceNow AI Research, Montréal, Canada · Department of Civil Engineering, McGill University, Montréal, Canada · DIRO, Université de Montréal, Montréal, Canada +1

  23. GreenRFM: Learning a resource-efficient radiology vision-language foundation model via supervision-centric pre-training

    Mar 6, 2026Yingtai Li, Shuai Ming, Qiuli Wang +13Medical Imaging Foundation ModelsRadiology VLMs

    School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China (USTC), Hefei, Anhui, 230026, China · Center for Medical Imaging, Robotics, Analytic Computing & Learning (MIRACLE), Suzhou Institute for Advanced Research, USTC, Suzhou, Jiangsu, 215123, China · Department of Radiology, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, USTC, Hefei, Anhui, 230001, China +7

  24. MPFlow: Multi-modal Posterior-Guided Flow Matching for Zero-Shot MRI Reconstruction

    Mar 4, 2026Seunghoi Kim, Chen Jin, Henry F. J. Tregidgo +2Cross-Modal AlignmentMultimodal Pretraining

    University College London, UK · Centre for AI, DS&AI, AstraZeneca, UK

  25. MedGPT-oss: Training a General-Purpose Vision-Language Model for Biomedicine

    Mar 1, 2026Kai Zhang, Zhengqing Yuan, Cheng Peng +10HealthcareMultimodal Pretraining

    Department of Computer Science and Engineering, Lehigh University · Department of Computer Science and Engineering, University of Notre Dame · Department of Health Outcomes & Biomedical Informatics, University of Florida +3

  26. SleepLM: Natural-Language Intelligence for Human Sleep

    Feb 27, 2026Zongzhe Xu, Zitao Shuai, Eideen Mozaffari +3Multimodal PretrainingCross-Modal Retrieval

    University of California, Los Angeles

  27. PRIMA: Pre-training with Risk-integrated Image-Metadata Alignment for Medical Diagnosis via LLM

    Feb 26, 2026Yiqing Wang, Chunming He, Ming-Chen Lu +4Cross-Modal AlignmentMultimodal Pretraining

    Department of Biomedical Engineering, Duke University, Durham, NC, USA · Department of Ophthalmology and Visual Sciences, University of Michigan, Ann Arbor, MI, USA

  28. Xray-Visual Models: Scaling Vision models on Industry Scale Data

    Feb 18, 2026Shlok Mishra, Tsung-Yu Lin, Linda Wang +24Vision Foundation ModelsMultimodal Pretraining

    Meta-AI · MIT (work done while at Meta AI) · University of Maryland (work done while at Meta AI)

  29. PrismSSL: One Interface, Many Modalities; A Single-Interface Library for Multimodal Self-Supervised Learning

    Nov 21, 2025Melika Shirian, Kianoosh Vadaei, Kian Majlessi +3Self-Supervised LearningMultimodal Learning

    dept. Computer Engineering University of Isfahan Isfahan, Iran · dept. Computer Engineering University of Texas at Dallas Isfahan, Iran · dept. Artificial Intelligence University of Isfahan Isfahan, Iran

  30. Theoretical Refinement of CLIP by Utilizing Linear Structure of Optimal Similarity

    Oct 17, 2025Naoki Yoshida, Satoshi Hayakawa, Yuhta Takida +3Cross-Modal LearningKernel Mean Embeddings

    The University of Tokyo · Sony Group Corporation · Sony AI