Self-Supervised Pre-Training

Follow this topic Ask about this topic

Momentum

18 papers in the last four weeks, up 64% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 155

All topics

RadJEPA: Radiology Encoder for Chest X-Rays via Joint Embedding Predictive Architecture
Jan 22, 2026Anas Anwarul Haq Khan, Mariam Husain, Pratik Jalan +1Radiology Report Generation Medical Imaging
Department of Computer Science (CSE), Indian Institute of Technology Bombay, India · Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Department of Biomedical Engineering, Johns Hopkins University, USA +1
Stochastic Siamese MAE Pretraining for Longitudinal Medical Images
Dec 29, 2025Taha Emre, Arunava Chakravarty, Thomas Pinetz +9Masked Autoencoders Self-Supervised Pre-Training
Institute of Artificial Intelligence, Center for Medical Data Science, Medical University of Vienna, Austria · Department of Ophthalmology and Optometry, Medical University of Vienna, Austria · BioMedIA, Department of Computing, Imperial College London, London, United Kingdom +8
Radiance-Field Guided Pretraining: Scaling Localization Models with Unlabeled Wireless Signals
Dec 8, 2025Guosheng Wang, Shen Wang, Lei YangSelf-Supervised Pre-Training Wireless Localization
Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China
Pianist Transformer: Towards Expressive Piano Performance Rendering via Scalable Self-Supervised Pre-Training
Dec 2, 2025Hong-Jie You, Jie-Jing Shao, Xiao-Wen Yang +3Efficient Transformer Inference Self-Supervised Pre-Training
State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing 210023, China · School of Artificial Intelligence, Nanjing University, Nanjing 210023, China · School of Intelligence Science and Technology, Nanjing University, Suzhou 215163, China
Pre-train to Gain: Robust Learning Without Clean Labels
Nov 25, 2025David Szczecina, Nicholas Pellegrino, Paul FieguthSelf-Supervised Pre-Training Learning with Noisy Labels
Systems Design Engineering University of Waterloo Waterloo, Ontario, Canada

Jan 22, 2026, cs.CVJ/K move · Enter open · S save

RadJEPA: Radiology Encoder for Chest X-Rays via Joint Embedding Predictive Architecture

Anas Anwarul Haq Khan, Mariam Husain, Pratik Jalan, Kshitij Jadhav

Department of Computer Science (CSE), Indian Institute of Technology Bombay, India · Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Department of Biomedical Engineering, Johns Hopkins University, USA+1

Radiology Report Generation Medical Imaging Self-Supervised Pre-Training Joint-Embedding Predictive Architecture Medical Image Representation Learning

Open paper AskPDF arXiv

Vision-language pretraining has driven progress in medical image representation learning, but it depends on paired image-text data and can inherit reporting bias from clinical narratives. We study whether language-free predictive pretraining can produce an image encoder that transfers effectively to radiology report generation. RadJEPA is a chest-X-ray adaptation of I-JEPA, pretrained on approximately 840K unlabeled radiographs using latent context-to-target prediction. Our primary contribution is an extensive empirical evaluation of this language-free encoder for report generation: the frozen image encoder is coupled to a trainable two-layer projector and language decoder, and is also substituted into four established vision-language backbones. Across MIMIC-CXR and IU-Xray, RadJEPA matches or exceeds the evaluated image-only and image-text baselines on lexical, entity-relation, and clinical-label metrics. Controlled MIMIC-only comparisons provide evidence that the predictive objective contributes beyond domain-specific pretraining, while broader comparisons also reflect differences in pretraining data, model capacity, and input resolution. Complementary classification and segmentation experiments assess transfer beyond report generation.

Momentum

Related topics

Latest papers 155

RadJEPA: Radiology Encoder for Chest X-Rays via Joint Embedding Predictive Architecture

Stochastic Siamese MAE Pretraining for Longitudinal Medical Images

Radiance-Field Guided Pretraining: Scaling Localization Models with Unlabeled Wireless Signals

Pianist Transformer: Towards Expressive Piano Performance Rendering via Scalable Self-Supervised Pre-Training

Pre-train to Gain: Robust Learning Without Clean Labels