Masked Image Modeling

Also known as MIM

Masked image modeling (MIM) is a self-supervised approach that hides patches or regions and trains a model to recover missing pixels, features, or semantic targets, reducing reliance on labeled images. Methods range from masked autoencoders and latent prediction to teacher-guided distillation, with random, saliency-guided, structure-aware, and domain-specific masking. Pretraining is used for transfer to classification, segmentation, detection, and scientific or medical imaging, often tested on benchmarks such as ImageNet, Cityscapes, and domain-specific datasets with limited labels. A central open tension is that masks and reconstruction objectives that improve pretraining loss do not always improve downstream performance, so research continues to test which information should be hidden and predicted.

Latest papers

42 papers7 papers in the last four weeks, up 133% on the four weeks before. 0.0% of all new papers.

Co-distilled attention guided masked image modeling with noisy teacher for self-supervised learning on medical images
Apr 16, 2026Jue Jiang, Aneesh Rangnekar, Harini VeeraraghavanMasked Autoencoders Self-Supervised Pre-Training
Memorial Sloan Kettering Cancer Center
TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignment
Apr 13, 2026Bingyi Cao, Koert Chen, Kevis-Kokitsi Maninis +16Vision-Language Pretraining Vision-Language Alignment
Google DeepMind · xAI · Epsilon Health +1
Suppressing Non-Semantic Noise in Masked Image Modeling Representations
Mar 31, 2026Martine Hjelkrem-Tan, Marius Aasan, Rwiddhi Chakraborty +3Self-Supervised Visual Representation Learning Masked Image Modeling
University of Oslo · UiT The Arctic University of Norway
MEDiC: Multi-objective Exploration of Distillation from CLIP
Mar 30, 2026Konstantinos Georgiou, Maofeng Tang, Hairong QiMasked Image Modeling VLM Distillation
Min H. Kao Department of Electrical Engineering and Computer Science The University of Tennessee Knoxville, TN 37996
SpecTM: Spectral Targeted Masking for Trustworthy Foundation Models
Mar 23, 2026Syed Usama Imtiaz, Mitra Nasr Azadani, Nasrin AlamdariMasked Image Modeling Geospatial Foundation Models
Department of Civil and Environmental Engineering Florida State University, Tallahassee, FL, USA
From Semantics to Pixels: Coarse-to-Fine Masked Autoencoders for Hierarchical Visual Understanding
Mar 10, 2026Wenzhao Xiang, Yue Wu, Hongyang Yu +3Self-Supervised Visual Representation Learning Masked Autoencoders
Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China · University of Chinese Academy of Sciences, Beijing 100049, China · Pengcheng Laboratory, Shenzhen 518108, China +1
SIGMAE: A Spectral-Index-Guided Foundation Model for Multispectral Remote Sensing
Mar 8, 2026Xiaokang Zhang, Bo Li, Chufeng Zhou +2Remote Sensing Image Understanding Remote Sensing
School of Artificial Intelligence, Wuhan University, Wuhan, 430072, China · School of Artificial Intelligence and Automation, Wuhan University of Science and Technology, Wuhan, 430081, China · School of Electronic Information, Wuhan University of Science and Technology, Wuhan, 430081, China +2
NeighborMAE: Exploiting Spatial Dependencies between Neighboring Earth Observation Images in Masked Autoencoders Pretraining
Mar 3, 2026Liang Zeng, Valerio Marsocci, Wufan Zhao +2Self-Supervised Pre-Training Masked Autoencoders
KU Leuven · ESA Φ-lab · HKUST(GZ) +1
Accelerating Masked Image Generation by Learning Controlled Latent Dynamics
Feb 27, 2026Kaiwen Zhu, Quansheng Zeng, Yuandong Pu +9Latent Dynamics Modeling T2I Generation
Shanghai Jiao Tong University Shanghai, China · Shanghai AI Laboratory Shanghai, China · Shanghai Innovation Institute Shanghai, China +6
Non-Contact Physiological Monitoring in Pediatric Intensive Care Units via Adaptive Masking and Self-Supervised Learning
Feb 17, 2026Mohamed Khalil Ben Salah, Philippe Jouvet, Rita NoumeirRemote Photoplethysmography Heart Rate Estimation
Biomedical Information Processing Laboratory, École de Technologie Supérieure, University of Quebec, Montreal, QC, Canada · Pediatric Intensive Care Unit, CHU Sainte-Justine, Montreal, QC, Canada
Vascular anatomy-aware self-supervised pre-training for X-ray angiogram analysis
Feb 12, 2026De-Xing Huang, Chaohui Yu, Xiao-Hu Zhou +8Medical Image Representation Learning Medical Imaging
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · DAMO Academy, Alibaba Group, Hangzhou 310023, China +2
FastMap: Real-Time Semantic Map Completion via Bitwise Masked Modeling
Jun 9, 2025Yijie Deng, Shuaihang Yuan, Congcong Wen +2Semantic Mapping Masked Image Modeling
NYUAD Center for Artificial Intelligence and Robotics, Abu Dhabi, UAE. · New York University Abu Dhabi, Electrical Engineering, Abu Dhabi 129188, UAE. · New York University, Electrical & Computer Engineering Dept., Brooklyn, NY 11201, USA. +1

Apr 16, 2026J/K move · Enter open · S save

Co-distilled attention guided masked image modeling with noisy teacher for self-supervised learning on medical images

Jue Jiang, Aneesh Rangnekar, Harini Veeraraghavan

Memorial Sloan Kettering Cancer Center

Masked Autoencoders Self-Supervised Pre-Training Medical Image Representation Learning Masked Image Modeling

Open paper AskPDF arXiv

Masked image modeling (MIM) is a highly effective self-supervised learning (SSL) approach to extract useful feature representations from unannotated data. Predominantly used random masking methods make SSL less effective for medical images due to the contextual similarity of neighboring patches, leading to information leakage and SSL simplification. Hierarchical shifted window (Swin) transformer, a highly effective approach for medical images cannot use advanced masking methods as it lacks a global [CLS] token. Hence, we introduced an attention guided masking mechanism for Swin within a co-distillation learning framework to selectively mask semantically co-occurring and discriminative patches, to reduce information leakage and increase the difficulty of SSL pretraining. However, attention guided masking inevitably reduces the diversity of attention heads, which negatively impacts downstream task performance. To address this, we for the first time, integrate a noisy teacher into the co-distillation framework (termed DAGMaN) that performs attentive masking while preserving high attention head diversity. We demonstrate the capability of DAGMaN on multiple tasks including full- and few-shot lung nodule classification, immunotherapy outcome prediction, tumor segmentation, and unsupervised organs clustering.