Masked Autoencoders

Follow this topic Ask about this topic

Masked autoencoders study how models can learn useful representations from unlabeled data by predicting hidden patches, tokens, or time segments. Common approaches encode only the visible input and use a decoder for reconstruction, while newer variants guide masking with domain structure, predict latent features, or combine reconstruction with contrastive and cross-modal learning. These methods are used across images, video, medical scans, audio, biosignals, and remote sensing, with evaluations spanning tasks such as ImageNet classification, NTU action recognition, and medical image segmentation. A central open question is when reconstruction pretraining transfers better than alternatives: results depend on the masking strategy, data scale, and downstream task, and faithful reconstruction does not always produce the most useful features.

Latest papers

103 papers12 papers in the last four weeks, down 8% on the four weeks before. 0.1% of all new papers.

Where Do Flow Semantics Reside? A Protocol-Native Tabular Pretraining Paradigm for Encrypted Traffic Classification
Mar 9, 2026Sizhe Huang, Zitong Li, Shujie YangEncrypted Traffic Classification Encrypted Network Traffic Analysis
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China.
SIGMAE: A Spectral-Index-Guided Foundation Model for Multispectral Remote Sensing
Mar 8, 2026Xiaokang Zhang, Bo Li, Chufeng Zhou +2Remote Sensing Image Understanding Remote Sensing
School of Artificial Intelligence, Wuhan University, Wuhan, 430072, China · School of Artificial Intelligence and Automation, Wuhan University of Science and Technology, Wuhan, 430081, China · School of Electronic Information, Wuhan University of Science and Technology, Wuhan, 430081, China +2
ICHOR: A Robust Representation Learning Approach for ASL CBF Maps with Self-Supervised Masked Autoencoders
Mar 5, 2026Xavier Beltran-Urbano, Yiran Li, Xinglin Zeng +10Medical Image Representation Learning Self-Supervised Pre-Training
University of Pennsylvania, Philadelphia, USA · University of Maryland School of Medicine, Baltimore, USA
Clinical-Injection Transformer with Domain-Adapted MAE for Lupus Nephritis Prognosis Prediction
Mar 4, 2026Yuewen Huang, Zhitao Ye, Guangnan Feng +3Clinical Outcome Prediction Computational Pathology
Sun Yat-sen University · Department of Nephrology, Guangzhou Women and Children’s Medical Center, Guangzhou Medical University, Guangzhou, China
NeighborMAE: Exploiting Spatial Dependencies between Neighboring Earth Observation Images in Masked Autoencoders Pretraining
Mar 3, 2026Liang Zeng, Valerio Marsocci, Wufan Zhao +2Self-Supervised Pre-Training Masked Autoencoders
KU Leuven · ESA Φ-lab · HKUST(GZ) +1
AFFMAE: Scalable Vision Pre-Training for High-Resolution Microscopy Segmentation on Desktop Hardware
Feb 18, 2026David Smerkous, Zian Wang, Behzad NajafianMasked Autoencoders Self-Supervised Pre-Training
Department of Laboratory Medicine & Pathology, University of Washington, Seattle WA, USA · Electrical Engineering & Computer Science, Oregon State University, Corvallis, OR, USA · Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle WA, USA
Benchmarking Self-Supervised Models for Cardiac Ultrasound View Classification
Feb 17, 2026Youssef Megahed, Salma I. Megahed, Robin Ducharme +4Self-Supervised Visual Representation Learning Echocardiography
Department of Systems and Computer Engineering, Carleton University, Ottawa, Ontario, Canada · Department of Methodological and Implementation Research, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada · Department of Acute Care Research, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada +4
Learning Representations from Incomplete EHR Data with Dual-Masked Autoencoding
Feb 16, 2026Xiao Xiang, David Restrepo, Hyewon Jeong +2Learning with Missing Data Masked Autoencoders
Massachusetts Institute of Technology · École Polytechnique Fédérale de Lausanne · Harvard University +1
OmniSpectra: A Unified Foundation Model for Universal Spectra Representation Learning
Jan 21, 2026Md Khairul Islam, Zeyu Xia, Adam S. Bolton +1Representation Learning Masked Autoencoders
Computer Science Dept, University of Virginia · School of Data Science, University of Virginia
Stochastic Siamese MAE Pretraining for Longitudinal Medical Images
Dec 29, 2025Taha Emre, Arunava Chakravarty, Thomas Pinetz +9Masked Autoencoders Self-Supervised Pre-Training
Institute of Artificial Intelligence, Center for Medical Data Science, Medical University of Vienna, Austria · Department of Ophthalmology and Optometry, Medical University of Vienna, Austria · BioMedIA, Department of Computing, Imperial College London, London, United Kingdom +8
Structure is Supervision: Multiview Masked Autoencoders for Radiology
Nov 27, 2025Sonia Laguna, Andrea Agostini, Alain Ryser +9Medical Image Representation Learning Masked Autoencoders
Department of Computer Science, ETH Zurich · Department of Computer Science, UC Irvine · Department of Quantitative Biomedicine, University of Zurich
Scaling Vision Transformers for Functional MRI with Flat Maps
Oct 15, 2025Connor Lane, Mihir Tripathy, Leema Krishna Murali +15fMRI Foundation Models Masked Autoencoders
MedARC · Sophont · Baylor College of Medicine +3
Occlusion-Aware 3D Hand-Object Pose Estimation with Masked AutoEncoders
Jun 12, 2025Hui Yang, Wei Sun, Jian Liu +3Masked Autoencoders 3D Hand Pose Estimation
National Engineering Research Center for Robot Visual Perception and Control Technology, School of Artificial Intelligence and Robotics, Hunan University, Changsha, 410082, China · School of Architecture and Art, Central South University, Changsha, 410082, China · Department of Computer Science and Software Engineering, The University of Western Australia, WA 6009, Australia

Mar 9, 2026J/K move · Enter open · S save

Where Do Flow Semantics Reside? A Protocol-Native Tabular Pretraining Paradigm for Encrypted Traffic Classification

Sizhe Huang, Zitong Li, Shujie Yang

State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China.

Encrypted Traffic Classification Encrypted Network Traffic Analysis Self-Supervised Pre-Training Masked Autoencoders Tabular ML

Open paper AskPDF arXiv

Self-supervised masked modeling shows promise for encrypted traffic classification by masking and reconstructing raw bytes. Yet recent work reveals these methods fail to reduce reliance on labeled data despite costly pretraining: under frozen encoder evaluation, accuracy drops from greater than 0.9 to less than 0.47. We argue the root cause is inductive bias mismatch: flattening traffic into byte sequences destroys protocol-defined semantics. We identify three specific issues: 1) field unpredictability, random fields like ip.id are unlearnable yet treated as reconstruction targets; 2) embedding confusion, semantically distinct fields collapse into a unified embedding space; 3) metadata loss, capture-time metadata essential for temporal analysis is discarded. To address this, we propose a protocol-native paradigm that treats protocol-defined field semantics as architectural priors, reformulating the task to align with the data's intrinsic tabular modality rather than incrementally adapting sequence-based architectures. Instantiating this paradigm, we introduce FlowSem-MAE, a tabular masked autoencoder built on Flow Semantic Units (FSUs). It features predictability-guided filtering that focuses on learnable FSUs, FSU-specific embeddings to preserve field boundaries, and dual-axis attention to capture intra-packet and temporal patterns. FlowSem-MAE significantly outperforms state-of-the-art across datasets. With only half labeled data, it outperforms most existing methods trained on full data.