WildMAE: Structure-Aware Masked Autoencoding for Wildlife Image Reconstruction

Yicheng Zhang; Yujie Jin; Ke Zhang; Jiaxin Wu

cs.CVOct 20, 2026

WildMAE: Structure-Aware Masked Autoencoding for Wildlife Image Reconstruction

Authors: Yicheng Zhang, Yujie Jin, Ke Zhang, Jiaxin Wu

Organizations: The Chinese University of Hong Kong, Shenzhen

Abstract

Camera-trap imagery presents substantial variation in animal appearance, illumination, background clutter, and visual degradation. Masked autoencoders (MAEs) offer a label-free framework for modeling such data through reconstruction of randomly hidden image patches. We present WildMAE, a decoder-side extension of MAE that combines local structure-aware refinement with masked-to-visible cross-attention. A zero-gated convolutional residual path introduces a two-dimensional neighborhood prior, while gated cross-attention allows masked positions to retrieve contextual information from visible-position features without replacing the original Transformer decoder. The associated training recipe progressively introduces normalized-pixel, Sobel-edge, and perceptual objectives. In the original Snapshot Serengeti study, covering 4,744 test images at 75% masking, the reported reconstruction scores were 27.90 versus 24.33 dB PSNR, 0.840 versus 0.780 SSIM, and 0.160 versus 0.320 LPIPS for WildMAE and vanilla MAE, respectively. Additional historical records document favorable average differences under four synthetic degradation conditions and describe component variants. We present the method, experimental observations, and their scope together: the retained metrics are historical protocol-dependent results, rather than a newly reproduced, harmonized comparison or a validation of restoration from genuinely unavailable pixels. WildMAE provides a practical architectural case study for combining global token mixing with spatial and targeted contextual refinement in wildlife masked-image modeling.

Figures & tables

Figure 1: High-level WildMAE pipeline: the original MAE encoder and Transformer decoder are retained, followed by a local spatial refinement, masked-to-visible contextual refinement, and the pixel-prediction head. Exact internal normalizations and residual gates are specified in Sections 3.2 – 3.3 . The diagram is conceptual.

Figure 2: Conceptual illustration of the two appended refinement operations, adapted from the original project report. The actual implementation uses depthwise convolution and pre-normalization in the local branch, plus separately zero-gated cross-attention and MLP updates in each cross-refinement block (Equations 2 – 6 ).

Model	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Vanilla MAE	24.33	0.780	0.320
WildMAE	27.90	0.840	0.160

Table 1: Original reported reconstruction results at nominal 75% masking on 4,744 test images. Historical course-report values; not newly reproduced under a common evaluator.

Variant	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Vanilla MAE + basic pixel loss	24.33	0.780	0.320
Cross-attention refinement only + basic loss	25.90	0.813	0.217
Local structural refinement only + basic loss	25.73	0.812	0.224
WildMAE dual refinement + basic loss	25.75	0.810	0.221
WildMAE dual refinement + staged loss	27.90	0.840	0.160

Table 2: Component-variant results as reported in the original project at 75% masking. The rows are descriptive historical records, not a newly verified matched-control series.

Input condition	$\Delta$ PSNR $\uparrow$	$\Delta$ SSIM $\uparrow$	LPIPS reduction $\uparrow$
Clean	+2.591	+0.075	0.095
Gaussian noise	+1.128	+0.031	0.052
Low light	+0.078	+0.033	0.018
Motion blur	+1.537	+0.026	0.033
Random occlusion	+0.228	+0.030	0.035

Table 3: Descriptive differences in retained degradation records at nominal 75% masking.

\Delta

PSNR and

\Delta

SSIM are WildMAE minus baseline; LPIPS reduction is baseline minus WildMAE. The original report rounded these quantities to three decimal places.

Masked Image Modeling

Masked image modeling trains visual representations by hiding image regions and learning to reconstruct their pixels or features.

See all 42 papers

Structure-Guided Masked Autoencoders for Ultra-High Resolution Scientific Image Understanding
Sep 25, 2026Enzhi Zhang, Du Wu, Rui Zhong +18Masked Autoencoders Masked Image Modeling
Hokkaido University, Japan · RIKEN Center for Computational Science (R-CCS), Japan · Oak Ridge National Laboratory, USA +4
The hidden advantage of mask resampling: a theory of masked autoencoders
Oct 1, 2026Jorge Medina Moreira, Lorenzo Bardone, Lenka ZdeborováMasked Autoencoders Masked Language Modeling
Statistical Physics of Computation Laboratory, École Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland
Masked Swingers: Harnessing Data Augmentation to Advance Autoencoders for Self-Supervised Learning
Sep 29, 2026Anthony Fuller, Scott C. Lowe, Daniel G. Kyrollos +3Masked Autoencoders Invariant Representation Learning
Vector Institute · Carleton University · University of Guelph +1

Sep 25, 2026J/K move · Enter open · S save

Structure-Guided Masked Autoencoders for Ultra-High Resolution Scientific Image Understanding

Enzhi Zhang, Du Wu, Rui Zhong, Cong Ma+17

Hokkaido University, Japan · RIKEN Center for Computational Science (R-CCS), Japan · Oak Ridge National Laboratory, USA+4

Masked Autoencoders Masked Image Modeling Self-Supervised Pre-Training Image Tokenization

Open paper AskPDF arXiv

Self-supervised pre-training with Vision Transformers, including Masked Autoencoders (MAE), is difficult to apply to gigapixel scientific images. Random masking is poorly matched to the structured, multi-scale morphology of scientific data, while uniform tokenization produces prohibitively long sequences that make

O(N^2)

attention impractical. We propose SGMA, a structure-guided masked autoencoding framework for ultra-high-resolution scientific images. SGMA couples two components: a content-adaptive quadtree tokenizer that compresses gigapixel images into a fixed-length sequence, and a structure-conditioned masking process that biases reconstruction toward spatially informative regions. To stabilize this process across scales, we introduce Damped Accumulation (DA), which aggregates signal-dependent responses across the tree into a structure canvas used to guide masking. The resulting pre-training task preserves fine microstructure while remaining compatible with standard ViT encoders and MAE-style reconstruction. Across electron microscopy, whole-slide optical microscopy, and X-ray CT datasets, SGMA consistently outperforms MAE baselines. It achieves 95.68% Dice on the 8K x 8K x 28K SpringXCT dataset, improving over the same-architecture MAE baseline by +13.00 points, and 83.21% Dice on the 32K^2 WSI PAIP dataset, improving by +16.84 points, while providing up to a 24.8x inference speedup.

Abstract

Masked Image Modeling

Explore similar work

Structure-Guided Masked Autoencoders for Ultra-High Resolution Scientific Image Understanding

The hidden advantage of mask resampling: a theory of masked autoencoders

Masked Swingers: Harnessing Data Augmentation to Advance Autoencoders for Self-Supervised Learning