Bridging the Synthetic-to-Real Gap for Few-Shot Cryo-ET Classification

Siddhant Bharadwaj; Ashish Vashist; Rashi Singh; Pranav Vinodh; Nishanth Artham; Runmin Jiang; Xingjian Li; Min Xu

cs.CVSep 12, 2026

Bridging the Synthetic-to-Real Gap for Few-Shot Cryo-ET Classification

Authors: Siddhant Bharadwaj, Ashish Vashist, Rashi Singh, Pranav Vinodh, Nishanth Artham, Runmin Jiang, Xingjian Li, Min Xu

Organizations: Columbia University · Carnegie Mellon University · National Institute of Technology Karnataka · University of Minnesota – Twin Cities

Abstract

Subtomogram classification in cryo-electron tomography (cryo-ET) is a challenging problem due to the scarcity of labeled examples. While cryo-ET simulators can be adopted to generate unlimited synthetic data, the substantial domain gap between synthetic and real subtomograms hinders its practical utilization. In this work, we propose a novel synthetic-to-real adaptation framework with a learnable transformation module, bridging this gap at both the input and feature levels. Extensive experiments demonstrate that our method consistently outperforms existing transfer learning baselines in few-shot settings.

Figures & tables

Figure 1: Hierarchical Synthetic-to-Real Adaptation Framework. Synthetic source volumes

x^{S}

are transformed by a learned input adaptation module (

T_{\phi}

) before passing through a shared 3D Video Swin Transformer backbone (

f_{\theta}

), while experimental real target volumes

x^{R}

are directly embedded. Domain alignment losses (

\mathcal{L}_{\text{align}}

) backpropagate gradients into transformation parameters

\phi

Category	Operation	Learnable param.	Discrepancy addressed
Spatial	3D Affine Warp	$\theta_{\text{aff}}\in\mathbb{R}^{3\times 4}$	Pose & geometric distortion
Spatial	Instance-adaptive Channel	$M_{x}\in\mathbb{R}^{C\times C},\,b_{x}\in\mathbb{R}^{C}$	Per-sample channel correction
Intensity	Diff.-of-Gaussians	$\sigma_{1},\sigma_{2}$	Noise & frequency profile
	Brightness & Contrast	$\alpha,\beta$	Global intensity offset
	Gamma Correction	$\gamma$	Nonlinear intensity mapping
Channel	Global Channel Transform	$M\in\mathbb{R}^{C\times C},\,b\in\mathbb{R}^{C}$	Channel-level statistical bias

Table 1: Components of the learnable sim-to-real transformation module

T_{\phi}

Figure 2: Step-by-step progression of the learnable sim-to-real transformation pipeline. Synthetic subtomograms undergo sequential residual adaptation across spatial (STN), intensity (DoG frequency filtering, contrast adjustment, gamma correction), and global channel alignment stages.

Dataset	Method / Architecture	Input Transform	Feature DA	Test Acc (%)	Balanced Acc (%)	Macro-F1 (%)
Noble 3-shot	ResNet-34 Baseline	–	–	18.40 $\pm$ 6.12	18.16 $\pm$ 6.05	6.62 $\pm$ 4.39
	Swin3D Baseline (Target Only)	–	–	55.79 $\pm$ 4.00	55.67 $\pm$ 4.01	53.89 $\pm$ 4.66
	Naive Joint (Syn + Real, No DA)	–	–	62.26 $\pm$ 4.31	62.14 $\pm$ 4.28	61.91 $\pm$ 4.17
	Yu et al. (2021) [DANN]	–	DANN	51.46 $\pm$ 7.47	51.18 $\pm$ 7.58	49.69 $\pm$ 8.36
	Swin3D + MMD (DA Only)	–	MMD	58.80 $\pm$ 6.09	58.76 $\pm$ 6.20	57.56 $\pm$ 7.00
	Swin3D + CORAL (DA Only)	–	CORAL	61.46 $\pm$ 5.19	61.35 $\pm$ 5.08	61.50 $\pm$ 5.12

Table 2: Main Classification Benchmark Results (Mean

\pm

Std over 10 Seeds). Held-out test performance across 3-shot and 5-shot regimes on Noble and Qiang datasets. Statistical significance against DA-Only is indicated by

{}^{\dagger}(p<0.05)

Configuration	Active Input Transformations	Noble 3-shot (%)	Qiang 3-shot (%)
Full Proposed Model	STN + Intensity + Channel	62.35 $\pm$ 3.81	54.75 $\pm$ 9.69
w/o Channel Calibration	STN + Intensity	53.02 $\pm$ 7.39	51.00 $\pm$ 8.15
w/o Intensity Transform	STN + Channel	53.85 $\pm$ 4.96	53.62 $\pm$ 8.40
w/o Spatial Transformer (STN)	Intensity + Channel	53.61 $\pm$ 7.70	47.04 $\pm$ 7.20
Naive Joint (Syn + Real)	None	62.26 $\pm$ 4.31	51.93 $\pm$ 5.60
Target-Only Swin3D Baseline	None	55.79 $\pm$ 4.00	41.08 $\pm$ 14.50

Table 3: Component Ablation Study. Dissecting individual transformation sub-modules on 3-shot benchmarks.

Dataset & Method	$\lambda_{\text{res}}=0.0$	$\lambda_{\text{res}}=0.2$	$\lambda_{\text{res}}=0.4$	$\lambda_{\text{res}}=0.6$	$\lambda_{\text{res}}=0.8$	$\lambda_{\text{res}}=1.0$
Noble 3-shot (CORAL)	62.35 $\pm$ 3.81	59.49 $\pm$ 4.77	57.48 $\pm$ 7.81	57.42 $\pm$ 4.54	53.17 $\pm$ 8.74	56.63 $\pm$ 6.18
Noble 3-shot (MMD)	54.94 $\pm$ 7.42	54.65 $\pm$ 8.12	54.68 $\pm$ 6.25	53.60 $\pm$ 7.75	53.24 $\pm$ 7.43	53.22 $\pm$ 7.01
Noble 5-shot (CORAL)	62.13 $\pm$ 5.66	57.00 $\pm$ 4.27	60.28 $\pm$ 4.06	57.30 $\pm$ 4.62	62.11 $\pm$ 5.52	61.12 $\pm$ 6.21
Noble 5-shot (MMD)	67.45 $\pm$ 3.06	60.95 $\pm$ 6.44	56.96 $\pm$ 5.36	57.98 $\pm$ 4.31	57.89 $\pm$ 7.73	57.04 $\pm$ 7.27
Qiang 3-shot (CORAL)	50.76 $\pm$ 7.40	53.18 $\pm$ 6.35	54.30 $\pm$ 7.15	52.69 $\pm$ 9.67	54.04 $\pm$ 8.21	53.32 $\pm$ 7.31
Qiang 3-shot (MMD)	54.04 $\pm$ 7.66	54.75 $\pm$ 9.69	51.88 $\pm$ 12.41	47.80 $\pm$ 8.04	52.15 $\pm$ 8.67	51.61 $\pm$ 5.99

Table 4: Test Accuracy across Residual Blending Sweeps (

\lambda_{\text{res}}

). Held-out test accuracy (%) over 10 seeds across all datasets and alignment losses.

Dataset & Method	$\lambda_{\text{res}}=0.0$	$\lambda_{\text{res}}=0.2$	$\lambda_{\text{res}}=0.4$	$\lambda_{\text{res}}=0.6$	$\lambda_{\text{res}}=0.8$	$\lambda_{\text{res}}=1.0$
Noble 3-shot (CORAL)	74.00 $\pm$ 9.07	58.35 $\pm$ 4.70	62.23 $\pm$ 8.72	58.35 $\pm$ 3.00	72.20 $\pm$ 8.73	71.00 $\pm$ 11.25
Noble 3-shot (MMD)	75.50 $\pm$ 10.39	69.52 $\pm$ 12.14	69.15 $\pm$ 6.79	66.21 $\pm$ 9.24	67.15 $\pm$ 14.11	70.37 $\pm$ 13.78
Noble 5-shot (CORAL)	76.50 $\pm$ 6.26	73.93 $\pm$ 8.90	69.43 $\pm$ 11.53	72.58 $\pm$ 7.49	67.28 $\pm$ 11.00	71.63 $\pm$ 7.09
Noble 5-shot (MMD)	73.50 $\pm$ 7.47	70.91 $\pm$ 10.65	68.20 $\pm$ 12.02	72.91 $\pm$ 8.45	66.20 $\pm$ 10.94	68.57 $\pm$ 7.92
Qiang 3-shot (CORAL)	56.91 $\pm$ 6.90	60.31 $\pm$ 9.84	63.90 $\pm$ 5.40	55.61 $\pm$ 9.21	60.85 $\pm$ 10.34	58.83 $\pm$ 8.16
Qiang 3-shot (MMD)	55.96 $\pm$ 9.59	61.39 $\pm$ 7.96	59.06 $\pm$ 11.87	55.52 $\pm$ 9.95	59.96 $\pm$ 8.64	54.53 $\pm$ 9.04

Table 5: Validation Accuracy across Residual Blending Sweeps (

\lambda_{\text{res}}

). Corresponding validation set accuracy (%) over 10 seeds demonstrating model selection stability.

Panel of Figure 3 — Figure 3: Supplementary Visualizations. (a, b) Normalized test confusion matrices. (c) Feature distribution embeddings.

Synthetic-to-Real Domain Adaptation

Synthetic-to-real domain adaptation transfers models trained on simulated or generated data to real-world settings by reducing distribution shifts.

See all 137 papers

The microscope is the mask: privileged views and labels from a cryo-ET forward model
Sep 3, 2026Bogdan Toader, Kiarash Jamali, Tanmay A. M. Bharat +1Protein Representation Learning Cryo-Electron Microscopy
MRC Laboratory of Molecular Biology Cambridge, UK
A meta-algorithm for ab initio reconstruction of complex mixtures in cryo-EM
Aug 26, 2026Alkin Kaz, Arda Kaz, Ellen D. ZhongCryo-Electron Microscopy Electron Microscopy
Princeton University · University of California, Berkeley
TopoFuse: Topology-Aware Tri-Planar Fusion for 3D Cryo-Electron Tomography Segmentation
Aug 31, 2026Rohit Kumar Salla, Neelesh Gupta, Xingjian Li +1Topology-Preserving Image Segmentation Image Segmentation
Virginia Tech, Blacksburg, VA, USA · National Institute of Technology, Kurukshetra, India · Carnegie Mellon University, Pittsburgh, PA, USA

Sep 3, 2026J/K move · Enter open · S save

The microscope is the mask: privileged views and labels from a cryo-ET forward model

Bogdan Toader, Kiarash Jamali, Tanmay A. M. Bharat, Sjors H. W. Scheres

MRC Laboratory of Molecular Biology Cambridge, UK

Protein Representation Learning Cryo-Electron Microscopy Self-Supervised Learning Learning Using Privileged Information

Open paper AskPDF arXiv

We explore the use of simulated data for training a model for protein annotation in crowded cryo-electron tomography volumes reconstructed from images collected at limited tilt angles and severely corrupted by the measurement operator. Firstly, we leverage the corruptions imposed by the forward model to generate domain-specific augmented paired views of the exact same scene for an invariance objective integrated into the LeJEPA self-supervised training framework. Secondly, we use additional information from the simulation pipeline such as the positions and identity of proteins in the simulated volumes to inform the architecture of the model and the loss function, so that semantic information is localised at protein positions in the resulting dense feature volume. The resulting model, CARNIVAL, is evaluated without finetuning on classification and detection tasks in real tomograms, using a benchmark dataset containing multiple protein types and two tomogram processing types. We show that CARNIVAL outperforms a state-of-the-art model trained using a contrastive objective on simulated data but without forward model-based paired views or privileged information.

Abstract

Synthetic-to-Real Domain Adaptation

Explore similar work

The microscope is the mask: privileged views and labels from a cryo-ET forward model

A meta-algorithm for ab initio reconstruction of complex mixtures in cryo-EM

TopoFuse: Topology-Aware Tri-Planar Fusion for 3D Cryo-Electron Tomography Segmentation