Learning the Shared Latent Structure of Cryo-EM Inverse Problems

Nithin Chalapathi; Theo Sternlieb; Ethan Han; Sonya M. Hanson; Pilar Cossio; Aditi S. Krishnapriyan; Ritwik Gupta

cs.CVOct 15, 2026

Learning the Shared Latent Structure of Cryo-EM Inverse Problems

Authors: Nithin Chalapathi, Theo Sternlieb, Ethan Han, Sonya M. Hanson, Pilar Cossio, Aditi S. Krishnapriyan, Ritwik Gupta

Organizations: University of California, Berkeley · Flatiron Institute · University of Maryland, College Park

Abstract

Understanding biomolecular function requires characterizing three-dimensional structure and conformational variability, for which cryogenic electron microscopy (cryo-EM) is a powerful tool. Cryo-EM produces noisy two-dimensional particle images in which molecular structure, particle pose, and imaging conditions are jointly encoded. Recovering biological information from these observations spans inverse problems including pose estimation, heterogeneity analysis, and reconstruction. These tasks are usually addressed independently despite shared protein structure and image formation, motivating a common representation across inverse problems. Since biologically meaningful differences may be spatially localized, we study self-distillation with complementary global and masked patch objectives. Because experimental dataset discrimination can reflect acquisition-specific cues, we develop controlled synthetic and experimental evaluations and find that structural discrimination benefits from dense spatial features. We introduce the EMber family of vision transformers, pre-trained on >50 million experimental particle images from 214 EMPIAR datasets together with simulated projections from 268,000 molecular assemblies. EMber improves upon existing methods at conformational and compositional discrimination by 14.85 and 14.26 percentage points, estimates conformational populations to within 1.59pp error with a linear probe, and produces particle assignments yielding reference-quality reconstructions in fewer epochs. We demonstrate how to learn representations that capture biological information for use across multiple cryo-EM inverse problems.

Figures & tables

Figure 1: Cryo-EM overview and shared inverse problems . (A) Proteins share similar structural features across molecular systems. (B) Cryo-EM images are generated with an image formation process involving projection, imaging effects, and measurement noise. (C) Downstream tasks attempt to recover different variables of the same underlying generative process, relying on overlapping structural and imaging information.

Figure 2: Augmentations can be unphysical . (A) 2D rotations of an image do not span all possible rotations of a 3D protein. (B) Image-space crops and blurs do not translate to the cryo-EM forward model.

Figure 3: Compositional heterogeneity classification on CAHRA experimental data. (A) Fine-tuning accuracy versus model FLOPs, with a supervised ResNet-18 reference. (B) Accuracy versus model FLOPs when linear probing frozen backbones, with a reference supervised ResNet-18. (C) Pooled linear/MLP readouts with a convolutional head that preserves spatial information.

Figure 4: Fine-tuning for Hsp90 conformation classification. We discretize Hsp90 conformations according to CMA and CMB chain-opening angles, each with 20 classes, and average accuracy across the two prediction heads (random chance: 5%). (A) Representative conformations and rendered particles at different SNRs. (B) Top-1 accuracy at SNR 0.1 versus model forward FLOPs. (C) Accuracy across SNRs. Model differences largely disappear at SNR 0.001, where all methods approach chance.

Figure 6: Particle grouping measured through reconstruction resolution. After each epoch of CryoDRGN-AI pose search, particles are clustered in the learned latent space and reconstructed using ground truth poses. (A) Tomotwin-8: Boxes show the spread in FSC=0.5 resolution across eight structures. The dashed line is an Oracle reconstruction obtained with ground truth group assignments. Notably, EMber is already near the Oracle after one epoch. EMber reaches near reference resolution within two epochs, and has less variation across the reconstructed clusters compared to baseline methods. (B) Hsp90: Reconstructions visualized after 2 epochs of CryoDRGN-AI pose search. Colors represent the signed distance to the reference surface (negative inside, positive outside).

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Dataset Composition. Visualization of statistics coming from our EMPIAR dataset and the OpenProteinSet containing assemblies. (A) Training set EMPIAR entries organized by coarse biological categories. (B) Pixel sizes of EMPIAR entries. At the same pixel dimensions, larger pixel sizes correspond to wider field of views. (C) Amino acid counts of synthetic and experimental datasets.

Parameter	Value or range	Sampling
Accelerating voltage	$300\,\mathrm{kV}$	Fixed
Spherical aberration ( $C_{s}$ )	$2.7\,\mathrm{mm}$	Fixed
Amplitude contrast ratio	$0.1$	Fixed
Mean defocus	$0.9$ – $2.5\,\mu\mathrm{m}$	Uniform
Astigmatism	$0$ – $200\,\mathrm{\text{\AA}}$	Uniform
Astigmatism angle	$0^{\circ}$ – $180^{\circ}$	Uniform

Appendix

Table 2: CTF and envelope parameters used for synthetic pre-training.

Figure 8: Example Dense Features. Experimental cryo-EM image, low-pass filtered for visibility (left), followed by the image features colored by PC1, PC2, and PC3, L2 vector norms of each patch, and, finally, the cosine distance between each patch’s features and those of the center patch. EMber produces dense, semantically meaningful features.

	EMber				Reference models
Property	Small	Base	Large	Giant	DINOv2	DINOv3
Backbone	ViT-S/16	ViT-B/16	ViT-L/16	ViT-g/16	ViT-g/14	ViT-7B/16
Backbone parameters	21.4M	85.3M	304.2M	1133.7M	1.1B	6.7B
Backbone + heads parameters	233.2M	301.8M	523.9M	1359.7M	—	—
Transformer blocks	12	12	24	40	40	40
Input channels	1	1	1	1	3	3

Appendix

Table 3: Hyperparameter comparison between our models and the published DINOv2/v3.

Figure 9: Experimental data schedule. Visualization of the proportion of training batch that comes from experimental data. We pre-train on primarily synthetic images, before using a cosine schedule to mid-training where we sample using a dispersion equation.

Figure 10: Visualized CryoPPP Experimental Images. (Top) EMPIAR 10017 $\beta$ -galactosidase and (Bottom) EMPIAR 10061 also $\beta$ -galactosidase but bound to an inhibitor. Structurally similar, but present visually different, potentially due to experimental conditions.

Figure 11: Exploratory TRPV1 visualization. (A) Three randomly sampled query particles from each entry, resized to $384\times 384$ and standardized separately. Intensities are displayed over $[-3,3]$ standard deviations; this display clipping is not applied during evaluation. The images are not shown at a common physical scale. (B,C) PCA and UMAP fitted to normalized CLS features from all 29,000 feature-bank particles, using a ViT-S teacher checkpoint at step 79,999. Blue circles indicate EMPIAR-10005, orange triangles indicate EMPIAR-10059, and the other 27 entries are gray. UMAP uses 15 neighbors, a minimum distance of 0.1, and seed 42. This earlier checkpoint and CLS-only representation differ from those used for the reported $k$ NN results.

Figure 12: Hsp90 Fine-tuning loss curves. We visualize the training loss across different learning rates when fine-tuning on the Hsp90 dataset.

Figure 13: Spike protein structures from the Spike MD simulation used for Spike population estimation task plotted as a function of the collective variables (principal component 1, principal component 2). The original distribution of structures is shown in (A), with the filtered structures in (B).

Figure 14: True Spike-Up population vs predicted Spike-Up population across a range of training dataset sizes used to fit the linear classifier, and stratified based on evaluation SNR level.

Figure 15: Fourier Shell Correlation plots of Hsp90 reconstructions. Rows correspond to different methods, and columns correspond to pose search epochs. The two curves represent the open and closed states. The dashed line indicates FSC=

0.5

	Epoch 1		Epoch 2
Method	Closed	Open	Closed	Open
Oracle	4.48	4.50	-	-
Lookup Table	8.71	9.12	8.78	9.10
CNN Encoder	7.27	6.43	5.08	4.78
Cryo-IEF-B	6.43	5.46	7.42	6.08
CryoUNI-B	5.37	5.39	4.78	4.97

Appendix

Table 4: Class-wise reconstruction resolution at FSC

{}=0.5

for Hsp90 at an SNR of 0.01, reported in Å. Reference class 0 corresponds to the closed state, and reference class 1 corresponds to the open state. Lower is better. The corresponding ground-truth resolutions are 4.48 Å (closed) and 4.50 Å (open).

Figure 16: Fourier Shell Correlation plots of Tomotwin-8 reconstructions. Rows correspond to different methods, and columns correspond to pose search epochs. Each color represents one of the 8 reconstructed structures. The dashed line indicates FSC=

0.5

Cryo-Electron Microscopy

Cryo-electron microscopy reconstructs biological structures from images of frozen specimens, from cellular volumes to atomic-resolution molecular models.

See all 27 papers

Fold'EM: Direct atomic structure inference from Cryo-EM particles
Oct 1, 2026Advaith Maddipatla, Märt-Erik Mäeots, Marco Pegoraro +5Cryo-Electron Microscopy Electron Microscopy
Institute of Science and Technology Austria · University of Zurich · Frankfurt Institute for Advanced Studies +2
ARCHER: Amortized cross-specimen pose estimation for cryo-electron microscopy
Aug 22, 2026Nhan D. Nguyen, Bao PhamCryo-Electron Microscopy Object Pose Estimation
Pritzker School of Molecular Engineering, University of Chicago, Chicago, Illinois 60637, United States · Department of Computer Science, Rensselaer Polytechnic Institute, Troy, New York 12180, United States
Score-Based Matching with Target Guidance for Cryo-EM Denoising
Apr 20, 2026Xiaoqi Wu, Xueying Zhan, Wen Li +3Cryo-Electron Microscopy Score-Based Generative Modeling
Carnegie Mellon University · Towson University

Oct 1, 2026J/K move · Enter open · S save

Fold'EM: Direct atomic structure inference from Cryo-EM particles

Advaith Maddipatla, Märt-Erik Mäeots, Marco Pegoraro, Nikolaus Dräger+4

Institute of Science and Technology Austria · University of Zurich · Frankfurt Institute for Advanced Studies+2

Cryo-Electron Microscopy Electron Microscopy

Open paper AskPDF arXiv

Single-particle cryo-electron microscopy (cryo-EM) has become a widely adopted technique for biomolecular structure determination. The conventional cryo-EM computational pipeline first combines many particle images to reconstruct an electrostatic potential (ESP) map and then fits an atomic model to the recovered map. Density reconstruction has high sample complexity, requiring large numbers of particle images and making structure determination high-cost and low-throughput, particularly for heterogeneous samples. Downstream atomic model building, in turn, becomes increasingly difficult as the resolution of the reconstructed map deteriorates. Protein structure prediction models provide strong sequence-derived priors on atomic structure, and experiment-guided approaches can use these priors to recover structures consistent with experimental measurements. Yet, in cryo-EM, such priors are typically integrated only after density reconstruction during atomic model fitting. We introduce Fold'EM, an inference-time framework that combines priors from protein generative models directly with cryo-EM particle images to determine atomic models from a small number of single particle images, bypassing both intermediate density reconstruction and downstream model building against the reconstructed map. Across synthetic and experimental cryo-EM datasets, Fold'EM recovers accurate atomic structures both with known particle orientations and in an ab-initio setting where orientations are inferred jointly with structure. In heterogeneous datasets, Fold'EM further resolves distinct conformational states from mixed particle populations without separately reconstructing a density map and building an atomic model for each state. We believe these results open new avenues for structure determination in the low-sample regime and for characterizing low-population conformational states directly from cryo-EM particles.

Abstract

Cryo-Electron Microscopy

Explore similar work

Fold'EM: Direct atomic structure inference from Cryo-EM particles

ARCHER: Amortized cross-specimen pose estimation for cryo-electron microscopy

Score-Based Matching with Target Guidance for Cryo-EM Denoising