CryoAtlas: A Large Curated Dataset and Unified Benchmark for Cryo-EM Atomic Model Building
Organizations: ShanghaiTech University · Cellverse Co., Ltd.
Abstract
Deep learning has accelerated automated atomic model building from cryo-EM maps, yet progress remains limited by the lack of large, high-quality datasets and standardized evaluation benchmarks. We address this gap by introducing CryoAtlas, a rigorously curated dataset and unified benchmark for density-guided atomic model building of protein chains. By integrating and filtering EMDB and PDB entries (up to March 2026), CryoAtlas provides 17,809 map-model-sequence entries, including a curated 16,657-entry training split. From this resource, we construct a 618-target benchmark with a shared evaluation protocol that jointly assesses structural accuracy, model-to-map fit, and stereochemical validity. Using this framework, we benchmark state-of-the-art learning-based methods against Phenix and AlphaFold3, comprehensively analyzing the impact of map resolution, sequence length, protein category and sequence identity on modeling accuracy. As a case study, retraining the ModelAngelo CNN component on CryoAtlas improves performance on the benchmark. CryoAtlas provides an open, reproducible foundation for future cryo-EM atomic model building research. Project details and resource release updates are available at https://oulling.github.io/cryoatlas/.
Figures & tables
| Method | Geom Score ( ) | C RMSD ( ) | TM-score ( ) | Seq. Recall ( ) | GT Score ( ) | Q-score ( ) | Map Score ( ) |
|---|---|---|---|---|---|---|---|
| Phenix (24h) † | 0.462 | 3.562 | 0.075 | 0.063 | 0.194 | 0.539 | 0.193 |
| ModelAngelo | 0.462 | 1.014 | 0.648 | 0.739 | 0.723 | 0.600 | 0.520 |
| CryoAtom2 | 0.537 | 1.198 | 0.763 | 0.899 | 0.787 | 0.585 | 0.568 |
| EMProt | 0.484 | 1.224 | 0.639 | 0.734 | 0.702 | 0.588 | 0.517 |
| E3-CryoFold | 0.165 | 3.278 | 0.739 | 0.622 | 0.539 | 0.332 | 0.305 |
| AlphaFold 3 | 0.598 | 3.655 | 0.813 | 0.647 | 0.552 | 0.202 | 0.220 |
| Method | Geom Score ( ) | C RMSD ( ) | TM-score ( ) | Seq. Recall ( ) | GT Score ( ) | Q-score ( ) | Map Score ( ) |
|---|---|---|---|---|---|---|---|
| MA (Official) | 0.462 | 1.014 | 0.648 | 0.739 | 0.723 | 0.600 | 0.520 |
| MA (Cryo2Struct) | 0.455 | 1.108 | 0.604 | 0.708 | 0.697 | 0.599 | 0.505 |
| MA (CryoAtlas) | 0.470 | 0.769 | 0.680 | 0.753 | 0.755 | 0.603 | 0.532 |
| Training data | EMA loss ( ) | ( ) | ( ) |
|---|---|---|---|
| CryoAtlas (3 seeds) | |||
| Cryo2StructData (raw) | 0.1127 | 0.2836 | 0.2069 |
| Cryo2StructData (normalized) | 0.1475 | 0.2299 | 0.1565 |
| CryoAtlas + discarded | 0.1140 | 0.2841 | 0.1956 |
| Cryo2StructData + filters | 0.0865 | 0.2261 | 0.1911 |
Appendix figures & tables27 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | Geom Score ( ) | C RMSD ( ) | TM-score ( ) | Seq. recall ( ) | GT Score ( ) | Q-score ( ) | Map Score ( ) |
|---|---|---|---|---|---|---|---|
| Phenix (24h) † | |||||||
| ModelAngelo | |||||||
| CryoAtom2 | |||||||
| EMProt | |||||||
| E3-CryoFold | |||||||
| AlphaFold 3 |
| Method | Spearman | |
|---|---|---|
| ModelAngelo | 617 | |
| CryoAtom2 | 618 | |
| EMProt | 618 | |
| E3-CryoFold | 416 | |
| AlphaFold 3 | 618 | |
| Pooled (all five) | 2887 |
| Method | Rama-F (%) | Rama-O (%) | Rot-O (%) | C Dev | Clash | Bonds | Angles | MolProb |
|---|---|---|---|---|---|---|---|---|
| ModelAngelo | 91.262 | 3.074 | 10.388 | 0.000 | 69.207 | 0.067 | 5.481 | 3.530 |
| CryoAtom2 | 95.629 | 1.210 | 6.278 | 0.000 | 39.856 | 0.030 | 2.324 | 2.860 |
| EMProt | 92.244 | 2.399 | 13.227 | 0.000 | 69.703 | 0.031 | 2.369 | 3.583 |
| E3-CryoFold | 54.632 | 29.198 | 28.939 | 1136.202 | 431.612 | 0.347 | 20.753 | 5.125 |
| AlphaFold 3 | 94.821 | 1.200 | 1.442 | 1.068 | 12.511 | 0.013 | 1.648 | 1.978 |
| Method | Geom Score | ||||
|---|---|---|---|---|---|
| ModelAngelo | 0.477 | 0.464 | 0.491 | 0.417 | 0.462 |
| CryoAtom2 | 0.561 | 0.544 | 0.536 | 0.506 | 0.537 |
| EMProt | 0.474 | 0.488 | 0.467 | 0.505 | 0.484 |
| E3-CryoFold | 0.242 | 0.143 | 0.182 | 0.094 | 0.165 |
| AlphaFold 3 | 0.705 | 0.542 | 0.605 | 0.539 | 0.598 |
| Method | TM | RMSD-L | RMSD-G | Res. Cov. | Seq. Match | Seq. Recall | Compl. | GT Score |
|---|---|---|---|---|---|---|---|---|
| ModelAngelo | 0.648 | 1.014 | 4.436 | 0.760 | 0.952 | 0.739 | 0.779 | 0.723 |
| CryoAtom2 | 0.763 | 1.198 | 6.418 | 0.932 | 0.961 | 0.899 | 1.010 | 0.787 |
| EMProt | 0.639 | 1.224 | 5.472 | 0.766 | 0.937 | 0.734 | 0.779 | 0.702 |
| E3-CryoFold | 0.739 | 3.278 | 12.700 | 0.788 | 0.727 | 0.622 | 0.936 | 0.539 |
| AlphaFold 3 | 0.813 | 3.655 | 15.307 | 0.752 | 0.820 | 0.647 | 1.348 | 0.552 |
| Method | CC | CC | CC | CC | cc_mean | Q-score | Recall Q | Map Score |
|---|---|---|---|---|---|---|---|---|
| ModelAngelo | 0.680 | 0.678 | 0.404 | 0.553 | 0.579 | 0.600 | 0.462 | 0.520 |
| CryoAtom2 | 0.660 | 0.665 | 0.486 | 0.602 | 0.603 | 0.585 | 0.533 | 0.568 |
| EMProt | 0.692 | 0.689 | 0.402 | 0.555 | 0.584 | 0.588 | 0.449 | 0.517 |
| E3-CryoFold | 0.399 | 0.439 | 0.270 | 0.432 | 0.385 | 0.332 | 0.224 | 0.305 |
| AlphaFold 3 | 0.298 | 0.299 | 0.148 | 0.374 | 0.280 | 0.202 | 0.161 | 0.220 |
| MA variant | Geom Score ( ) | C RMSD ( ) | TM-score ( ) | Seq. recall ( ) | GT Score ( ) | Q-score ( ) | Map Score ( ) |
|---|---|---|---|---|---|---|---|
| Official | |||||||
| C2S | |||||||
| CryoAtlas |
| Method | Rama-F (%) | Rama-O (%) | Rot-O (%) | C Dev | Clash | Bonds | Angles | MolProb |
|---|---|---|---|---|---|---|---|---|
| ModelAngelo | 91.262 | 3.074 | 10.388 | 0.000 | 69.207 | 0.067 | 5.481 | 3.530 |
| MA (Cryo2Struct) | 90.854 | 3.297 | 10.410 | 0.000 | 71.524 | 0.077 | 5.823 | 3.553 |
| MA (CryoAtlas) | 91.399 | 2.860 | 10.110 | 0.000 | 66.189 | 0.059 | 5.164 | 3.489 |
| Method | Geom Score | ||||
|---|---|---|---|---|---|
| ModelAngelo | 0.477 | 0.464 | 0.491 | 0.417 | 0.462 |
| MA (Cryo2Struct) | 0.472 | 0.458 | 0.492 | 0.400 | 0.455 |
| MA (CryoAtlas) | 0.485 | 0.470 | 0.494 | 0.430 | 0.470 |
| Method | TM | RMSD | Res. Cov. | Seq. Match | Seq. Recall | Compl. | GT Score |
|---|---|---|---|---|---|---|---|
| ModelAngelo | 0.648 | 1.014 | 0.760 | 0.952 | 0.739 | 0.779 | 0.723 |
| MA (Cryo2Struct) | 0.604 | 1.108 | 0.728 | 0.947 | 0.708 | 0.748 | 0.697 |
| MA (CryoAtlas) | 0.680 | 0.769 | 0.771 | 0.955 | 0.753 | 0.787 | 0.755 |
| Method | CC | CC | CC | CC | cc_mean | Q-score | Recall Q | Map Score |
|---|---|---|---|---|---|---|---|---|
| ModelAngelo | 0.680 | 0.678 | 0.404 | 0.553 | 0.579 | 0.600 | 0.462 | 0.520 |
| MA (Cryo2Struct) | 0.671 | 0.669 | 0.381 | 0.539 | 0.565 | 0.599 | 0.445 | 0.505 |
| MA (CryoAtlas) | 0.693 | 0.690 | 0.418 | 0.562 | 0.591 | 0.603 | 0.473 | 0.532 |
| Statistic | CryoAtlas Full | CryoAtlas | Cryo2StructData Normalized | Cryo2StructData Raw EMDB |
|---|---|---|---|---|
| Mean | 0.596 | 0.611 | 0.416 | 0.595 |
| Q1 | 0.520 | 0.540 | 0.290 | 0.540 |
| Median | 0.610 | 0.620 | 0.440 | 0.610 |
| Q3 | 0.680 | 0.680 | 0.560 | 0.670 |
| Min | 0.300 | 0.300 | ||
| Max | 0.950 | 0.950 | 0.910 | 0.950 |
| Threshold | CryoAtlas Full | CryoAtlas | Cryo2StructData Normalized | Cryo2StructData Raw EMDB |
|---|---|---|---|---|
| Q-score | 0 | 0 | 181 | 25 |
| Q-score | 0 | 0 | 472 | 50 |
| Q-score | 0 | 0 | 962 | 67 |
| Q-score | 0 | 0 | 1,845 | 136 |
| Q-score | 3,239 | 2,182 | 4,542 | 1,194 |
| Resolution (Å) | Candidate | Curated | Test pool | Benchmark | Rejected | Q-score removal |
|---|---|---|---|---|---|---|
| 8.0 | 8.3 | 6.2 | 4.4 | 6.5 | 1.6 | |
| 2.5–3.0 | 25.8 | 27.3 | 24.0 | 28.6 | 18.0 | 1.3 |
| 3.0–3.5 | 35.9 | 37.3 | 35.2 | 38.8 | 29.0 | 1.5 |
| 3.5–4.0 | 19.6 | 19.2 | 22.1 | 15.9 | 21.6 | 3.8 |
| 4.0–4.5 | 7.0 | 6.1 | 10.8 | 11.8 | 11.7 | 9.1 |
| 4.5–5.0 | 2.1 | 1.2 | 1.1 | 0.2 | 6.4 | 21.6 |
| Taxonomic group | Candidate | Curated | Test pool | Benchmark | Rejected |
|---|---|---|---|---|---|
| Eukaryota | 53.3 | 56.9 | 55.4 | 56.1 | 34.8 |
| Bacteria | 16.6 | 17.0 | 21.3 | 18.8 | 14.2 |
| Viruses | 10.9 | 7.2 | 7.0 | 5.7 | 30.5 |
| Mixed | 17.8 | 17.7 | 14.8 | 18.4 | 18.1 |
| Archaea | 1.0 | 0.9 | 0.9 | 0.3 | 1.6 |
| Other | 0.4 | 0.3 | 0.7 | 0.6 | 0.8 |
| Protein chains | Candidate | Curated | Test pool | Benchmark | Rejected |
|---|---|---|---|---|---|
| 1 | 8.7 | 7.5 | 6.1 | 6.7 | 14.8 |
| 2 | 13.8 | 14.6 | 12.3 | 15.7 | 10.0 |
| 3–4 | 27.0 | 26.9 | 25.2 | 30.8 | 27.4 |
| 5–8 | 27.0 | 28.1 | 28.4 | 31.8 | 21.3 |
| 9–16 | 12.9 | 12.6 | 14.3 | 11.2 | 14.1 |
| 10.7 | 10.3 | 13.7 | 3.7 | 12.4 |
| Map size (MiB) | Candidate | Curated | Test pool | Benchmark | Rejected |
|---|---|---|---|---|---|
| 15.7 | 15.7 | 19.2 | 17.6 | 15.8 | |
| 64–128 | 29.7 | 31.8 | 32.8 | 34.3 | 18.7 |
| 128–256 | 34.8 | 35.5 | 31.0 | 31.9 | 31.1 |
| 256–512 | 10.1 | 9.5 | 9.8 | 9.9 | 12.9 |
| 512–1,024 | 7.8 | 6.5 | 5.9 | 5.0 | 14.8 |
| 1.9 | 1.0 | 1.3 | 1.3 | 6.8 |
| Sequence length (residues) | Candidate | Curated | Test pool | Benchmark | Rejected |
|---|---|---|---|---|---|
| 0.7 | 0.1 | 0.1 | 0 | 3.8 | |
| 301–600 | 4.2 | 3.0 | 2.3 | 2.8 | 10.7 |
| 601–1,000 | 8.8 | 7.3 | 7.6 | 10.1 | 16.8 |
| 1,001–2,000 | 32.1 | 33.6 | 34.8 | 43.5 | 24.1 |
| 2,001–5,000 | 37.6 | 40.1 | 37.4 | 43.7 | 24.7 |
| 16.6 | 15.9 | 17.8 | 0 | 19.9 |
| Resolution (Å) | CryoAtlas test | Cryo2StructData | ModelAngelo |
|---|---|---|---|
| 6.2 | 1.1 | 11.3 | |
| 2.5–3.0 | 24.0 | 3.2 | 27.1 |
| 3.0–3.5 | 35.2 | 26.7 | 39.0 |
| 3.5–4.0 | 22.1 | 43.9 | 22.6 |
| 4.0–4.5 | 10.8 | 24.5 | 0 |
| 4.5–5.0 | 1.1 | 0 | 0 |
| Curation/design choice | MA /CA | Cryo2StructData | CryoDataBot | CryoAtlas |
| Map–model misalignment | Manual/model screening | No | VOF screening | Fitmap correction |
| Q-score filtering | No | No | Yes | Yes |
| Unmodeled-density filtering | Manual/model screening | No | VOF | 3D ratio |
| Sequence–structure alignment | No | Yes | No | Yes |
| Per-entry removal reasons | No | No | Partial | Yes |
| Fixed ML-ready corpus | No | Yes | Custom generation | Yes |
Cryo-Electron Microscopy
Cryo-electron microscopy reconstructs biological structures from images of frozen specimens, from cellular volumes to atomic-resolution molecular models.
See all 29 papers