4D-TEMPO: Differentiable 4D-STEM ptychography via fast Physics Optimizers
Organizations: Computer Science and UMIACS, University of Maryland, College Park · Materials Science and Engineering, University of Maryland, College Park
Abstract
Electron ptychography recovers atomic structure from 4D-STEM data: diffraction patterns at multiple positions of a scanned probe. Current codes reconstruct an image and read atoms off it, or fit atoms under an loss; neither gives per-atom uncertainty. We propose 4D-TEMPO, a differentiable physical renderer that fits the atoms, the microscope, and the detector directly to the recorded electron counts under a Poisson likelihood. The output is a list of atoms, each a Gaussian primitive with a 3D position, a thermal width, and a model-conditional error bar from the Fisher information. The Fisher information also conditions the solver, gives the dose (electrons per Å) each parameter needs, and separates calibration from imaging. On a simulated WSe bilayer, starting from the published reconstruction, fitted atoms cut in-plane error by a third (3.2 vs. 4.8 pm) and depth error by 40-61% against its 3D peaks. Depth comes from multiple scattering into the dark field, the faint counts outside the direct beam. The two layers, closer than the depth of field, separate; per-atom depth then follows the layer geometry. When the model is well specified, the bars are calibrated (69% of position coordinates within ). On experimental WSe, fitted atoms predict held-out patterns better than the three published codes we test. With free probe modes where atoms are incomplete, the same likelihood predicts best on all three held-out benchmarks, matches every WSe baseline in 5 min against up to 20.5 h, and passes torchslice at its released setting 180 sooner on PrScO. Per-atom error bars, planned doses, and fast reconstruction point to low-dose, real-time 3D imaging of beam-sensitive and dynamic materials.
Figures & tables
| dataset (kV, mrad) | source | use | (Å) | (Å) | step | e - per | , W / Se 2 (rad) | |
|---|---|---|---|---|---|---|---|---|
| WSe 2 twisted bilayer (80, 25) | [ 69 ] | main: held-out, bars | 1.68 | 67 | 0.26 | 0.15 | 0.89 / 1.02 | |
| WSe 2 twisted bilayer, published (80, 25) | [ 22 ] | vs. published recon. (§S15) | 1.68 | 67 | 0.26 | 0.15 | 0.89 / 1.02 | |
| simulated, same conditions | [ 22 ] | accuracy, coverage | 1.68 | 67 | 0.26 | 0.15 | 0.89 / 1.02 | |
| WSe 2 monolayer (80, 18.4) | unpubl. | warm start outside training range (§S9) | 2.26 | 123 | 0.13 | 0.05 | 0.64 / 0.74 | |
| WSe 2 bilayer (200, 7.5–22.5) | [ 46 ] | uncorrected lens (§S10) | 1.11–3.34 | 50–446 | 0.16–0.36 | 0.01–0.13 | 0.9– | 0.28–0.95 / 0.34–1.06 |
| PrScO 3 crystal (300, 21.4) | [ 6 ] | thick: held-out | 0.92 | 43 | 0.45 | 4.88 | – |
| model | loss | gap | time | swap | |
|---|---|---|---|---|---|
| WSe 2 twisted bilayer, 25 mrad | |||||
| ours, atoms, physical instrument | Poisson | 0.4496 | 0.005 | 42 h † | – |
| + calibrated detector ellipse | Poisson | 0.4312 | 0.005 | +24 h † | – |
| ours, 8 probe modes | Poisson | 0.4025 | 0.027 | 53 min | – |
| fold_slice (400 + 125 it.) | amp. | 0.4552 | 0.007 | 20.5 h | – |
| PtyRAD , sub-scan | amp. | 0.4735 | 0.012 | 22 min | – |
| method | in-plane | depth | priors | time |
|---|---|---|---|---|
| start: peaks on 2 planes | 4.9 | 49 / 46 | no | – |
| ours, atoms | 3.2 | 50 / 49 | no | 18 h |
| ours, atoms, 1 h budget | 3.5 | 45 / 44 | no | 0.9 h |
| published, peaks [ 22 ] | 4.8 | 84 / 127 | no | 1.9 h ‡ |
| published, refined [ 22 ] | 4.4 | 5.3 / 5.5 | yes | 1.9 h ‡ |
| quantity (fitted numbers) | status | source / transfer |
| Specimen : 1,702 atoms, 236 free (78 W, 158 Se) | ||
| in-plane positions of free atoms (472) | fit p | each scan |
| positions of the other 1,466 atoms | fixed | earlier fit, lattice |
| thermal widths , one per species (2) | fit | material |
| interlayer distance (1) | fit | each scan |
| species, occupancy 1, Se planes Å | fixed | lattice |
| WSe 2 bilayer | PrScO 3 | PZT | |
| e/px, dark field | 0.35 | 2.3 | 3.1 |
| 0.88 | 0.56 | 0.77 | |
| ours, Poisson | 0.4077 (0.028) | 0.4224 0.0006 (0.131) | 0.5981 0.0017 (0.151) |
| ours, L1 | 0.4657 0.0005 (0.023) | 0.4265 0.0002 (0.127) | 0.5826 0.0006 (0.137) |
| ours, amp. | 0.4618 | 0.4237 0.0001 (0.131) | – |
| ours, counts | 0.4733 | 0.4570 0.0003 (0.122) | – |
| method | what the paper reports | held out? | gap | ||
|---|---|---|---|---|---|
| WSe 2 twisted bilayer, 80 kV, 25 mrad | |||||
| fold_slice , 400 + 125 it. † | info. limit 0.31 Å; rms against simulated truth 4.5 pm in-plane [ 22 ] ; no data-fit value | no | 0.4479 | 0.4552 | 0.007 |
| PtyRAD , sub-scan | info. limit 30 pm (other data); SSIM; data error rejected as a metric [ 31 ] | no | 0.4615 | 0.4735 | 0.012 |
| PtyRAD , full field | as above | no | 0.4680 | 0.4817 | 0.014 |
| torchslice , L1, 50 ep. | L1 training loss; 2–4 pm s.d. over 25 unit cells (PZT) [ 11 ] | no | 1.098 | 1.105 | 0.007 |
| torchslice , L1, 500 ep. | as above | no | 0.631 | 0.688 | 0.058 |
| dataset | (Å) | (Å) | step | pixel | dose (e/Å 2 ) | e per | (Å) | W / Se 2 | ||
|---|---|---|---|---|---|---|---|---|---|---|
| in | in | (rad) | ||||||||
| WSe 2 bilayer, Nguyen 2024, 7.5 mrad | 3.34 | 446 | 0.16 | 0.039 | 0.01 | – | 4.1e+05 | 4.6e+06 | 525 | 0.28 / 0.34 |
| WSe 2 bilayer, Nguyen 2024, 10.3 mrad | 2.43 | 236 | 0.22 | 0.053 | 0.03 | 4.2e+05 | 2.5e+06 | 525 | 0.42 / 0.50 | |
| WSe 2 bilayer, Nguyen 2024, 22.5 mrad | 1.11 | 50 | 0.36 | 0.187 | 0.13 | – | 7.1e+05 | 8.8e+05 | 525 | 0.95 / 1.06 |
| WSe 2 twisted bilayer [ 69 ] , 25 mrad | 1.68 | 67 | 0.26 | 0.078 | 0.15 | 5.1e+05 | 1.4e+06 | – | 0.89 / 1.02 | |
| WSe 2 monolayer (unpublished), 18.4 mrad | 2.26 | 123 | 0.13 | – | 0.05 | – | 9.5e+05 | 4.9e+06 | – | 0.64 / 0.74 |
| Source | Size | Model | W | Se | W/Se | data gain | ||
|---|---|---|---|---|---|---|---|---|
| med/p90 | med/p90 | p90 | pm | no refit/refit | ||||
| 1. Shot noise | ||||||||
| Poisson counts | e/pattern | includes | 1 | 1 | 1 | sd 1.09/2.05 | – | – |
| 2–4. Detector, scan, specimen | ||||||||
| Gain | 151 140 ADU/e ( ) | fits (dose) | 0.01/ 0.01 | 0.01/ 0.01 | 0.01 | 0.01/ 0.01 | 49.9/ 0.01 | – |
| Flat background | e/px | fits | 0.01/ 0.01 | 0.01/ 0.01 | 0.01 | 0.01/ 0.01 | 2.14/ 0.01 | – |
| method | priors | in-plane (pm) | W | Se | depth W / Se (pm) |
|---|---|---|---|---|---|
| published, 2-D, projected phase | none | 7.0 [3.0] | 7.1 | 7.3 | – |
| published, 2-D, max-phase image | none | 5.5 [3.0] | 5.9 | 5.3 | – |
| published, 3-D peaks | none | 4.8 [3.1] | 3.9 | 5.2 | 84 / 127 |
| refined coordinates | bond, interlayer | 4.3 [2.9] | 3.7 | 4.5 | 5.3 / 5.5 |
| ours, from published peaks | none | 3.2 | 2.8 | 3.4 | 50 / 49 |
| ours, true layer geometry (§S6) | none | 3.1 | 2.8 | 3.3 | 18.8 / 15.5 |
| dataset | all | W | Se 2 | interior | diff. / sd | ||
|---|---|---|---|---|---|---|---|
| main, 25 mrad | 127 | 13.7 | 0.082 | 14.1 | 12.2 | 9.0 | 10.4 |
| public, 10.3 mrad | 445 | 19.2 | 0.079 | 17.6 | 19.3 | 14.0 | – |
| 6 ∘ bilayer | 766 | 31.9 | 0.19 | 33.3 | 32.6 | 31.6 | – |
| 6 ∘ bilayer, ideal-lattice start | 1,033 | 17.9 | 0.11 | 17.1 | – |