Molecular dynamics predicts molecular motion from atomic forces and trajectory integration. Density functional theory (DFT) provides first-principles forces, but its O(N3) cost limits simulations to thousands of atoms; empirical force fields scale to millions but miss bond breaking and charge transfer. Equivariant message-passing Transformers learn the ab initio potential at linear cost. Stacked layers propagate information beyond a single cutoff, giving four layers a receptive field up to 23Å. At this frontier, scale is limited by communication: layers exchange boundary features across devices, force backpropagation repeats the exchange, and moving atoms change neighbor graphs and tensor shapes. We present Tessera, a distributed runtime that fixes workload shapes between rebuilds. It sorts atoms along a space-filling curve into equal-count shards, ensuring identical atom counts and tensor shapes per rank. Peer sets scale as K0.5, not K. A conservative neighbor skin preserves topology across updates, enabling each rebuild interval to replay as one CUDA Graph. Fused kernels, three-level neighbor search, and overlapped All-to-All complete it. On 512 GPUs, Tessera sustains 30.7M atoms at 89.7% weak-scaling efficiency (77--90% across three per-GPU loads); per-step time rises 1.11--1.29x as machine and problem grow 16x. Graph reuse cuts full-step time 1.87×. Across an 8× device increase, neighbor-search cost grows 1.7×, versus 3.0× for Morton order. In a live 10.24M-atom salt solution, NVE energy drift is 0.42μeV/atom/ps over 10 ps, and structure matches a serial reference. A 10.06M-atom Norovirus capsid sustains 100 ps of NVT at ≈8 ps/day on 256 A100 GPUs. Over 200,000 steps, it has no broken bonds or anomalously close pairs and uses three compiled variants, with no graph breaks or eager fallbacks.
Figures & tables
Figure 1: Overview of the algorithm–system co-design. a , Equal-count cut of a 1D space-filling-curve order (Hilbert shown), giving invariant tensor shapes and a peer set growing as roughly K0.5 rather than K . b , Per-GPU E2Former V2 stack: sparse neighbor gather, Clebsch–Gordan tensor products, equivariant attention over irreps up to lmax , stacked to a 23 Å receptive field. c , Three-level neighbor filter—AABB, balanced chunking, exact FAISS search, conservative throughout—with cross-step cache. d , Communication and compute on separate streams, the overlapped All-to-All hidden under local work in both passes. e , Per-rank on-GPU integration (Velocity–Verlet, autograd forces Fi=−∂E/∂ri ), synchronizing a fixed handful of scalars, re-partitioning every 100 steps. f , Measured weak scaling against the ideal linear reference (Fig. 12 ).
Figure 2: One partitioning rule, two curves: equal atom counts either way, different shard geometry. An 8×8 grid cut into four shards of equal atom count (cell counts differ, since density does). (a) Morton’s power-of-two jumps attach a remote sliver to a shard (orange arrow): rank 2 fragments into two pieces whose bounding box—what the Level-1 filter tests—encloses 23 cells it does not own. (b) Hilbert’s consecutive indices stay adjacent, keeping shards connected and boxes tight (mean AABB/owned-cell ratio: 1.42× Morton vs. 1.06× Hilbert). Shown in 2D for legibility (the real partition is 3D); consequences at scale in Fig. 11 .
Figure 3: Multi-level neighbor search. (a) Per-device AABBs and a K×K overlap test bound each device’s active peers—the sparse peer set the communication layer reuses. (b) Recursive bisection into compact chunks Qi , each screened by its own tight box, cuts candidates from C to Ci≪C (detail: Fig. 4 ). (c) The exact cutoff search then scores only block-diagonal Qi – Ci pairs, at ∑iQiCi rather than QlocCcand .
Figure 4: Level 2 in detail: why chunking shrinks the candidate set. The mechanism behind card b of Fig. 3 . (a–c) Without chunking: a single device-level AABB is stretched by a few spatial outliers, so the retained candidate set C is large and the exact search scores all Q×C pairs ( ∝QlocCcand ). (d–f) With chunking: recursive bisection along the longest axis produces tight per-chunk AABBs that isolate those outliers, so only the block-diagonal Qi – Ci pairs are scored and the cost drops to ∑iQiCi with Ci≪Ccand . Same atoms in both rows.
Figure 5: Communication-plan reuse eliminates per-round handshake overhead. Every simulation step issues R variable-length All-to-All ( all2allv ) rounds—one per Transformer block—among the same sparse set of ≤k spatially adjacent peers (top). a , A naive implementation renegotiates transfer lengths, offsets, and buffer addresses (handshake, orange) before every round, so control-plane activity recurs each round. b , Because the blocks at one cutoff share a neighbor topology, we pay the handshake once per cutoff to build a cached communication plan recording the participating peers, send/receive lengths, offsets, index mappings, and pre-registered buffer addresses. c , All subsequent rounds reuse the cached plan and perform pure P2P payload transfer (blue) with no further negotiation. Across the Rr blocks at cutoff r the exposed cost drops from Rr×(handshake+payload) to handshake+Rr×payload , pushing the communication path into a bandwidth-bound regime; the production stack has two cutoffs and so pays two handshakes per step, not one and not R (Section 3.4 ).
Figure 6: Sender-side projection with fused SKV communication. a , In the conventional scheme the Key, source-embedding, and Value projections are repeated on the receiver, so three separate collective launches fire; two of them (Key, Value) must still carry the full equivariant intermediate hi∈R(lmax+1)2×C , since only Value is aggregated equivariant and the receiver cannot recover a sender-side scalar Key without it, while the source embedding is already a C -wide scalar and was never large. b , In our scheme, the Key ( H×dk ), source-embedding ( dembed ), and Value ( (lmax+1)2×C ) tensors are projected locally on the sender and packed into a single peer-specific contiguous buffer, exchanged in one fused all2allv ; the receiver reconstructs structured tensor views from deterministic offsets and consumes them directly. This lowers the transmitted volume by ≈26% , realized entirely on the Key (Section 3.4 ), and reduces the number of collective launches from three to one.
Figure 7: Pooled buffer management with one-time registration. a , Allocating buffers and performing Memory-Region (MR) registration on the critical path of every communication round incurs repeated setup cost and leaves scattered, short-lived buffers that fragment device memory. b , We instead allocate a single contiguous device-memory pool once and pre-register it as a pinned MR (together with symmetric-memory registration), entirely outside the steady-state path. c , During communication, buffers are borrowed from and returned to the pool with no further allocation or registration, removing dynamic-registration latency from every transfer and leaving only the useful payload on the critical path.
Figure 8: Forward-pass communication–computation overlap. a , The attention operator is split into a local branch (source atoms already resident on the GPU) and a remote branch; the remote feature exchange is launched asynchronously on the communication stream while the local branch executes on the compute stream, and the remote branch waits on the completion event only immediately before it consumes the received data; the panel’s cost model gives the resulting exposed cost. b , Launch order is decisive: if the heavy local kernels are enqueued first, the exchange is issued late behind them and the overlap window collapses. c , Issuing the asynchronous exchange first gets the communication kernels resident before the local branch fills the device, preserving a large overlap window.
Figure 9: Autograd-compatible overlap via paired pre / post operators. a , A naive forward-only overlap works in the forward pass, where pre launches the remote exchange before local computation. In backward, however, communication is not relocated early enough: the remote exchange, remote-gradient computation, and local-gradient computation appear as a serial tail. b , With paired operators, pre launches communication in the forward pass, while post is a forward no-op whose backward method launches the reverse remote exchange. Because post.backward is reached early under reverse traversal, the exchange goes out as soon as its payload—the gradient with respect to the received remote features—exists, and overlaps with unblocked local-gradient computation. Writing the forward halo gather as xhalo=Gxowned makes the bar after the wait explicit: the owner applies G⊤ , accumulating ∇xowned+=G⊤∇xhalo over every rank and periodic image that requested the atom.
Figure 10: Conventional versus decentralized molecular-dynamics engine. a , A standard distributed loop is rank-0 centric: each step scatters atom positions from rank 0 to all ranks ( 1→K ), runs distributed inference, reduces all forces back to rank 0 ( K→1 ), and lets rank 0 integrate and re-encode. The two collective hot spots through rank 0 form a communication bottleneck, and the serial curve encoding and integration on rank 0 form a compute bottleneck—both worsening with system size. b , Our engine keeps every operation on the GPU. After the initial scatter, each rank integrates its own atoms locally using the globally computed forces and emits only a fixed-size synchronization signal, eliminating the per-step K→1 reduction and the central integration. A global gather and re-partition are performed only every 100 steps ( ≈50 fs at the 0.5 fs step), which is safe because atoms move far less than the neighbor-search margin over that interval (Section 3.2 ). The scorecard contrasts the two engines per step: the decentralized design removes the O(N) collective bottlenecks, the rank-0 serial-compute bottleneck, and the per-step CPU–GPU transfers entirely.
Configuration
GPUs
Atoms/GPU
TF32
Recomp.
Cache
Cluster
Single-GPU ablation
1
10 k
off/on
off/on
n/s
H100
Neighbor, 25 k
64 – 512
25 k
on
n/s
1 / 10
H100
Neighbor, 50 k
64 – 512
50 k
on
n/s
1 / 10
H100
Overlap A/B
32 – 512
25 k
on
off
10
H100
Weak scaling, 10 k
32 – 512
10 k
on
off
10
H100
Weak scaling, 25 k
32 – 512
25 k
on
off
10
H100
Table 1: Run configurations. “n/s” marks a value the experimental text does not state. TF32 is the mixed-precision mode of Section 3.1 ; recomputation is activation recomputation; the cache interval is the rebuild period in steps. The H100 cluster is the 64-node, 8-GPU-per-node NVLink/InfiniBand system above, the A100 cluster the 256 -GPU PCIe system; the two are never mixed within a single comparison.
Config
Latency(ms)
Mem(MiB)
∣ΔE∣ (eV)
max∣ΔF∣
base
363.4
67918
0
0
+ T
234.9
21078
3.05×10−5
2.38×10−7
+ T+C
141.1
15370
3.05×10−5
1.01×10−6
+ T+C+F
95.7
15370
8.93×10−1
1.43×10−3
+ T+C+R
165.0
7368
3.05×10−5
1.19×10−6
+ T+C+F+R
114.4
7368
8.96×10−1
1.37×10−3
Table 2: Ablation on a 10k-atom toy system on H100. Config abbreviates T=Triton, C= torch.compile , F=TF32, R=activation recomputation, applied cumulatively left to right. Precision metrics are relative to the base (no T/C/F/R).
Figure 11: Weak-scaling ablation of the distributed neighbor search on H100 , rebuilding every step; color is the Level-1 exchange scheme, line style the partition curve. (a,b) Neighbor-pipeline latency at 25 k/ 50 k atoms per GPU—Morton bends upward ( 2.8 – 2.9× across the two panels), Hilbert rises gently ( 1.6 – 1.7× ); rank-box (default) is bold, allgather (within 11% of it throughout) is lightened for clutter, spatial-bin for being the slowest scheme on Hilbert instead, as detailed next. (c) Source ranks requested by the heaviest rank, all schemes at equal weight (all-gather =K by construction, drawn once for (c,d)). (d) Candidate population: spatial-bin prunes hardest— 2.6 – 11.3× below all-gather, 2.2 – 3.6× below rank-box—yet is slowest on Hilbert at every scale: volume alone does not predict cost.
Figure 12: H100 results across varying GPU counts. (a,b) Neighbor-graph cache : (a) per-step cost at 50 k atoms/GPU—cache hits (orange) an order of magnitude cheaper than rebuilds; (b) cumulative reduction composing curve + exchange + cache at 512 GPUs, the cache bar an upper estimate (Appendix D ). (c,d) Overlap on the full step at 25 k atoms/GPU: (c) per-step time on vs. off, gap widening with scale; (d) reduction by step type, larger on cache hits. (e-g) End-to-end weak scaling , 32–512 GPUs, three per-GPU loads: (e) step time, rising 1.11 – 1.29× over 16× ; (f) parallel efficiency vs. 32 GPUs; (g) aggregate throughput, each load against its own 32 -GPU-anchored linear reference (dashed)—a curve’s gap to its line is (f)’s efficiency. Heaviest load: 30.7 M atoms on 512 GPUs. ∗ At 60 k atoms/GPU (e-g), recomputation is on since the load exceeds the 80 GB HBM budget otherwise; 10 k/ 25 k run without it.
Partition
Exchange
ms
Reduc.
Growth
Morton
a-g (base)
393.6
1.0×
3.0×
Hilbert
a-g
170.8
2.3×
1.6×
Hilbert
r-b
176.3
2.2×
1.6×
Hilbert
r-b + cache
25.1
15.7×
1.7×
Table 3: Composing the three neighbor-search mechanisms (a-g=all-gather, r-b=rank-box) at 512 GPUs, 50 k atoms/GPU; each row adds one mechanism to the row above. “Growth” is the 512-/64-GPU cost ratio, so 1.0 is ideal weak scaling. Cached rows are a composed interval- 10 amortized cost, not a timed interval- 10 run; the composition and its known bias are stated below. Neighbor pipeline only. Rank-box costs 3% against all-gather at this scale rather than saving: the two differ by under 9% at every Hilbert point in the observed runs, a margin we do not treat as separable given the single run per point (Section 4.3 ), and rank-box is the default for its small, fixed-size metadata and reusable peer set, not for latency. At 25 k atoms/GPU the baseline grows 3.2× over the same range.
Configuration
GPUs
Rebuild (ms)
Cache hit (ms)
thit/trebuild
Amortized (ms)
Amortized/rebuild
Morton + spatial-bin
64
162.8
4.8
0.029
20.6
0.126
128
184.2
5.6
0.031
23.5
0.128
256
204.0
8.0
0.039
27.6
0.135
512
316.2
14.4
0.046
44.6
0.141
Hilbert + multi-box
64
91.5
4.4
0.048
13.1
0.143
128
122.4
5.1
0.041
16.8
0.137
Table 4: Neighbor-stage cost measured during full-inference-mode runs at 20 k atoms per GPU (the timer is the neighbor pipeline, not end-to-end step wall time), the largest per-GPU configuration in this cache-validation sweep that runs without activation recomputation. These runs execute the complete model—forward, autograd forces, torch.compile , TF32, and remote-communication overlap—over a 20-step window after step 0 holding exactly 2 interval- 10 rebuilds and 18 hits, so the amortized column is measured, not composed—though that 2 : 18 split makes it equal to (Rebuild+9×Cache hit)/10 . Each row’s candidate-exchange scheme is tuned for its own curve, so the two rows are not a controlled exchange-scheme comparison; that remains the neighbor-only matrix of Fig. 11 . The load-bearing column is thit/trebuild , which licenses composing the amortized cost for Table 3 ; the last column is the resulting 12.6 – 16.0% of the rebuild-every-step cost.
Figure 13: Structural validation of the distributed trajectory. K–O, Cl–O, and O–O radial distribution functions, 256 -GPU/ 10.24 M-atom run (solid) vs. 14 k-atom single-GPU serial reference (dashed, Section 4.6 ’s protocol at single-GPU scale), same concentration and temperature. First-shell peaks match the 0.08 Å bin width in all three; peak heights agree to about 2% (Cl–O, the widest gap) and 0.4% for O–O, the bulk solvent structure.
Figure 14: Strong scaling on a fixed 1.28 M-atom KCl-in-water system , NVE, 2,000 steps. (a) Step time, activation recomputation on (forward-pass detail and recomputation off are in Table 5 , Appendix D ). (b) Efficiency (K)=T(K)⋅KT(64)⋅64×100% , referenced to 64 GPUs, not 32 : that point sits near recomputation’s own memory ceiling, allocator garbage collection inflating its step time. (c) Repartition (rank- 0 sort + broadcast) and rebuild cost per event. Amortized, repartition stays under 0.1% of step time; neighbor-cache management overall (rebuild + hit handling) runs 2.2→6.8% from 64 – 256 GPUs— 4.6 points against (b)’s 36 -point drop. (d) Per-atom forward time (utilization proxy, not profiled)—bottoms at 64 GPUs’ 20 k atoms/GPU, rising both ways, consistent with allocator pressure at 32 and too small a batch at 256 .
GPUs
Step, on
Step, off
Fwd, hit
Fwd, rebuild
32
3915.1
OOM
3766.7
3776.0
64
1404.5
OOM
1369.7
1366.4
96
1092.0
888.6
1072.2
1070.9
128
829.3
728.5
827.9
823.6
160
699.4
615.1
680.0
711.1
192
706.2
598.2
674.8
655.8
Table 5: Strong-scaling step time with recomputation on/off, and the forward-pass hit/rebuild split (recomputation on).
KCl ( 10.24 M)
Virus ( 10.06 M)
Diff.
Atoms per GPU
40.0 k
39.3 k
−1.8%
Step time
4552.3 ms
5212.6 ms
+14.5%
forward, cache hit
3650.8 ms
4258.0 ms
+16.6%
forward, rebuild
3369.0 ms
4030.5 ms
+19.6%
rebuild overhead
334.9 ms
367.4 ms
+9.7%
hit overhead
33.8 ms
33.7 ms
−0.3%
Table 6: Per-step cost under matched hyperparameters on the same A100 cluster, near-uniform KCl vs. heterogeneous Norovirus capsid, over steps 3000 – 4000 . The capsid holds fewer atoms per GPU, so the penalty is not a load artifact. Rows compose as Step ≈ hit rate × (fwd,hit + hit ovh.) + ( 1− hit rate) × (fwd,rebuild + rebuild ovh.) + residual.
System
Model class
Per-layer exch.
Max GPUs
Max atoms
Exact forces
Live-graph reuse
Static-shape capture
DPMD [ 17 , 50 ]
strictly local
no
—
100M
yes
—
—
LAMMPS [ 32 , 41 ]
strictly local
no
—
—
yes
yes
no
Allegro scale-out [ 26 ]
strictly local
no
—
—
yes
—
—
SevenNet [ 29 ]
message passing
yes
32
—
yes
—
—
Xia et al. [ 48 ]
message passing
yes
8
108 (CPU)
—
—
—
NequIP rebuild [ 38 ]
message passing
—
—
—
—
—
—
Table 7: Where the design sits against systems cited in this paper, with the cells confined to what those citations and our own results establish. “Model class” separates potentials whose energy depends on one cutoff ball from those that propagate features across layers. “Per-layer exch.” asks whether a distributed step synchronizes remote features once per layer, as propagation requires, rather than once per step as a ghost-atom halo does. Exact forces means forces are the gradient of the evaluated energy rather than a separately regressed output. Live-graph reuse means the neighbor topology, or the exchange plan built from it, survives across steps of a moving trajectory. Static-shape capture means a step is replayed under a captured graph whose allocations and launch geometry are fixed. An em dash marks an item this paper could not establish from the cited work; no cell is a ranking, the systems differ in task, model, and hardware, so the columns are not additive.
Term
Grows in
What the design does
Per-GPU edges
local density
O(1) in K at fixed atoms/GPU
Peer count
geometry, 6→19 over 64 – 512 GPUs
sublinear in K , roughly K0.5
K×K request matrix
K2 nominally
sparse rows, coarse AABBs
Metadata all-gather
K
small fixed descriptor ( <2 KiB)
World-collective latency
logK
constant payload per step
Rank-0 O(NlogN) sort
N
amortized over 100 steps
Table 8: What still grows per step, and in what. Weak scaling holds per-GPU atom count fixed, so the question is which terms grow while the local forward–backward work does not. The last column gives what the design does about each term; none is removed outright. The table scopes the extension argument rather than supporting it.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Model / Backbone setting
Symbol
Value
Backbone
—
e2former (E2Former V2)
# layers
R
num_layers=4 ( 3 short-range +1 long-range)
Cutoff radius
—
short-range 5 Å ( ×3 layers), long-range 8 Å ( ×1 )
Clustering
—
with_cluster=False
Node embedding irreps
C,lmax
"256x0e+256x1e+256x2e" ( C=256 , lmax=2 )
Head irreps
—
"2x0e+2x1e+2x2e"
Appendix
Table 9: E2Former V2 backbone hyperparameters used for all runs in this paper. Keys are configuration fields under backbone_config ; the Symbol column gives the notation used for each in this appendix ( C channels, H attention heads, lmax maximum degree, dk scalar head dimension, R Transformer blocks).
Figure 15: Single-GPU attention-kernel benchmark on H100. Latency (a,c) and effective throughput (b,d) of Sparse-Triton against a vanilla sparse baseline and two dense SDPA variants, at D=8 (a,b) and D=64 (c,d); the dashed red line marks the device’s dense FP32 peak. Dense SDPA is a hardware reference point, not a deployable alternative for this sparse workload: it computes the full N2 attention matrix, which is why its throughput is high while its latency explodes with N . The takeaway is the comparison within sparse implementations—Sparse-Triton dominates the vanilla baseline at practical sizes, while SDPA-Mask collapses because it still materializes dense score matrices. Exact values in Table 10 .
N
head
D
Sparse-Triton
Sparse-Vanilla
SDPA-Mask
SDPA-Dense
512
16
8
0.097 (0.161)
0.105 (0.148)
0.164 (0.095)
2.178 (0.071)
2048
16
8
0.192 (0.325)
0.141 (0.440)
0.088 (0.708)
4.028 (0.617)
8192
16
8
0.605 (0.412)
0.157 (1.583)
0.026 (9.630)
5.419 (7.331)
32768
16
8
0.692 (1.439)
0.163 (6.130)
0.007 (150.934)
5.783 (109.918)
512
16
64
0.648 (0.166)
0.193 (0.557)
0.974 (0.110)
14.045 (0.078)
2048
16
64
1.792 (0.240)
0.220 (1.955)
0.528 (0.813)
25.491 (0.687)
Appendix
Table 10: Exact throughput and latency of the attention kernels of Fig. 15 . Each cell reports effective throughput in TFLOPS, with kernel latency in milliseconds in parentheses. The dense SDPA result is a hardware-oriented reference point rather than a deployable alternative for the sparse molecular workload. The sweep runs at 16 heads to isolate the N and D axes; production uses 128 (Appendix A ), so these are kernel-level comparisons rather than production-shape measurements.
Figure 16: TF32 on/off structure over the same 5 ps window. All six radial distribution functions of the 1.28 M-atom KCl-in-water run of Fig. 14 , TF32 off (dashed) against TF32 on (solid), 101 frames each. (a–c) Ion–water and water–water structure is indistinguishable. (d–f) The ion–ion pairs differ, but the two gray curves are the two halves of the TF32-off run by itself , 51 frames each and no precision change involved: they set a noise floor of 10.3% , 14.1% and 12.5% on these first peaks, which is the scale of the on/off gap. With 1181 ions of each sign and ion decorrelation slower than the window, the effective sample behind a K–K peak is tens of pairs, not the 103 the frame count suggests.
Machine-learned interatomic potentials extend atomistic simulations beyond the length- and timescales accessible to electronic-structure methods. However, the computational cost of equivariant architectures limits the local correlations they can represent in practice and therefore their achievable accuracy. Here we introduce FlashCart, which makes higher-order correlations affordable by combining generated GPU kernels with an architecture that recursively builds equivariant features and compresses them to a fixed width at each step. We express tensor products in independent Cartesian components and symbolically simplify them and their derivatives, producing fused kernels that often outperform optimized spherical counterparts. We then show that increasing correlation order improves accuracy more efficiently than increasing width, depth, or tensor rank. On SPICE-MACE-OFF, FlashCart models advance the measured accuracy-efficiency frontier: a model with 5.6 million parameters achieves lower energy and force errors and 10× faster inference than a transformer with 189 million parameters.
Viktor Zaverkin, Payman Goodarzi, Sergey V. Sukhomlinov +4
INM – Leibniz Institute for New Materials · Saarland University · German Research Center for Artificial Intelligence (DFKI) +2
Equivariant machine learning interatomic potentials (MLIPs) promise atomistic simulations with accuracy approaching that of quantum-mechanical calculations. However, the high computational cost of their core Clebsch--Gordan tensor product (CGTP) convolutions severely limits model scalability and efficiency. In this paper, we rethink the parallel execution of CGTP convolutions and introduce LunaTP. Built on node-centric execution, LunaTP exploits path-group parallelism and multi-level reuse to accelerate CGTP convolutions and higher-order derivative computation while preserving the underlying mathematical formulation. On an NVIDIA H100 GPU, LunaTP achieves kernel speedups of up to 250.0× over e3nn and 17.1× over OpenEquivariance. It also delivers end-to-end training throughput gains of up to 28.7× over e3nn and 3.11× over OpenEquivariance. Finally, LunaTP demonstrates scalability and robust performance across models, workload scales, and GPUs, delivering substantial acceleration for training and atomistic simulation with high-accuracy equivariant MLIPs.
Yuanchang Zhou, Mingzhen Li, Chen Wang +3
State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences
GROMACS is a de-facto standard for classical Molecular Dynamics (MD). The rise of AI-driven interatomic potentials that pursue near-quantum accuracy at MD throughput now poses a significant challenge: embedding neural-network inference into multi-GPU simulations retaining high-performance. In this work, we integrate the MLIP framework DeePMD-kit into GROMACS, enabling domain-decomposed, GPU-accelerated inference across multi-node systems. We extend the GROMACS NNPot interface with a DeePMD backend, and we introduce a domain decomposition layer decoupled from the main simulation. The inference is executed concurrently on all processes, with two MPI collectives used each step to broadcast coordinates and to aggregate and redistribute forces. We train an in-house DPA-1 model (1.6 M parameters) on a dataset of solvated protein fragments. We validate the implementation on a small protein system, then we benchmark the GROMACS-DeePMD integration with a 15,668 atom protein on NVIDIA A100 and AMD MI250x GPUs up to 32 devices. Strong-scaling efficiency reaches 66% at 16 devices and 40% at 32; weak-scaling efficiency is 80% to 16 devices and reaches 48% (MI250x) and 40% (A100) at 32 devices. Profiling with the ROCm System profiler shows that >90% of the wall time is spent in DeePMD inference, while MPI collectives contribute <10%, primarily since they act as a global synchronization point. The principal bottlenecks are the irreducible ghost-atom cost set by the cutoff radius, confirmed by a simple throughput model, and load imbalance across ranks. These results demonstrate that production MD with near ab initio fidelity is feasible at scale in GROMACS.
Luca Pennati, Andong Hu, Ivy Peng +2
KTH Royal Institute of Technology, Stockholm, Sweden