Benchmarking Machine Learning for Chaotic Three-Body Dynamics in Astrophysics
Organizations: École Polytechnique, IP Paris · CITA, University of Toronto · Google DeepMind · LIX, CNRS, École Polytechnique, IP Paris
Abstract
Machine learning (ML) and generative AI have recently made impressive progress in understanding physics. We explore a hierarchy of prediction problems stemming from chaotic three-body systems and benchmark existing ML approaches, including general-purpose prediction algorithms and those that build on top of foundational physics models. We first simulate the trajectories of triple systems near the critical boundary of stable/unstable initial conditions. We then design a hierarchy of learning tasks, focusing mainly on two broad problems: stability prediction and trajectory prediction. The former classifies whether a given triple system has diverged or not up to a specific horizon, while the latter predicts the exact trajectory, both given suitable input features. We benchmark a range of ML approaches, from simple regression methods to state-of-the-art foundational physics models, across these tasks, and compare them against astrophysical methods such as the Mardling-Aarseth stability criterion. Our experiments reveal a clear order of task difficulties in chaotic three-body dynamics: early dynamics enable strong stability prediction, while precise long-horizon trajectory forecasting remains difficult. Code and dataset are publicly available at https://github.com/Simonwan03/ThreeBodyMLBench.
Figures & tables
| Prediction setting | Representative method | Key result | Main finding |
|---|---|---|---|
| Initial-condition stability | XGBoost | Binary: AUROC , Fraction: MAE | Limited predictability |
| Prefix-based stability | Dynamical summaries + XGBoost | Binary: AUROC , Fraction: MAE | Early dynamics improve predictability |
| Multi-horizon survival | Chronos-2 + direct pooling | MAE , IBS | Strong survival estimation |
| Trajectory forecasting | Chronos-2 (zero-shot) | MAE | Competitive short-term forecasting |
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Nominal | Perturbed | ||
|---|---|---|---|
| 1 | 17 | ||
| 2 | 18 | ||
| 3 | 19 | ||
| 4 | 20 | ||
| 5 | 21 | ||
| 6 | 22 |
| Key | Physical meaning |
|---|---|
| aratios | Ratio of the inner to outer semimajor axes, , characterizing the hierarchical structure of the system. |
| mratios | Two mass ratios, and . The former describes the mass ratio of the inner binary, while the latter measures the mass of the outer body relative to the total mass of the inner binary. |
| eccs | Eccentricities of the inner and outer orbits, . |
| incs | Inclinations of the inner and outer orbits with respect to the reference plane, . These are not the mutual inclination; the mutual inclination must be computed from the angular-momentum directions of the two orbits. |
| APs | Arguments of periapsis of the inner and outer orbits, . |
| LANs | Longitudes of ascending node of the inner and outer orbits, . |
| Key | Physical meaning |
|---|---|
| times_complete | Time grid for each trajectory. The dataset spans from to initial outer-orbital periods. |
| aratios_complete | Time evolution of the instantaneous osculating semimajor-axis ratio, . |
| mratios_complete | Time series of the mass ratios and . For point-mass integrations without mass transfer, these quantities remain constant. |
| eccs_complete | Instantaneous osculating eccentricities of the inner and outer orbits, . |
| incs_complete | Instantaneous inclinations of the inner and outer orbits with respect to the reference plane, . |
| APs_complete | Instantaneous arguments of periapsis of the inner and outer orbits, . |
| Statistic | Definition |
|---|---|
| Mean | |
| Standard deviation | |
| Minimum | |
| Maximum | |
| Range | |
| First value |
| Split | Ensembles | Individuals |
|---|---|---|
| Train | 3,498 | 111,936 |
| Validation | 750 | 24,000 |
| Test | 750 | 24,000 |
| Split | Ensembles | Individuals |
|---|---|---|
| Train | 2,692 | 44,988 |
| Validation | 577 | 9,961 |
| Test | 578 | 9,648 |
| Total | 3,847 | 64,597 |
| Method | Accuracy | Precision | Recall | Stable F1 | AUROC | AUPRC |
|---|---|---|---|---|---|---|
| Mardling–Aarseth criterion [ 22 ] | 0.6187 | 0.2875 | 0.9496 | 0.4414 | 0.7529 | 0.2810 |
| Vynatheya et al. criterion [ 37 ] | 0.6600 | 0.3046 | 0.8908 | 0.4540 | 0.7536 | 0.2887 |
| Vynatheya et al. MLP classifier [ 37 , 38 ] | 0.7267 | 0.3547 | 0.8824 | 0.5060 | 0.9086 | 0.7184 |
| MLP | 0.7507 | 0.3629 | 0.7563 | 0.4905 | 0.8416 | 0.5139 |
| XGBoost | 0.8053 | 0.4308 | 0.7059 | 0.5350 | 0.8525 | 0.5089 |
| Method | MSE | RMSE | MAE | |
|---|---|---|---|---|
| Mardling-Aarseth criterion [ 22 ] | 0.2488 | 0.4988 | 0.3073 | -0.5653 |
| Vynatheya et al. criterion [ 37 ] | 0.2268 | 0.4763 | 0.2853 | -0.4269 |
| Vynatheya et al. MLP classifier [ 37 , 38 ] | 0.1056 | 0.3249 | 0.2176 | 0.3359 |
| MLP | 0.0901 | 0.3001 | 0.2247 | 0.4335 |
| XGBoost | 0.0884 | 0.2972 | 0.2330 | 0.4442 |
| Individual-level classification | Ensemble-level prediction | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Prefix length | Representation | Model | Acc. | Stable F1 | AUROC | AUPRC | MAE | MSE | |
| 96 | PANDA encoder | Linear | 0.8640 | 0.8070 | 0.9467 | 0.8911 | 0.1031 | 0.0256 | 0.8390 |
| Chronos-2 encoder | Linear | 0.8623 | 0.8067 | 0.9531 | 0.9083 | 0.1112 | 0.0281 | 0.8230 | |
| Dynamical summaries | XGBoost | 0.9000 | 0.8404 | 0.9649 | 0.9319 | 0.0703 | 0.0153 | 0.9040 | |
| orbital15 | SVM | 0.7493 | 0.5800 | 0.8135 | 0.6166 | 0.2373 | 0.0927 | 0.4165 | |
| orbital15 | CNN | 0.7937 | 0.6092 | 0.8888 | 0.7526 | 0.1759 | 0.0786 | 0.5056 | |
| At-risk cohort | Representation | Ensembles | MAE | MSE |
| Initial conditions | 257 | 0.3432 | 0.1551 | |
| Dynamical summary, prefix 96 | 257 | 0.1103 | 0.0269 | |
| Initial conditions | 174 | 0.3985 | 0.1940 | |
| Dynamical summaries, prefix 96 | 174 | 0.0897 | 0.0187 | |
| Dynamical summaries, prefix 512 | 174 | 0.0520 | 0.0079 |
| At-risk cohort | Prefix | Representation | Predictor | Individuals | Stable % | Acc. | Stable F1 | AUROC | AUPRC |
|---|---|---|---|---|---|---|---|---|---|
| 96 | PANDA | Linear probe | 14,192 | 54.04 | 0.7709 | 0.8075 | 0.8716 | 0.8923 | |
| Chronos-2 | Linear probe | 0.7672 | 0.8067 | 0.8848 | 0.9088 | ||||
| Dynamical summaries | XGBoost | 0.8309 | 0.8404 | 0.9123 | 0.9320 | ||||
| 96 | PANDA | Linear probe | 10,189 | 75.27 | 0.7877 | 0.8631 | 0.8172 | 0.9301 | |
| Chronos-2 | Linear probe | 0.7885 | 0.8649 | 0.8341 | 0.9398 | ||||
| Dynamical summaries | XGBoost | 0.7999 | 0.8611 | 0.8655 | 0.9537 |
| Method | Aggregation | Overall MAE | Final MAE | IBS |
|---|---|---|---|---|
| Baselines | ||||
| Train-mean | – | 0.3099 | 0.3510 | 0.1315 |
| Raw orbital15 CNN | – | 0.0910 | 0.1728 | 0.0752 |
| Raw orbital15 Dynamical summaries | – | 0.0670 | 0.1475 | 0.0632 |
| Raw orbital15 TCN | – | 0.0662 | 0.1564 | 0.0606 |
| Pretrained time-series representations | ||||
| Model | Setting | MSE (z) | MAE (z) | Pearson | Unit-circle Error |
|---|---|---|---|---|---|
| Chronos-2 [ 1 ] | Zero-shot | 0.2331 | 0.2282 | 0.8580 | 0.2265 |
| PANDA [ 16 ] | Zero-shot | 0.2217 | 0.2371 | 0.8625 | 0.2862 |
| iTransformer [ 21 ] | Supervised | 0.2123 | 0.2429 | 0.8673 | 0.3152 |
| TOTO2 [ 15 ] | Zero-shot | 0.2262 | 0.2303 | 0.8601 | 0.2310 |
| TiRex2 [ 26 ] | Zero-shot | 0.2422 | 0.2408 | 0.8508 | 0.2277 |
Chaotic Dynamical Systems
Chaotic dynamical systems study how sensitive, nonlinear evolution produces complex trajectories, attractors, and statistical patterns.
See all 50 papers