BHCast: Unlocking Black Hole Plasma Dynamics from a Single Blurry Image with Long-Term Forecasting

Renbo Tu; Ali SaraerToosi; Nicholas S. Conroy; Gennady Pekhimenko; Aviad Levis

cs.CVMar 24, 2026

BHCast: Unlocking Black Hole Plasma Dynamics from a Single Blurry Image with Long-Term Forecasting

Authors: Renbo Tu, Ali SaraerToosi, Nicholas S. Conroy, Gennady Pekhimenko, Aviad Levis

Organizations: University of Toronto · Vector Institute · University of Illinois at Urbana-Champaign · NVIDIA

Abstract

The Event Horizon Telescope (EHT) delivered the first image of a black hole by capturing the light from its surrounding accretion flow, revealing structure but not dynamics. Simulations of black hole accretion dynamics are essential for interpreting EHT images but costly to generate and impractical for inference. Motivated by this bottleneck, BHCast presents a framework for forecasting black hole plasma dynamics from a single, blurry snapshot, such as those captured by the EHT. At its core, BHCast is a neural model that transforms a static image into forecasted future frames, revealing the underlying dynamics hidden within one snapshot. With a multi-scale pyramid loss, we demonstrate how autoregressive forecasting can simultaneously super-resolve and evolve a blurry frame into a coherent, high-resolution movie that remains stable over long time horizons. From forecasted dynamics, we can then extract interpretable spatio-temporal features, such as pattern speed (rotation rate) and pitch angle. Finally, BHCast uses gradient-boosting trees to recover black hole properties from these plasma features, including the spin and viewing inclination angle. The separation between forecasting and inference provides modular flexibility, interpretability, and robust uncertainty quantification. We demonstrate the effectiveness of BHCast on simulations of two distinct black hole accretion systems, Sagittarius A* and M87*, by testing on simulated frames blurred to EHT resolution and real EHT images of M87*. Ultimately, our methodology establishes a scalable paradigm for solving inverse problems, demonstrating the potential of learned dynamics to unlock insights from resolution-limited scientific data.

Figures & tables

Figure 1: From a Single Blurry Frame to Black-Hole Dynamics. A single blurry image, reconstructed from Event Horizon Telescope measurements, is used to forecast a dynamic sequence of high-resolution frames. From this forecasted movie, features of accretion disk plasma can be extracted to infer the black hole’s physical properties. This sequential framework—forecasting, feature extraction, and parameter estimation—is robust, versatile, and fully interpretable, providing a new pathway for analyzing horizon-scale dynamics.

Figure 2: Training and inference pipelines of BHCast . We train the dynamics forecast model (U-Net) to predict the next frame and the physics inference model (XGBoost) to estimate black hole parameters. During inference, we autoregressively forecast with the U-Net to produce cylinder plots to extract plasma features, which are inputs to XGBoost for physics inference.

Figure 3: Loss ablation study. LPIPS across three losses:

\ell_{2}

only, multi-scale without mean-flux, and the full BHCast loss.

\ell_{2}

-only performs poorly at all times, while the multi-scale variant works briefly but becomes unstable. The full loss, combining multi-scale and mean-flux terms, yields stable long-term forecasts.

Figure 4: Super-Resolution Evaluation : Power Spectral Density (PSD) and log-scale visualizations of the input, prediction after 6 steps (30 $\,GMc^{-3}$ ), and prediction after 100 steps (500 $\,GMc^{-3}$ ) for (A) face-on GRMHD and (B) edge-on GRMHD. At input, BHCast lacks high-frequency details but recovers them by step 6 to match the ground truth PSD. Its forecasts also visually converge to the ground truth. The oracle optical flow baseline starts with the deconvolution result but shows limited super-resolution. The learning-based baseline is omitted due to divergence after a few steps.

Figure 5: Forecasting Fidelity Evaluation. Perceptual similarity (LPIPS; lower is better) between forecasts and the unblurred ground-truth simulation. BHCast ’s forecast remains visually close to the ground truth over long horizons, while baseline forecasts are on par with the EHT-resolution blurred input.

Plasma Feature	BHCast	ResNet Baseline
Pattern Speed ( $\Omega_{p}$ )	0.46 $\pm$ 0.05	0.64 $\pm$ 0.07
Pitch Angle ( $\Phi$ )	0.13 $\pm$ 0.01	0.14 $\pm$ 0.02
Asymmetry	0.30 $\pm$ 0.03	0.23 $\pm$ 0.02
Rotation Curve Slope	0.24 $\pm$ 0.03	0.25 $\pm$ 0.03

Table 1: Mean Absolute Error

\pm

Standard Error (

\downarrow

) of extracted plasma features, averaged across all test cases. Our BHCast model is compared against a ResNet baseline.

Figure 6: Pattern Speed Correlation . Extracted pattern speed from a 60-frame forecast shows a very strong correlation of 0.927 with the ground truth values. In addition, BHCast identifies separate clusters for clockwise and counter-clockwise rotation, showing significantly less bias than the baseline when estimating counterclockwise rotation.

Blurring Level	Model	Inclination Accuracy	Spin Accuracy
20 $\mu$ as (Baseline)	BHCast	56.41%	69.22%
20 $\mu$ as (Baseline)	ResNet	47.19%	67.66%
25 $\mu$ as	BHCast	56.72% ( +0.31 )	71.09% ( +1.87 )
25 $\mu$ as	ResNet	31.41% ( -15.78 )	54.53% ( -13.13 )
30 $\mu$ as	BHCast	53.44% ( -2.97 )	65.78% ( -3.44 )
30 $\mu$ as	ResNet	25.47% ( -21.72 )	44.37% ( -23.29 )

Table 2: Classification accuracy of black hole parameters, comparing BHCast to a ResNet baseline. The 20

\mu

as (EHT-like) resolution is the baseline. The 25

\mu

as and 30

\mu

as rows show the performance degradation (or stability) as blurring increases, with differences (

\Delta

) relative to the baseline.

Metric	Spin Model	Inclination Model
Validation Accuracy	0.94	0.87
Aleatoric Uncertainty	$0.15(\in[0,1])$	$0.42(\in[0,2])$
Epistemic Uncertainty	$\textbf{0.003}(\in[0,1])$	$\textbf{0.004}(\in[0,1]$ )
Mean Confidence	$0.96\pm 0.10$	$0.88\pm 0.15$

Table 3: An ensemble of XGBoost bootstrapped models achieves high accuracy with low epistemic uncertainty. This demonstrates that the extracted plasma features are effective and lead to stable, unanimous predictions.

Figure 7: Extrapolation on EHT M87* image . We use the April 6 EHT image as input and validate our forecasting ability by comparing the blurred forecast after three autoregressive steps with the April 11 EHT image. BHCast successfully resolves the correct ring structure (red circle) and predicts a counterclockwise rotation, consistent with the EHT reconstructions.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Parameters	FLOPs (G)
EDSR (Super-Resolution)	$1.37$ M	$6.87$
ConvLSTM (Dynamics)	$0.45$ M	$17.81$
EDSR + ConvLSTM (Combined)	$1.82$ M	$24.68$
BHCast (Ours - U-Net)	$31.04$ M	$8.12$

Appendix

Table 4: Comparison of models in terms of trainable parameters and floating-point operations (FLOPs).

Model	Param	Roll-out	Training	Inference
Name	Count(M)	FLOPs (G)	GPU Hrs	FPS
FNO	409.01	39.63	19.1	110.45
Diffusion	28.64	75340.88	146.5	17.91
U-Net	31.04	486.95	5.5	485.20

Appendix

Table 5: Compute/throughput comparison. Diffusion uses 20 denoising steps and is evaluated with FP16 AMP due to memory constraints; other models are evaluated in FP32.

Model	Parameters (M)	FLOPs (G)
ResNet50 (Single Task)	23.50	0.99
4 $\times$ ResNet50 (All Tasks)	94.02	3.96
BHCAST (Ours)	31.04	8.12

Appendix

Table 6: Comparison of parameter count and FLOPs count between our framework and ResNet50. Note that baseline is task-specific; extracting all four physical parameters requires training four separate ResNet models.

Figure 8: Forecasting Fidelity of FNO and Video Diffusion.

Figure 9: Pattern Speed Error Plateaus vs. Horizon

Figure 10: Rotation Curve Slope Correlation: Two methods show comparable correlation.

Figure 11: Pitch Angle Correlation: BHCast demonstrates a stronger correlation.

Figure 12: Asymmetry Correlation: although ResNet has a stronger correlation, its predictions are highly biased towards 0 when asymmetry is low.

Scenario	Model	Inclination Acc.	Spin Acc.
Only Blurring	ResNet50	47.19	67.66
Only Blurring	BHCast	56.41	69.22
Blurring	ResNet50	33.12 (-14.07)	52.19 (-15.47)
+ Salt & Pepper (1%)	BHCast	45.16 (-11.25)	67.50 (-1.72)
Blurring	ResNet50	28.75 (-18.44)	53.91 (-13.75)
+ Translation (5%)	BHCast	33.44 (-22.97)	73.44 (+4.22)

Appendix

Table 8: Robustness analysis of BHCast versus the ResNet baseline under input noise scenarios. BHCast demonstrates stability in spin accuracy under noise and translation.

Plasma Feature	Global (Joint)	Inclination	Spin
Pattern Speed $(\Omega_{p})$	0.2625	0.3683	0.2314
Pitch Angle $(\Phi)$	0.2686	0.2065	0.2567
Asymmetry	0.3314	0.2809	0.3833
Rot. Curve Slope	0.1376	0.1443	0.1285

Appendix

Table 9: XGBoost Feature Importance Scores. The Global column represents the importance for the joint (Spin, Inclination) classification task. The specific Inclination and Spin columns show which features dominate when predicting either of them. Notably, Pattern Speed is the primary feature for Inclination inference, while asymmetry is the critical feature for Spin.

Panel of Figure 13 — Figure 13: SHAP analysis for spin classification: SHAP values indicate asymmetry acts as the main feature for determining spin. High asymmetry values (red dots) push the models towards predicting positive spin (top plot), while low asymmetry values (blue dots) are strongly associated with negative spin predictions (bottom plot).

Panel of Figure 14 — Figure 14: SHAP analysis for inclination classification: Different from spin, pattern speed is the main feature for determining inclination. High pattern speed values (red dots) indicate positive inclinations, and vice versa. Notably, high pitch angle values contribute to face-on classifications, whereas low rotation curve slopes contribute to edge-on.

Augmentation	Count	Correct Rotation (%)
Base (original)	5	4/5 (80.0)
Translation ( $\Delta\!\in\!\{-2,-1,1\}$ )	15	10/15 (66.7)
Blur (PSF scale $\times\{0.9,1.1\}$ )	10	7/10 (70.0)
Correlated noise (corr=1.0)	20	14/20 (70.0)
All TTA (perturbed only)	45	31/45 (68.9)

Appendix

Table 10: M87* EHT eval via test-time augmentation (TTA).

Figure 15: M87* Pattern Speed Uncertainty Quantification.

Astrophysics

See all 29 papers

Learning Neural Operator Surrogates for the Black Hole Accretion Code
Apr 28, 2026Matthias Nägele, Cedric Bös, Chester Tan +3Computational Fluid Dynamics Transformer Neural Operator
Institute for Theoretical Physics and Astrophysics, Julius-Maximilians-Universität Würzburg, Würzburg, Germany · Chair for Machine Learning for Complex Networks, Julius-Maximilians-Universität Würzburg, Würzburg, Germany
NeuralDMD: Interpretable Neural Representation of Dynamics from Sparse and Noisy Measurements
Jul 3, 2025Ali SaraerToosi, Renbo Tu, Esther Y. H. Lin +2Spatiotemporal Forecasting Inverse Problems
University of Toronto, Toronto, ON, Canada · Vector Institute, Toronto, ON, Canada · NVIDIA Corporation, USA
Transformers with Physics-Informed Encodings and Simulation-Based Inference for Robust Detection of Eccentric Binary Black Holes in Pulsar Timing Array Data
Jul 4, 2026Subhajit Dandapat, Alvin J. K. ChuaGravitational-Wave Astronomy Transformer
Department of Physics, National University of Singapore, 21 Lower Kent Ridge Road, Singapore 119077 · Department of Mathematics, National University of Singapore, Singapore 119076

Apr 28, 2026J/K move · Enter open · S save

Learning Neural Operator Surrogates for the Black Hole Accretion Code

Matthias Nägele, Cedric Bös, Chester Tan, Christian M. Fromm+2

Institute for Theoretical Physics and Astrophysics, Julius-Maximilians-Universität Würzburg, Würzburg, Germany · Chair for Machine Learning for Complex Networks, Julius-Maximilians-Universität Würzburg, Würzburg, Germany

Computational Fluid Dynamics Transformer Neural Operator PDE Surrogate Modeling Fourier Neural Operator Physics-Informed Neural Operators Neural Operators+1

Open paper AskPDF arXiv

General-relativistic magnetohydrodynamic (GR-MHD) simulations are essential for studying black hole accretion, relativistic jets, and magnetic reconnection, yet their computational cost severely limits systematic parameter exploration. We investigate neural operator surrogates for two astrophysically relevant simulation scenarios produced by the Black Hole Accretion Code (BHAC). First, a Physics Informed Fourier Neural Operator (PINO) is trained on the special-relativistic resistive MHD (SRRMHD) evolution of the Orszag-Tang vortex over a range of resistivities spanning the Sweet-Parker and fast reconnection regimes. By embedding the governing equations as an additional loss term evaluated at finer temporal resolution than the available data supervision, the model learns dynamics at time steps where no simulation data is provided, enabling recovery of plasmoid formation that a data-only baseline trained on the same sparse snapshots fails to reproduce. To our knowledge, the present work is the first application of a physics informed neural operator to special relativistic resistive MHD, and the first to investigate the capability of such models to resolve plasmoid formation in SRRMHD. In a second line of investigation, an OFormer-style Transformer Neural Operator is trained on the evolution of spine-sheath relativistic jets created with BHAC, in special-relativistic MHD (SRMHD). The model is directly applied on the adaptive mesh, highlighting the need for linear attention due to long sequences. The neural surrogate model is capable of capturing most of the major details, especially in early predictions. To our knowledge, this constitutes the first application of a neural operator directly on a high resolution adaptive mesh refinement grid in the context of MHD simulations.

Abstract

Astrophysics

Explore similar work

Learning Neural Operator Surrogates for the Black Hole Accretion Code

NeuralDMD: Interpretable Neural Representation of Dynamics from Sparse and Noisy Measurements

Transformers with Physics-Informed Encodings and Simulation-Based Inference for Robust Detection of Eccentric Binary Black Holes in Pulsar Timing Array Data