A systematic comparison of physics-informed neural networks and mechanistic models for epidemic modeling and forecasting
Authors: Martina Rama, Yuhan Li, Bruno Lepri, Nicola Perra, Michele Tizzoni
Organizations: Department of Information Engineering and Computer Science, University of Trento, Trento, Italy. · Mobile and Social Computing (MobS) Lab, Fondazione Bruno Kessler (FBK), Trento, Italy. · School of Mathematical Sciences, Queen Mary University, London, United Kingdom. · Department of Sociology and Social Research, University of Trento, Trento, Italy.
Physics-Informed Neural Networks (PINNs) have recently been applied to epidemic modeling and forecasting with promising results. These methods integrate mechanistic knowledge with data-driven learning. Yet, their comparative performance against well-established, and simpler epidemic models is underexplored. Here, we conduct a systematic comparison of a PINN-based framework and a stochastic mechanistic model. Both approaches share the same age-structured SEIRD core and rolling-window forecasting pipeline, in a deliberately minimal setting. We evaluate their retrospective fits and probabilistic forecasts in two synthetic scenarios and the first COVID-19 wave in London. Interestingly, while PINNs underperform in synthetic scenarios, they achieve better performance on real data. Furthermore, PINNs provide narrower uncertainty intervals and sharper estimation of parameters in synthetic scenarios. Our findings confirm PINNs as a promising tool in epidemiology. At the same time, the results reaffirm the value of simpler approaches such as traditional compartmental models, and promote additional benchmarking studies.
Figures & tables
Figure 1: Overview of the two forecasting frameworks. The PINN-based approach is shown in violet and the mechanistic approach in teal. Both rely on the same age-structured Susceptible-Exposed-Infectious-Recovered-Dead (SEIRD) model ( a=1,…,16 ; left panel), with known parameters ψ={ε,ξ,C,IFRa,Na} , unknown parameters λ={R0,1/δ} , and solution u(t;λ,ψ) . Step 1 (offline, once). The PINN uN(t,R0,1/δ) is trained on a parameter grid, with the ODE system embedded in the loss through LODE , to approximate u . The mechanistic approach builds a pool of stochastic chain-binomial simulations, one for each parameter set. Step 2 (every window). The unknown parameters are inferred from the weekly incidence data in the look-back window, via MCMC with a Poisson likelihood for the PINN and via likelihood-free ABC-SMC with a Weighted Mean Absolute Percentage Error (WMAPE) distance for the mechanistic model. For the London COVID-19 data, the observed quantity is weekly deaths and both R0 and 1/δ are estimated; for the synthetic scenarios, the observed quantity is weekly new infections and only R0 is estimated. Step 3 (every window). Probabilistic forecasts at horizons h1 – h4 (1–4 weeks ahead) are obtained by evaluating the PINN at the posterior samples with their uncertainty and by selecting the pool trajectories associated with the accepted particles for the mechanistic-based framework. The central panel illustrates a single window: the dashed line separating Steps 2 and 3 passes through the last available observation at time tc , from which the forecasts start; filled circles are the observed data used in Step 2, open circles the future observations against which the forecasts are evaluated. Steps 2–3 are repeated every week in a rolling-window manner. The right panel shows, as an example, the resulting 1-week-ahead forecasts ( h1 ) over the whole period (median, 50% and 90% prediction intervals), against the observed data (black); the same is done for horizons h2 – h4 .
Figure 2: Retrospective fit and R0 posteriors in the two synthetic scenarios. Top row: first scenario ( R0=1.5 ), with data up to week 34 ; bottom row: second scenario ( R0=2.0 ), with data up to week 24 . Left panels: retrospective fit of the weekly new infections obtained by the PINN -based (violet) and mechanistic-based (teal) frameworks, calibrated on all the noisy observations up to the last week (white dots). Shaded areas show the 90% (lighter) and 50% (darker) prediction intervals, and lines the medians. Right panels: posterior distributions of R0 estimated by the two frameworks at the same week, with densities scaled to a maximum of 1 . Dotted lines mark the posterior medians and the dashed black line the true value of R0 used to generate the data.
Scenario
Model
MAE
RMSE
WIS
IS 80
Cov. 50%
Cov. 90%
Scenario 1 R0=1.5
PINN
0.227
0.283
0.224
0.222
0.343
0.546
Mechanistic
0.189
0.222
0.191
0.191
0.806
1.000
Baseline
1.000
1.000
1.000
1.000
0.333
0.583
Scenario 2 R0=2.0
PINN
0.127
0.162
0.126
0.137
0.206
0.324
Mechanistic
0.094
0.119
0.094
0.099
0.956
1.000
Baseline
1.000
1.000
1.000
1.000
0.412
0.588
Table 1: Comparison of overall forecasting performance on the two synthetic scenarios. Forecasts of weekly new infections, on the noisy synthetic data with R0=1.5 and R0=2.0 , aggregated over all horizons (from 1 to 4) and rounds. MAE, RMSE, WIS, and IS 80 are normalized by the baseline score of the same scenario. Bold indicates the best performance among PINN , Mechanistic , and Baseline within each scenario; for coverage, the value closest to the nominal level.
Scenario 1: R0=1.5
Scenario 2: R0=2.0
Horizon
Model
MAE
RMSE
WIS
IS 80
C50%
C90%
MAE
RMSE
WIS
IS 80
C50%
C90%
H1
PINN
0.477
0.583
0.513
0.532
0.259
0.444
0.295
0.340
0.320
0.369
0.176
0.294
Mechanistic
0.382
0.453
0.359
0.346
0.778
1.000
0.204
0.228
0.186
0.181
0.941
1.000
Baseline
1.000
1.000
1.000
1.000
0.296
0.630
1.000
1.000
1.000
1.000
0.412
0.647
H2
PINN
0.272
0.337
0.277
0.288
0.333
0.556
0.136
0.191
0.139
0.156
0.235
0.471
Mechanistic
0.257
0.303
0.240
0.237
0.778
1.000
0.104
0.115
0.100
0.112
1.000
1.000
Table 2: Comparison of horizon-specific forecasting performance on the two synthetic scenarios. Forecasts of weekly new infections on the noisy synthetic data with R0=1.5 and R0=2.0 , aggregated over all rounds (27 rounds in the first scenario and 17 in the second) for each horizon, from 1 to 4. MAE, RMSE, WIS, and IS 80 are normalized by the baseline score of the same scenario and horizon. Bold indicates the best performance among PINN , Mechanistic , and Baseline within each scenario and horizon; for coverage, the value closest to the nominal level.
Figure 3: Horizon-specific probabilistic forecasts on the two synthetic scenarios. Forecasts from the PINN -based (in violet) and mechanistic (in teal) frameworks of the weekly new infections on the noisy synthetic data with R0=1.5 (on the left) and R0=2.0 (on the right), at fixed horizons of 1 to 4 weeks ahead (panels from top to bottom). In each panel, every box refers to the week on the x-axis and shows the forecast made, for that week, as many weeks earlier as the horizon of the panel, using only the data available at that time. The lighter and darker boxes show the 90% and 50% prediction intervals, respectively, and the horizontal line inside each box shows the median. The observed weekly new infections are shown in black.
Figure 4: Posterior distributions of Rt across rounds in the two synthetic scenarios. For each round, the split violin shows the posterior distribution of the effective reproduction number by the PINN-based framework with MCMC (left half, violet) and by the mechanistic framework via ABC-SMC (right half, teal). Each split violin shows the full shape of the 1000 tail posterior samples for that window; horizontal ticks mark the per-window median for each model. In addition, the solid horizontal line marks the epidemic threshold Rt=1 , below which the outbreak is expected to decline, and the shaded vertical band marks the week of the peak in weekly new infections, week 16 in the first scenario ( R0=1.5 , top) and week 10 in the second ( R0=2.0 , bottom).
Model
MAE
RMSE
WIS
IS 80
Cov. 50%
Cov. 90%
PINN
2.532
2.926
2.629
2.725
0.076
0.394
Mechanistic
3.577
4.954
3.879
4.144
0.182
0.439
Baseline
1.000
1.000
1.000
1.000
0.621
0.788
Table 3: Comparison of overall forecasting performance on the real COVID-19 scenario. Forecasts of weekly deaths for the first COVID-19 wave in London (UK), aggregated over all horizons (from 1 to 4) and rounds. MAE, RMSE, WIS, and IS 80 are normalized by the baseline score.
Figure 5: Horizon-specific probabilistic forecasts of weekly COVID-19 deaths. Forecasts of the PINN -based and mechanistic frameworks for the first COVID-19 wave in London (UK), at fixed horizons of 1 to 4 weeks ahead (panels from top to bottom). In each panel, every box refers to the week on the x-axis and shows the forecast made for that week, as many weeks earlier as the horizon of the panel, using only the data available at that time. The lighter and darker boxes show the 90% and 50% prediction intervals, respectively, and the horizontal line inside each box shows the median. The observed weekly deaths are shown in black.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Param
Description
Value / bounds
PINN-based
Mechanistic-based
ε
Latency rate
1/3.7 day -1 Kissler et al. (2020) ; Ma et al. (2020) ; Backer et al. (2020) ; Zhao et al. (2021) ; Xin et al. (2022)
Fixed
Fixed
ξ
Removal rate
1/2.5 day -1 Kissler et al. (2020) ; Ma et al. (2020) ; Backer et al. (2020) ; Zhao et al. (2021) ; Xin et al. (2022)
Fixed
Fixed
IFRa
Infection fatality ratio
Age-specific Verity et al. (2020)
Fixed
Fixed
C
Contact matrix
UK Prem et al. (2017)
Fixed
Fixed
Na
Population per age class
London Office for National Statistics (ONS) (2020)
Fixed
Fixed
R0
Basic reproduction number
[1.3,3]
Estimated ( MCMC )
Estimated ( ABC-SMC )
Appendix
Table 4: Parameters of the two frameworks: fixed and shared (top), estimated (middle), and derived (bottom). In the synthetic scenarios, the observed quantity (newly exposed individuals) does not depend on δ nor on IFRa .
Figure 6: Comparison of PINN and mechanistic forecasts of weekly new infections on the noisy synthetic data with R0=1.5 , across 28 forecast rounds (i.e., weeks). In each panel, black dots denote the observed weekly new infections used for model calibration up to the current week, while white dots denote the subsequent four weeks to forecast. Shaded bands show the 50% and 90% prediction intervals of each model over the 1- to 4-week-ahead forecast horizons, and solid lines indicate the corresponding medians.
Figure 7: Comparison of PINN and mechanistic forecasts of weekly new infections on the noisy synthetic data with R0=2.0 , across 16 forecast rounds (i.e., weeks). In each panel, black dots denote the observed weekly new infections used for model calibration up to the current week, while white dots denote the subsequent four weeks to forecast. Shaded bands show the 50% and 90% prediction intervals of each model over the 1- to 4-week-ahead forecast horizons, and solid lines indicate the corresponding medians.
Figure 8: Retrospective fit of the age-specific weekly incidence on the first synthetic scenario ( R0=1.5 ). Weekly new infections by age class estimated by the PINN -based (left) and mechanistic (right) frameworks over the 34 weeks of the simulation, for the age classes 0–4, 5–9, 10–14, 15–19, and 20–24 years (top row) and 30–34, 40–44, 50–54, 60–64, and 75+ years (bottom row). Lines show the median predictions of the models and shaded bands the 90% prediction interval of the estimates, and the dashed lines show the true incidence of the simulation, before the age-classes aggregation and noise addition.
Figure 9: Retrospective fit of the age-specific weekly incidence on the second synthetic scenario ( R0=2.0 ). Weekly new infections by age class estimated by the PINN -based (left) and mechanistic (right) frameworks over the 24 weeks of the simulation, for the age classes 0–4, 5–9, and 15–19 years (top row) and 30–34, 50–54, 60–64, and 75+ years (bottom row). Lines show the median predictions of the models and shaded bands the 90% prediction interval of the estimates, and the dashed lines show the true incidence of the simulation, before the agage-class aggregation and noise addition.
Figure 10: Retrospective fit and R0 posteriors in the two synthetic scenarios under stronger noise. Top row: first scenario ( R0=1.5 ), with data up to week 34 ; bottom row: second scenario ( R0=2.0 ), with data up to week 24 . Both are generated from the synthetic data with those R0 and stronger negative binomial noise ( r=50 ). Left panels: retrospective fit of the weekly new infections obtained by the PINN -based (violet) and mechanistic-based (teal) frameworks, calibrated on all the noisy observations up to the last week (white dots). Shaded areas show the 90% (lighter) and 50% (darker) prediction intervals, and lines the medians. Right panels: posterior distributions of R0 estimated by the two frameworks at the same week, with densities scaled to a maximum of 1 . Dotted lines mark the posterior medians and the dashed black line the true value of R0 used to generate the data.
Scenario
Model
MAE
RMSE
WIS
IS 80
Cov. 50%
Cov. 90%
Scenario 1 R0=1.5 stronger noise
PINN
0.325
0.465
0.345
0.360
0.204
0.593
Mechanistic
0.298
0.379
0.270
0.245
0.676
0.935
Baseline
1.000
1.000
1.000
1.000
0.352
0.611
Scenario 2 R0=2.0 stronger noise
PINN
0.215
0.294
0.212
0.233
0.132
0.515
Mechanistic
0.192
0.233
0.159
0.130
0.706
0.956
Baseline
1.000
1.000
1.000
1.000
0.397
0.588
Appendix
Table 5: Comparison of overall forecasting performance on the two synthetic scenarios under stronger noise. Forecasts of weekly new infections, generated from the synthetic data with R0=1.5 and stronger negative binomial noise ( r=50 ), aggregated over all horizons (from 1 to 4) and rounds. MAE, RMSE, WIS, and IS 80 are normalized by the baseline score of the same scenario. Bold indicates the best performance among PINN , Mechanistic , and Baseline within each scenario; for coverage, the value closest to the nominal level.
Scenario 1: R0=1.5 , stronger noise
Scenario 2: R0=2.0 , stronger noise
Horizon
Model
MAE
RMSE
WIS
IS 80
C50%
C90%
MAE
RMSE
WIS
IS 80
C50%
C90%
H1
PINN
0.555
0.774
0.651
0.744
0.185
0.519
0.365
0.458
0.398
0.423
0.118
0.353
Mechanistic
0.487
0.656
0.458
0.403
0.667
0.926
0.381
0.421
0.313
0.233
0.647
0.941
Baseline
1.000
1.000
1.000
1.000
0.296
0.667
1.000
1.000
1.000
1.000
0.412
0.706
H2
PINN
0.373
0.480
0.395
0.416
0.148
0.519
0.292
0.446
0.300
0.347
0.118
0.647
Mechanistic
0.359
0.471
0.326
0.311
0.667
0.926
0.238
0.317
0.194
0.145
0.765
1.000
Appendix
Table 6: Comparison of horizon-specific forecasting performance on the two synthetic scenarios under stronger noise. Forecasts of weekly new infections on the synthetic data with R0=1.5 and R0=2.0 and stronger negative binomial noise ( r=50 ), aggregated over all rounds (27 rounds in the first scenario and 17 in the second) for each horizon, from 1 to 4. MAE, RMSE, WIS, and IS 80 are normalized by the baseline score of the same scenario and horizon. Bold indicates the best performance among PINN , Mechanistic , and Baseline within each scenario and horizon; for coverage, the value closest to the nominal level.
Figure 11: Horizon-specific probabilistic forecasts on the two synthetic scenarios under stronger noise. Forecasts from the PINN -based (in violet) and mechanistic (in teal) of the weekly new infections on the noisy synthetic data with stronger negative binomial noise ( r=50 ) with R0=1.5 (on the left) and R0=2.0 (on the right), at fixed horizons of 1 to 4 weeks ahead (panels from top to bottom). In each panel, every box refers to the week on the x-axis and shows the forecast made, for that week, as many weeks earlier as the horizon of the panel, using only the data available at that time. The lighter and darker boxes show the 90% and 50% prediction intervals, respectively, and the horizontal line inside each box shows the median. The observed weekly new infections are shown in black.
Figure 12: Posterior distributions of Rt across rounds in the two synthetic scenarios under stronger noise. For each round, the split violin shows the posterior distribution of the effective reproduction number estimated by the PINN-based framework with MCMC (left half, violet) and by the mechanistic framework via ABC-SMC (right half, teal). Each split violin shows the full shape of the 1000 tail posterior samples for that window; horizontal ticks mark the per-window median for each model. In addition, the solid horizontal line marks the epidemic threshold Rt=1 , below which the outbreak is expected to decline, and the shaded vertical band marks the week of the peak in weekly new infections generated with stronger negative binomial noise ( r=50 ), week 16 in the first scenario ( R0=1.5 , top) and week 10 in the second ( R0=2.0 , bottom).
Figure 13: Comparison of PINN and mechanistic forecasts of weekly COVID-19 deaths in London across sixteen forecast rounds (i.e., weeks). In each panel, black dots denote the observed weekly deaths used for model calibration up to the current week, while white dots denote the subsequent four weeks to forecast. Shaded bands show the 50% and 90% prediction intervals of each model over the 1- to 4-week-ahead forecast horizons, and solid lines indicate the corresponding medians.
Horizon
Model
MAE
RMSE
WIS
IS 80
Cov. 50%
Cov. 90%
H1 (18 rounds)
PINN
2.124
2.002
1.926
1.689
0.056
0.500
Mechanistic
2.032
2.461
2.112
2.033
0.278
0.500
Baseline
1.000
1.000
1.000
1.000
0.667
0.833
H2 (17 rounds)
PINN
2.211
2.458
2.186
2.373
0.059
0.353
Mechanistic
2.880
3.757
3.036
3.399
0.235
0.412
Baseline
1.000
1.000
1.000
1.000
0.588
0.706
Appendix
Table 7: Comparison of horizon-specific forecasting performance on the real COVID-19 scenario. Forecasts of weekly deaths, for the first COVID-19 wave in London (UK), aggregated over all rounds for each horizon (from 1 to 4). MAE, RMSE, WIS, and IS 80 are normalized by the baseline score.
Figure 14: Posterior distributions of Rt between the two frameworks across weeks. For each round, the split violin shows the posterior distribution of the effective reproduction number estimated by the PINN-based framework with MCMC (left half, violet) and by the mechanistic framework via ABC-SMC (right half, teal). Each split violin shows the full shape of the 1000 tail posterior samples for that window; horizontal ticks mark the per-window median for each model. In addition, the solid horizontal line marks the epidemic threshold Rt=1 , below which the outbreak is expected to decline, and the shaded vertical band marks the week of the peak in reported weekly new cases.
Look-back window
Model
MAE
RMSE
WIS
IS 80
Cov. 50%
Cov. 90%
5 weeks
PINN
2.257
2.778
2.206
2.313
0.379
0.621
Mechanistic
3.507
5.071
3.841
4.121
0.167
0.364
All past data
PINN
2.532
2.926
2.629
2.725
0.076
0.394
Mechanistic
3.577
4.954
3.879
4.144
0.182
0.439
–
Baseline
1.000
1.000
1.000
1.000
0.621
0.788
Appendix
Table 8: Comparison of overall forecasting performance on the real COVID-19 scenario with different look-back windows. Forecasts of weekly deaths for the first COVID-19 wave in London (UK), aggregated over all horizons (from 1 to 4) and rounds. The first two rows report the results obtained with a look-back window of the last M=5 weeks of observations, without the weighting strategy, while the other two rows report the results of the main analysis (Table 3 ), where all the past data available are used, with the weighting strategy. Baseline is reported in the last row, since its implementation does not depend on the look-back window length. MAE, RMSE, WIS, and IS 80 are normalized by the baseline score.
Horizon
Model
MAE
RMSE
WIS
IS 80
Cov. 50%
Cov. 90%
H1 (18 rounds)
PINN
1.765
1.970
1.431
1.213
0.389
0.667
Mechanistic
1.868
2.447
1.920
1.803
0.278
0.500
Baseline
1.000
1.000
1.000
1.000
0.667
0.833
H2 (17 rounds)
PINN
1.864
2.247
1.781
1.967
0.412
0.588
Mechanistic
2.851
3.942
2.985
3.315
0.176
0.353
Baseline
1.000
1.000
1.000
1.000
0.588
0.706
Appendix
Table 9: Comparison of horizon-specific forecasting performance on the real COVID-19 scenario with a look-back window of length 5 . Forecasts of weekly deaths, for the first COVID-19 wave in London (UK), aggregated over all rounds for each horizon (from 1 to 4). Unlike in the main analysis, where all the past data available are used (see Table 3 for comparison), at each round both frameworks are calibrated on the last M=5 weeks of observations only, without the weighting strategy. MAE, RMSE, WIS, and IS 80 are normalized by the baseline score.
Epidemic forecasting predicts disease incidence, outbreaks, and healthcare burden across time and regions to guide surveillance and public-health decisions.