Machine learning-assisted calibration of Agent-based Models: surrogate-based optimization with Genetic Algorithm and Particle Swarm Optimization

Duguma Yeshitla Habtemariam; Jihwan Lee

cs.MASep 4, 2026

Machine learning-assisted calibration of Agent-based Models: surrogate-based optimization with Genetic Algorithm and Particle Swarm Optimization

Authors: Duguma Yeshitla Habtemariam, Jihwan Lee

Organizations: Department of Industrial and Data Engineering Major in Industrial Data Science and Engineering Pukyong National University, Busan, Republic of Korea

Abstract

Calibrating an agent-based model (ABM) is difficult because its objective landscape is stochastic and rugged, and can be evaluated only through costly black-box simulations. This study adapts inner-loop surrogate-assisted evolutionary computation (SAEC) to ABM calibration by embedding a machine-learning surrogate within genetic algorithm (GA) and particle swarm optimisation (PSO). At each iteration, the surrogate screens the candidates and the simulator validates only the top 50%, reducing simulation demand while correcting surrogate errors. We evaluate a full factorial of 48 configurations combining two optimisers, five surrogates, and four calibration objectives on two contrasting ABMs, the Brock-Hommes asset-pricing model and the Island growth model. Parameter recovery is measured against pseudo-true values. Relative to the strongest pure-optimiser baseline, the best ML-assisted configurations reduce RMSE by 20.0% on Brock-Hommes and 63.8% on Island, while reducing computation time by 32.1% and 61.1%, respectively. ANOVA with Dunnett's post-hoc tests confirms significant time savings for every surrogate under both Brock-Hommes optimisers and under GA on Island. No surrogate differs significantly from the pure-optimiser baseline in parameter-recovery accuracy on Island; the reported accuracy gains are therefore best-configuration outcomes rather than average effects resolved at this sample size. The best surrogate-optimiser-objective combination changes with ABM complexity, indicating that surrogate-assisted calibration depends on the model and does not admit a universal default recipe.

Figures & tables

Figure 1: Inner-loop SAEC workflow across the optimiser, ML surrogate, and ABM simulator. The simulator first evaluates $P$ random parameter sets to train the surrogate. At each iteration, the optimiser proposes $P$ candidates, the surrogate ranks them, and the simulator evaluates the best $50\%$ . The resulting true scores retrain the surrogate and update the population, while the remaining candidates retain predicted scores. The loop continues until the stopping criterion is met and returns the parameter set with the lowest estimated error.

Type	Parameter	Symbol	Pseudo-true value	Search bounds
Fixed global	Risk-free return	$r$	1.05	—
	Intensity of choice	$\beta$	1.0	—
	Noise std dev	$\sigma$	0.05	—
Fixed strategy	Strategy 1 trend	$g_{1}$	0	—
	Strategy 1 bias	$b_{1}$	0	—
	Strategy 4 trend	$g_{4}$	0	—

Table 1: BH model parameters for the pseudo-true data generation experiment.

Type	Parameter	Symbol	Pseudo-true value	Search bounds
Free (calibrated)	Knowledge diffusion locality	$\rho$	0.01	$[0,\,10]$
	Labour productivity	$\alpha$	1.5	$[0.8,\,2]$
	Cumulative learning effect	$\varphi$	0.4	$[0,\,1]$
	Island probability	$\pi$	0.4	$[0,\,1]$
	Willingness to explore	$\varepsilon$	0.1	$[0,\,1]$
	Initial number of firms	$m_{0}$	50	$[10,\,100]$

Table 2: Island model parameters for the pseudo-true data generation experiment.

Table 4

Objective	Parameter	Value
MSM	$L$	$-1$
KS	$w_{X}$ , $w_{\|X\|}$ , $w_{X^{2}}$	1.0, 1.0, 1.0
MIC	$L,r$	3, 2
GSL-div	$L,b$	3, 4

Table 5: Objective function hyperparameters.

Model	Parameter	Value	Parameter	Value
XGBoost	eta	0.3	max_depth	6
	num_rounds	100	subsample	1
	colsample_bytree	1	min_child_weight	1
	lambda	1	alpha	0
	seed	1
Linear regression	learning_rate	0.1	epochs	50

Table 6: ML surrogate hyperparameters.

Rank	Optimizer	Objective	ML Surrogate	Mean RMSE	SD	Time (s)
1 $\star$	PSO-ML	MIC	ANN	0.2286	0.1893	5932
2	PSO-ML	MIC	LR	0.2543	0.1472	5705
3	PSO-ML	GSL-div	ANN	0.2758	0.1667	5940
4	GA	MIC	—	0.2856	0.2600	8739
5	GA	MSM	—	0.3251	0.1567	8734
6	PSO-ML	MIC	Random Forest	0.3538	0.2672	5786

Table 7: Top-10 BH calibration configurations ranked by mean RMSE. Mean and standard deviation are taken over the five independent runs.

Figure 2: RMSE across calibration objectives for pure versus ML-augmented approaches on the BH model. Each axis represents an optimizer–objective pair; lower values (closer to the centre) indicate better parameter recovery. Circled points on each axis highlight the best-performing approach for that optimizer–objective combination.

Figure 3: Computation time across calibration objectives for pure versus ML-augmented approaches on the BH model. Each axis represents an optimizer–objective pair; lower values (closer to the centre) indicate faster calibration. Circled points highlight the fastest surrogate for each optimizer–objective combination.

Figure 4: SHAP beeswarm plot for the surrogate of the single most accurate BH run (PSO-ML + MSM + XGBoost). Each dot is one of $200$ parameter vectors sampled from the region the search covered; horizontal position = signed Shapley value (negative reduces MSM loss, positive increases it); vertical axis ranks parameters by mean absolute influence, largest at the top; colour encodes normalised parameter value (blue = low, red = high).

Figure 5: PSO convergence for the best BH configuration (PSO-ML + MIC + ANN). The curve traces the best-so-far MIC loss against PSO iteration; lower is better.

Figure 6: Surrogate $R^{2}$ across calibration iterations for the top-3 BH ML-assisted configurations, averaged over the five independent runs. Values below zero mark a surrogate whose squared error exceeds that of simply predicting the mean loss. Ranks 1 and 2 are fitted to MIC and rank 3 to GSL-div, so the three curves are not measured against a common loss surface.

Rank	Optimizer	Objective	ML Surrogate	Mean RMSE	SD	Time (s)
1 $\star$	GA-ML	KS	ANN	2.3862	1.0717	9307
2	GA-ML	GSL-div	ANN	3.3922	1.0623	11615
3	GA-ML	KS	LR	5.0387	3.3567	8935
4	GA-ML	GSL-div	LR	5.6716	2.4579	16865
5	GA-ML	KS	SVM	5.7752	5.0327	9558
6	PSO-ML	KS	Random Forest	5.9595	5.6590	8645

Table 8: Top-10 Island calibration configurations ranked by mean RMSE

Figure 7: RMSE across calibration objectives for pure versus ML-augmented approaches on the Island model. Each axis represents an optimizer–objective pair; lower values (closer to the centre) indicate better parameter recovery. Circled points on each axis highlight the best-performing approach for that optimizer–objective combination.

Figure 8: Computation time across calibration objectives for pure versus ML-augmented approaches on the Island model. Each axis represents an optimizer–objective pair; lower values (closer to the centre) indicate faster calibration. Circled points highlight the fastest approach for each optimizer–objective combination.

Figure 9: SHAP beeswarm plot for the surrogate of the single most accurate Island run (PSO-ML + GSL-div + LR). Each dot is one of $200$ parameter vectors sampled from the region the search covered; horizontal position = signed Shapley value (negative reduces GSL-div loss, positive increases it); vertical axis ranks parameters by mean absolute influence, largest at the top; colour encodes normalised parameter value (blue = low, red = high).

Figure 10: GA convergence for the best Island configuration (GA-ML + KS + ANN). The curve traces the best-so-far KS loss against GA generation; lower is better.

Figure 11: Surrogate $R^{2}$ across calibration iterations for the top-3 Island ML-assisted configurations, averaged over the five independent runs. Values below zero mark a surrogate whose squared error exceeds that of simply predicting the mean loss. Ranks 1 and 3 are fitted to KS and rank 2 to GSL-div, so the three curves are not measured against a common loss surface.

Surrogate	Mean	$\Delta$	$p$	Mean	$\Delta$	$p$
	GA			PSO
LR	0.940	$+$ 0.462	${<}0.001^{*}$	0.657	$+$ 0.006	1.000
SVM	0.832	$+$ 0.354	$0.004^{*}$	0.558	$-$ 0.094	0.864
RandomForest	0.710	$+$ 0.232	0.108	0.556	$-$ 0.095	0.859
XGBoost	0.645	$+$ 0.168	0.359	0.607	$-$ 0.044	0.994
ANN	0.563	$+$ 0.085	0.883	0.429	$-$ 0.223	0.169

Table 9: Dunnett’s post-hoc test results for calibration RMSE on the Brock–Hommes model. The control mean for GA is

0.478

and for PSO is

0.651

\Delta

denotes the difference (treatment

-

control); ∗ indicates

p<0.05

. Positive

\Delta

means the surrogate is less accurate than the pure optimiser.

Surrogate	Mean	$\Delta$	$p$	Mean	$\Delta$	$p$
	GA			PSO
LR	6.782	$-$ 1.376	0.768	11.851	$-$ 2.849	0.346
SVM	7.838	$-$ 0.320	0.999	12.762	$-$ 1.938	0.699
RandomForest	8.363	$+$ 0.205	1.000	13.928	$-$ 0.772	0.990
XGBoost	8.884	$+$ 0.726	0.978	14.465	$-$ 0.235	1.000
ANN	4.955	$-$ 3.204	0.081	11.442	$-$ 3.258	0.228

Table 10: Dunnett’s post-hoc test results for calibration RMSE on the Island growth model. The control mean for GA is

8.158

and for PSO is

14.700

\Delta

denotes the difference (treatment

-

control); negative values indicate better parameter recovery than the pure optimiser. No comparison reaches

p<0.05

under either optimiser, so no entry is starred.

Surrogate	Mean (s)	$\Delta$ (s)	$p$	Mean (s)	$\Delta$ (s)	$p$
	GA			PSO
LR	5,507	$-$ 3,275	${<}0.001^{*}$	5,717	$-$ 3,434	${<}0.001^{*}$
SVM	5,345	$-$ 3,438	${<}0.001^{*}$	5,712	$-$ 3,439	${<}0.001^{*}$
RandomForest	5,444	$-$ 3,338	${<}0.001^{*}$	5,844	$-$ 3,307	${<}0.001^{*}$
XGBoost	5,429	$-$ 3,353	${<}0.001^{*}$	5,785	$-$ 3,366	${<}0.001^{*}$
ANN	5,646	$-$ 3,136	${<}0.001^{*}$	5,887	$-$ 3,264	${<}0.001^{*}$

Table 11: Dunnett’s post-hoc test results for total computation time (seconds) on the Brock–Hommes model. The control mean for GA is

8{,}782

s and for PSO is

9{,}151

\Delta

denotes time saved (negative values indicate faster runtime); all

p<0.001^{*}

Surrogate	Mean (s)	$\Delta$ (s)	$p$	Mean (s)	$\Delta$ (s)	$p$
	GA			PSO
LR	14,817	$-$ 11,413	${<}0.001^{*}$	9,930	$-$ 3,255	0.691
SVM	15,475	$-$ 10,755	${<}0.001^{*}$	13,061	$-$ 123	1.000
RandomForest	16,909	$-$ 9,322	$0.001^{*}$	13,139	$-$ 46	1.000
XGBoost	17,280	$-$ 8,950	$0.002^{*}$	13,926	$+$ 741	0.999
ANN	13,355	$-$ 12,875	${<}0.001^{*}$	10,090	$-$ 3,095	0.730

Table 12: Dunnett’s post-hoc test results for total computation time (seconds) on the Island growth model. The control mean for GA is

26{,}230

s and for PSO is

13{,}185

\Delta

denotes time saved (negative values indicate faster runtime); ∗ indicates

p<0.05

. All five GA comparisons are significant; none of the PSO comparisons is.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

			Accuracy (RMSE)			Speed (seconds)
Optimizer	Objective	Best Surrogate	Pure	Surrogate	Impr. (%)	Pure	Surrogate	Impr. (%)
GA	GSL-div	ANN	0.4497	0.6086	$-$ 35.3	8795.2	5724.7	$+$ 34.9
GA	MSM	ANN	0.3251	0.3761	$-$ 15.7	8734.3	5549.3	$+$ 36.5
GA	MIC	ANN	0.2856	0.3985	$-$ 39.5	8739.1	5570.0	$+$ 36.3
GA	KS-test	ANN	0.8513	0.8682	$-$ 2.0	8860.5	5740.8	$+$ 35.2
PSO	GSL-div	ANN	0.4814	0.2758	$+$ 42.7	9215.0	5939.6	$+$ 35.5

Appendix

Table B1: Brock–Hommes model: pure-optimizer vs. best surrogate-assisted comparison for each optimizer–objective pair. The best surrogate is the one with the lowest mean RMSE across five independent runs; positive improvement means it reduces error or computation time relative to the pure baseline, negative that it is worse. Accuracy improvement is negative for all four GA pairs and positive for all four PSO pairs, the split also visible in Figure 2 , whereas the time improvement is between

34.4\%

and

36.6\%

for every pair.

			Accuracy (RMSE)			Speed (seconds)
Optimizer	Objective	Best Surrogate	Pure	Surrogate	Impr. (%)	Pure	Surrogate	Impr. (%)
GA	GSL-div	ANN	8.7008	3.3922	$+$ 61.0	30233.4	11614.9	$+$ 61.6
GA	MSM	ANN	6.5874	6.5874	$+$ 0.0	23899.5	24400.4	$-$ 2.1
GA	MIC	ANN	10.3711	7.4523	$+$ 28.1	27347.6	8097.9	$+$ 70.4
GA	KS-test	ANN	6.9732	2.3862	$+$ 65.8	23440.8	9307.5	$+$ 60.3
PSO	GSL-div	LR	13.5182	10.3046	$+$ 23.8	27237.2	17628.1	$+$ 35.3

Appendix

Table B2: Island growth model: pure-optimizer vs. best surrogate-assisted comparison, with columns and conventions as in Table B1 . In contrast to the BH pattern, accuracy improvement is non-negative for all eight pairs, from

0.0\%

on the two MSM pairs, where no surrogate alters the result at all, up to

65.8\%

. Time improvement is far more variable than the uniform

34

–

37\%

seen on BH, spanning

-6.2\%

+70.4\%

and negative only on those same two MSM pairs.

Explore similar work

Agentic Calibration of Grey-Box Simulation Models: An LLM-Driven Alternative
Jul 17, 2026David Gómez-Guillén, Mireia Diaz, Josep Lluis Arcos +1Parameter Estimation Black-Box Optimization
Unit of Infections and Cancer – Information and Interventions (UNIC-I&I), Cancer Epidemiology Research Program, Catalan Institute of Oncology-IDIBELL, L’Hospitalet de Llobregat, Spain · Autonomous University of Barcelona (UAB), Spain · Centro de Investigación Biomédica en Red de Epidemiología y Salud Pública (CIBERESP), Ministerio de Ciencia, Innovación y Universidades, Madrid, Spain +1
From Model-Based Screening to Data-Driven Surrogates: A Multi-Stage Workflow for Exploring Stochastic Agent-Based Models
Apr 3, 2026Paul Saves, Matthieu Mastio, Nicolas Verstaevel +1Sensitivity Analysis Agent-Based Modeling
IRIT, Université Toulouse Capitole, Toulouse, France
ABM-UDE: Developing Surrogates for Epidemic Agent-Based Models via Scientific Machine Learning
Feb 25, 2026Sharv Murgai, Utkarsh Utkarsh, Kyle C. Nguyen +3Neural Surrogate Modeling Epidemic Forecasting
Monta Vista High School · MIT CSAIL · Sandia National Laboratories

Jul 17, 2026J/K move · Enter open · S save

Agentic Calibration of Grey-Box Simulation Models: An LLM-Driven Alternative

David Gómez-Guillén, Mireia Diaz, Josep Lluis Arcos, Jesús Cerquides

Unit of Infections and Cancer – Information and Interventions (UNIC-I&I), Cancer Epidemiology Research Program, Catalan Institute of Oncology-IDIBELL, L’Hospitalet de Llobregat, Spain · Autonomous University of Barcelona (UAB), Spain · Centro de Investigación Biomédica en Red de Epidemiología y Salud Pública (CIBERESP), Ministerio de Ciencia, Innovación y Universidades, Madrid, Spain+1

Parameter Estimation Black-Box Optimization Large Language Model-Guided Optimization Model Calibration Constrained Optimization

Open paper AskPDF arXiv

Calibration of grey-box simulation models is a constrained optimization problem in which model evaluations are expensive, the parameter space can be high-dimensional, and the search must respect plausibility constraints. Although the simulation code is fully available to the analyst, the joint effect of multiple parameters remains difficult to predict analytically. Classical optimizers such as Nelder--Mead (NM) are simple to deploy but sample-inefficient, particularly under constraints. Modern Bayesian Optimization methods achieve competitive solutions with far fewer evaluations but require non-trivial modeling machinery for constraint handling. We introduce an agentic calibration method in which a large language model acts as the optimizer, with constraints incorporated as a plain-language section of the system prompt. We evaluate the agentic method, NM, and Bayesian Optimization (BO) on an anal cancer simulation model under both unconstrained and clinically constrained calibration. Under unconstrained calibration, the agentic method achieves substantially lower best error than BO and NM, while requiring fewer model evaluations. Under constrained calibration, the agentic method reaches comparable error levels and both outperform NM. These results are obtained at the cost of increased inference time per iteration. Agentic calibration achieves competitive performance with substantially fewer model evaluations, and constraint handling is essentially free at the modeller-facing interface through simple textual specifications rather than additional modelling machinery. The main trade-off lies in increased per-iteration inference cost, making the approach particularly suitable when simulation time dominates. Beyond performance, the per-iteration rationale makes the search auditable and explainable, so its decisions can be scrutinised and justified to third parties.

Abstract

Explore similar work

Agentic Calibration of Grey-Box Simulation Models: An LLM-Driven Alternative

From Model-Based Screening to Data-Driven Surrogates: A Multi-Stage Workflow for Exploring Stochastic Agent-Based Models

ABM-UDE: Developing Surrogates for Epidemic Agent-Based Models via Scientific Machine Learning