Synthesizing Feature Extractors: An Agentic Approach for Algorithm Selection

Hai Xia; Carlos Ansótegui; Stefan Szeider

cs.AIAug 17, 2026

Synthesizing Feature Extractors: An Agentic Approach for Algorithm Selection

Authors: Hai Xia, Carlos Ansótegui, Stefan Szeider

Organizations: Algorithms and Complexity Group, TU Wien, Vienna, Austria · Logic & Optimization Group, University of Lleida, Lleida, Spain

Abstract

Algorithm selection for constraint satisfaction problems requires extracting features that capture problem structure. Manually designing feature extractors demands deep domain expertise and quickly becomes a bottleneck when new problem classes appear. We present an automated approach that uses Large Language Models (LLMs) in an agentic check--fix--verify loop to synthesize executable Python scripts that act as interpretable, problem-specific feature extractors. Given a high-level MiniZinc model and an instance, the LLM agent generates code that constructs a typed graph representation and computes structural properties such as graph density, variable clustering, and constraint tightness. We evaluate our approach on three combinatorial problems (vehicle routing, car sequencing, fixed-length error-correcting codes) with a portfolio of five state-of-the-art solvers. The synthesized extractors yield algorithm selectors that consistently outperform both expert-curated mzn2feat features (up to $8.3$ percentage points (pp) test-set accuracy on FLECC) and the best transformer-based trans2feat variants. In the meanwhile, the synthesized feature extractors remain inspectable.

Figures & tables

Figure 1: Workflow for generating problem-specific feature extractors. The agent loops over Clear , Insert , Check/Fix , and Execute steps until an executable Python script that produces a validated feature vector is obtained.

Figure 2: Feature correlation matrices for VRP (feature names suppressed for clarity). LLM2feat features show $48.5\%$ lower average correlation ( $|r|=0.221$ ) than mzn2feat ( $|r|=0.429$ ).

Panel of Figure 3 — Figure 3: Feature-importance distribution. LLM2feat achieves better spread of importance across features than mzn2feat on all three problems.

Panel of Figure 4 — Figure 4: AS test accuracy as a function of feature-set size on the three benchmarks. LLM2feat matches or exceeds mzn2feat at every size and reaches higher peak accuracy.

	VRP		CS		FLECC
	$Acc$	$Rank$	$Acc$	$Rank$	$Acc$	$Rank$
SB	79.5%	1.221	49.0%	1.616	78.2%	1.420
mzn2feat +AF	79.5%	1.221	49.0%	1.616	78.2%	1.420
mzn2feat +RF	82.4%	1.195	54.7%	1.943	76.4%	1.426
mzn2feat +LLAMA	82.9%	1.184	59.4%	1.798	78.7%	1.397
mzn2feat +AutoSK	84.4%	1.166	62.1%	1.787	79.2%	1.396

Table 1: Test-set accuracy (

Acc

) and average ranking (

Rank

) for AS toolchains using mzn2feat (

95

features), LLM2feat (

50

features), and trans2feat (

116

features). Loss function

=Acc

(AutoFolio, LLAMA, and trans2feat use their accuracy-based defaults). Each LLM2feat row reports the strongest of the three initial synthesis runs per problem; best/mean/worst over all

10

runs appear in Table 2 .

\times

marks combinations unavailable because no released trans2feat features exist for VRP. Best per column in bold .

Problem	Toolchain	mzn2feat	trans2feat (Best / Mean / Worst)	LLM2feat (Best / Mean / Worst)
CS	RF	54.7%	52.9% / 43.9% / 31.0%	58.3% / 57.8% / 57.3%
CS	AutoSK	62.1%	54.5% / 48.0% / 35.8%	64.0% / 60.5% / 52.8%
FLECC	RF	76.4%	77.3% / 51.1% / 3.1%	84.7% / 83.1% / 81.6%
FLECC	AutoSK	79.2%	78.4% / 68.2% / 8.3%	84.6% / 83.6% / 82.2%

Table 2: Head-to-head with trans2feat : test-set

Acc

aggregated over

10

independent LLM2feat runs and the

20

released trans2feat variants. mzn2feat is a single deterministic extractor. Best/Mean/Worst refer to the best/mean/worst test accuracy across runs within each method; runs without a valid selector (TO/F, Appendix D ) are excluded, leaving

n=17

trans2feat variants and

n=9

LLM2feat runs on CS/AutoSK and

n=14

variants on FLECC/AutoSK.

Table 7

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value	Description
n_estimators	$300$	More trees for complex constraint patterns
max_depth	$20$	Deeper trees capture CP relationships
min_samples_split	$5$
min_samples_leaf	$2$
resampling_strategy	cv
resampling_strategy_arguments	{folds: 5}

Appendix

Table 5: Random Forest hyperparameters.

Parameter	Value
time_left_for_this_task	User-specified ( $300$ – $3600$ s)
per_run_time_limit	time_budget $\,/\,30$ s
initial_configurations_via_metalearning	$25$
ensemble_size	$50$
resampling_strategy	cv
resampling_strategy_arguments	{folds: 5}

Appendix

Table 6: AutoSklearn standard configuration.

Figure 5: Feature correlation matrices for FLECC. LLM2feat features have average absolute correlation $|r|=0.306$ vs. $|r|=0.330$ for mzn2feat .

Figure 6: Feature correlation matrices for CS. LLM2feat features have $|r|=0.551$ vs. $|r|=0.725$ for mzn2feat , a $24\%$ improvement in diversity.

	VRP		CS		FLECC
	Acc	Rank	Acc	Rank	Acc	Rank
SB	78.7%	1.237	49.4%	1.601	80.6%	1.377
mzn2feat +AF	78.7%	1.237	49.4%	1.601	80.6%	1.377
mzn2feat +RF	94.7%	1.057	85.7%	1.339	95.9%	1.063
mzn2feat +LLAMA	92.0%	1.089	87.5%	1.248	91.5%	1.167
mzn2feat +AutoSK	87.2%	1.138	65.0%	1.723	85.0%	1.295

Appendix

Table 7: Training-set accuracy/ranking with

Acc

as loss (problem-specific toolchains).

	VRP		CS		FLECC
	Acc	Rank	Acc	Rank	Acc	Rank
SB	79.5%	1.221	49.0%	1.616	78.2%	1.420
mzn2feat +RF	83.0%	1.184	58.5%	1.680	79.5%	1.391
mzn2feat +AutoSK	84.4%	1.166	62.1%	1.787	79.4%	1.394
LLM2feat +RF	85.7%	1.155	61.3%	1.681	83.5%	1.338
LLM2feat +AutoSK	85.4%	1.160	64.0%	1.738	84.6%	1.310

Appendix

Table 8: Test-set accuracy/ranking with

Rank

as loss.

	VRP		CS		FLECC
	Acc	Rank	Acc	Rank	Acc	Rank
SB	78.7%	1.237	49.4%	1.601	80.6%	1.377
mzn2feat +RF	91.7%	1.091	83.5%	1.231	88.3%	1.232
mzn2feat +AutoSK	87.2%	1.138	65.0%	1.723	85.0%	1.296
LLM2feat +RF	97.1%	1.032	87.9%	1.164	95.9%	1.083
LLM2feat +AutoSK	87.8%	1.133	68.9%	1.618	95.1%	1.103

Appendix

Table 9: Training-set accuracy/ranking with

Rank

as loss.

Extractor	Loss	Train Acc	Test Acc	Train Rank	Test Rank
Single Best (gecode)		0.806	0.782	1.377	1.420
mzn2feat	accuracy	0.959	0.764	1.063	1.426
mzn2feat	ranking	0.883	0.795	1.232	1.391
LLM-20250908123730	accuracy	0.981	0.847	1.029	1.286
LLM-20250908123925	accuracy	0.996	0.818	1.008	1.353
LLM-20250908124149	accuracy	0.994	0.836	1.011	1.315

Appendix

Table 10: LLM2feat +RF performance for FLECC.

Extractor	Loss	Train Acc	Test Acc	Train Rank	Test Rank
Single Best (cplex)		0.494	0.490	1.601	1.616
mzn2feat	accuracy	0.857	0.547	1.339	1.943
mzn2feat	ranking	0.835	0.585	1.231	1.680
LLM-20250908123608	accuracy	0.903	0.577	1.193	1.874
LLM-20250908123905	accuracy	0.913	0.578	1.181	1.861
LLM-20250908124041	accuracy	0.906	0.581	1.202	1.862

Appendix

Table 11: LLM2feat +RF performance for CS.

Extractor	Loss	Train Acc	Test Acc	Train Rank	Test Rank
Single Best (scip)		0.787	0.795	1.237	1.221
mzn2feat	accuracy	0.947	0.824	1.057	1.195
mzn2feat	ranking	0.916	0.830	1.091	1.184
LLM-20250908115627	accuracy	0.993	0.850	1.008	1.169
LLM-20250908121942	accuracy	0.994	0.848	1.007	1.170
LLM-20250908123205	accuracy	0.985	0.853	1.016	1.165

Appendix

Table 12: LLM2feat +RF performance for VRP.

Extractor	Loss	Train Acc	Test Acc	Train Rank	Test Rank
Single Best (gecode)		0.806	0.782	1.377	1.420
mzn2feat	accuracy	0.850	0.792	1.295	1.396
mzn2feat	ranking	0.850	0.794	1.296	1.394
LLM-20250908124149	accuracy	0.952	0.844	1.101	1.308
LLM-20250908123730	accuracy	0.951	0.846	1.104	1.310
LLM-20250908123925	accuracy	0.908	0.831	1.187	1.342

Appendix

Table 13: LLM2feat +AutoSK performance for FLECC.

Extractor	Loss	Train Acc	Test Acc	Train Rank	Test Rank
Single Best (cplex)		0.494	0.490	1.601	1.616
mzn2feat	accuracy	0.650	0.621	1.723	1.787
mzn2feat	ranking	0.650	0.621	1.723	1.787
LLM-20250908124041	accuracy	0.671	0.640	1.646	1.735
LLM-20250908123905	accuracy	0.679	0.634	1.667	1.781
LLM-20250908123608	accuracy	0.686	0.635	1.652	1.782

Appendix

Table 14: LLM2feat +AutoSK performance for CS.

Extractor	Loss	Train Acc	Test Acc	Train Rank	Test Rank
Single Best (scip)		0.787	0.795	1.237	1.221
mzn2feat	accuracy	0.872	0.844	1.138	1.166
mzn2feat	ranking	0.872	0.844	1.138	1.166
LLM-20250908121942	accuracy	0.851	0.852	1.161	1.162
LLM-20250908123205	accuracy	0.880	0.857	1.131	1.157
LLM-20250908115627	accuracy	0.872	0.854	1.138	1.160

Appendix

Table 15: LLM2feat +AutoSK performance for VRP.

Type	Extractor	Features	Accuracy		Avg Rank
			Train	Test	Train	Test
LLM2feat	LLM2feat-1	$50$	0.909	0.583	1.187	1.857
	LLM2feat-2	$50$	0.906	0.581	1.202	1.862
	LLM2feat-3	$50$	0.920	0.579	1.170	1.855
	LLM2feat-4	$50$	0.908	0.579	1.188	1.867
	LLM2feat-5	$50$	0.913	0.578	1.181	1.861

Appendix

Table 16: Car Sequencing: Random Forest with accuracy loss.

Type	Extractor	Features	Accuracy		Avg Rank
			Train	Test	Train	Test
LLM2feat	LLM2feat-1	$50$	0.981	0.847	1.029	1.286
	LLM2feat-2	$50$	0.989	0.842	1.018	1.310
	LLM2feat-3	$50$	0.984	0.842	1.025	1.305
	LLM2feat-4	$50$	0.994	0.836	1.011	1.315
	LLM2feat-5	$50$	0.995	0.835	1.009	1.325

Appendix

Table 17: FLECC: Random Forest with accuracy loss.

Type	Extractor	Features	Accuracy		Avg Rank
			Train	Test	Train	Test
LLM2feat	LLM2feat-1	$50$	0.671	0.640	1.646	1.735
	LLM2feat-2	$50$	0.686	0.635	1.652	1.782
	LLM2feat-3	$50$	0.679	0.634	1.667	1.781
	LLM2feat-4	$50$	0.709	0.630	1.578	1.756
	LLM2feat-5	$50$	0.669	0.629	1.631	1.729

Appendix

Table 18: Car Sequencing: AutoSklearn with accuracy loss. F

=

degenerate failed fit (test accuracy

{<}1\%

), excluded from all aggregates.

Type	Extractor	Features	Accuracy		Avg Rank
			Train	Test	Train	Test
LLM2feat	LLM2feat-1	$50$	0.951	0.846	1.104	1.310
	LLM2feat-2	$50$	0.946	0.845	1.113	1.311
	LLM2feat-3	$50$	0.952	0.844	1.101	1.308
	LLM2feat-4	$50$	0.886	0.838	1.231	1.325
	LLM2feat-5	$50$	0.924	0.837	1.157	1.324

Appendix

Table 19: FLECC: AutoSklearn with accuracy loss. TO

=

1800

s timeout without a model; F

=

degenerate failed fit.

Algorithm Selection

Algorithm selection predicts which solver, model, or strategy will work best for a given instance, context, or computational budget.

See all 50 papers

Using Weisfeiler-Leman Features for Algorithm Selection in Constraint Optimisation
Oct 8, 2026Alessio Pellegrino, Jacopo MauroAlgorithm Selection Constraint Programming
University of Southern Denmark, Denmark
Formalize, Don't Optimize: The Heuristic Trap in LLM-Generated Combinatorial Solvers
May 12, 2026Haoyu Wang, Yuliang Song, Tao Li +5Large Language Model-Guided Optimization LLM-Based Program Synthesis
University of Pennsylvania · University of Toronto · Google DeepMind +1
LLM-Guided Graph Generation for Structure-Based Local Improvement Methods
Aug 13, 2026Hai Xia, Vaidyanathan Peruvemba Ramaswamy, Stefan SzeiderAutomated Heuristic Design Metaheuristic Optimization
Algorithms and Complexity Group, TU Wien, Austria

Oct 8, 2026J/K move · Enter open · S save

Using Weisfeiler-Leman Features for Algorithm Selection in Constraint Optimisation

Alessio Pellegrino, Jacopo Mauro

University of Southern Denmark, Denmark

Algorithm Selection Constraint Programming Automated Feature Engineering Kernel Methods

Open paper AskPDF arXiv

Algorithm Selection is essential for efficient Constraint Programming. Over the years, many algorithm selectors based on machine learning methods have been successfully applied, yet traditional feature extraction methods often rely on manually decided instance-level statistics that fail to capture the underlying problem structure. In this paper we aim to bridge this gap by introducing a novel, automated feature extraction methodology that integrates graph conversion and Weisfeiler-Lehman graph kernels to generate robust structural representations of problem instances. The 1-WL test bounds the graph-distinguishing power of standard message-passing Graph Neural Networks (GNNs), and suitable GNN architectures match this bound \citep{Xuetal2018}. WL-based features offer an alternative that does not require training a GNN. Our primary contribution is a cut-based representation (WLc) designed to model structural partitions and provide a more nuanced predictive signal. We evaluate our approach on instances from the 2023--2025 MiniZinc Challenges across two tasks: maximizing Borda count scores and maximizing predictive accuracy. Experimental results across Support Vector Machines, Random Forests, and Multi-Layer Perceptrons demonstrate that cut-based features outperform fzn2feat with SVMs, while results with RFs and MLPs are closer.

Abstract

Algorithm Selection

Explore similar work

Using Weisfeiler-Leman Features for Algorithm Selection in Constraint Optimisation

Formalize, Don't Optimize: The Heuristic Trap in LLM-Generated Combinatorial Solvers

LLM-Guided Graph Generation for Structure-Based Local Improvement Methods