AlgoPilot: Cross-Paradigm Reasoning in Language Models via Strategy Selection and Guidance

Yilun Hao; Ruixiao Yang; Hudson Hilal; Yongchao Chen; Kaizhi Qian; Yada Zhu; Yang Zhang; Chuchu Fan

cs.AINov 2, 2026

AlgoPilot: Cross-Paradigm Reasoning in Language Models via Strategy Selection and Guidance

Authors: Yilun Hao, Ruixiao Yang, Hudson Hilal, Yongchao Chen, Kaizhi Qian, Yada Zhu, Yang Zhang, Chuchu Fan

Organizations: Massachusetts Institute of Technology · Harvard University · MIT-IBM Watson AI Lab

Abstract

Real-world problems span diverse domains, including program synthesis, symbolic reasoning, planning, and optimization, and often require fundamentally different solution paradigms. A central challenge for LLM-based reasoning is twofold: identifying an appropriate problem-solving approach and formulating it correctly. In this paper, we propose AlgoPilot, a cross-paradigm reasoning framework that enables adaptive selection of the solving approaches and structured problem formulation prior to execution. The AlgoPilot framework contains three components: 1) a SteerLM, trained via supervised fine-tuning (SFT) and reinforcement learning (RL), that adaptively selects the solving strategy (paradigm and modes) based on input problems, 2) the corresponding expert GuideLM, trained via SFT, that generates structured formulation guidance tailored to the selected strategy, and 3) an ExecutionLM that generates solutions conditioned on both the selected strategy and its formulation. Unlike prompting-based methods that rely on fixed reasoning templates, our approach learns to adaptively choose and structure problem-solving strategies based on the input. We build AlgoPilot on Qwen3-8B and evaluate across 5 problem categories and 13 benchmarks with 66 unique domains, comparing against both the latest similarly scaled small models and large LLMs. AlgoPilot significantly improves average accuracy for the same model from 38.4% to 72.9%, outperforming the best large LLM prompting baseline GLM-5 by 11.7%. Ablation studies confirm the effectiveness of each component. Notably, we show that the learned steering and formulation modules can transfer across execution models, suggesting their potential to improve other LLMs. Project page: https://sites.google.com/view/algopilot.

Figures & tables

Figure 1: An overview of AlgoPilot. Given a natural-language problem input, SteerLM first selects a solving paradigm and guidance mode. If guidance is needed, the corresponding paradigm-specific GuideLM generates formulation guidance. ExecuteLM then produces and executes the final solution conditioned on the selected paradigm and optional guidance.

Figure 2: Training data construction for AlgoPilot. We first generate and filter candidate guidances with execution feedback to train paradigm-specific GuideLMs. The trained GuideLMs are then used in cross-paradigm evaluation, where different paradigm-mode combinations are scored and ranked to construct supervision and rewards for SteerLM.

Figure 3: Distribution of the empirically best solving paradigm across problem categories (test set). For each category, the donut shows what fraction of solvable examples are best addressed by each paradigm family.

Large models
Model / Method	Code	CoT	PDDL	SMT	MILP	Avg.
GLM-5 ${}_{\textsc{Direct}}$	91.6	55.4	54.6	62.2	42.1	61.2
GLM-5 ${}_{\textsc{Reason}}$	92.5	29.6	46.0	82.2	47.7	59.6
Qwen3-Coder-Next ${}_{\textsc{Direct}}$	90.5	46.3	1.9	55.6	32.7	45.4
Qwen3-Coder-Next ${}_{\textsc{Reason}}$	83.3	40.4	7.9	62.9	43.9	47.7
Small-model baselines

Table 1: Accuracy (%) comparison of AlgoPilot with baselines across five categories. Bold numbers indicate the best result in each column, excluding the not comparable Top-3 analysis row.

Variant	Steer source	Guide source	Code	CoT	PDDL	SMT	MILP	Avg.
V1	None	None	67.0	42.8	0.0	63.7	18.3	38.4
V2	Self	None	71.0	26.4	0.0	62.2	20.7	36.1
V3	Self	GuideLM	54.0	69.1	38.6	38.5	26.4	45.3
V4	SteerLM	None	67.7	72.7	25.6	26.1	36.5	45.7
AlgoPilot ${}_{\textsc{SFT+GRPO}}$	SteerLM	GuideLM	72.9	75.8	91.5	64.6	59.6	72.9
Oracle1	Oracle	None	80.7	80.8	11.0	76.2	28.8	55.5

Table 2: Accuracy (%) comparison of AlgoPilot with ablation variants. Bold numbers indicate the best result in each column, excluding the not comparable oracle rows.

Method & Input	Code	CoT	PDDL	SMT	MILP	Avg.
AlgoPilot ${}_{\textsc{SFT+GRPO}}$ , Original	72.9	75.8	91.5	64.6	59.6	72.9
AlgoPilot ${}_{\textsc{SFT+GRPO}}$ , Paraphrased	69.7	74.2	90.5	50.7	58.0	68.6

Table 3: End-to-end accuracy (%) on original and paraphrased inputs.

Method	Code	CoT	SMT	MILP	PDDL	Avg.
Qwen3-8B ${}_{\textsc{Direct}}$	74.7	63.0	54.7	39.7	0.0	53.0
Qwen3-8B ${}_{\textsc{Reason}}$	92.0	67.3	74.3	56.0	0.0	66.2
AlgoPilot ${}_{\textsc{SFT+GRPO}}$	85.1	71.3	73.7	54.2	99.1	73.3

Table 4: Accuracy (%) on held-out domains, grouped by paradigm. Scores within each paradigm and overall are weighted by the number of problem instances.

Model / Method	Code	CoT	PDDL	SMT	MILP	Avg.
Gemma-4-E4B-it ${}_{\textsc{Direct}}$	72.4	32.5	2.5	58.6	29.1	39.0
Gemma-4-E4B-it ${}_{\textsc{Reason}}$	61.8	74.0	1.3	76.6	40.1	50.8
Gemma-4-E4B-it w/ AlgoPilot ${}_{\textsc{SFT+GRPO}}$	78.3	79.5	57.5	42.1	45.1	60.5

Table 5: Accuracy (%) comparison of transferring AlgoPilot directly to Gemma-4-E4B-it with its original prompting performance.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Reward Design	Top-1 Acc.	Top-3	NDCG@3	Valid-3
$A_{1}$	73.89	77.13	57.81	10.7
$A_{2}$	73.78	83.67	75.25	84.7
$A_{3}$	74.01	83.58	75.01	94.8
$A_{4}$	74.10	83.78	76.07	100.0

Appendix

Table 6: Reward-design ablation results for routing quality and valid structured generation.

Candidate construction	Best-route Recall@3	NDCG@3	Best candidate score
$A_{1}$	79.60	57.81	71.34
$\mathrm{Pad}_{1}$	83.17	61.66	73.41
$\mathrm{Pad}_{2}$	88.83	70.92	76.77
$\mathrm{Pad}_{3}$	89.19	72.89	77.08
$A_{2}$	91.72	75.25	78.99

Appendix

Table 7: Candidate-padding controls for

A_{1}

, compared with the learned candidate sets from

A_{2}

Selected route	Same-paradigm Direct better	Any-paradigm Direct better
Guided Code	14.14	25.72
Guided CoT	20.72	27.88
Guided PDDL	0.00	0.00
Guided SMT	2.38	21.43
Guided MILP	1.22	4.88
Overall	11.70	18.54

Appendix

Table 8: Frequency (%) with which a Direct alternative outperforms a selected Guided route across three executions. Each row is conditioned on selections of the indicated Guided route; Overall pools all Guided selections.

Method	Code	CoT	PDDL	SMT	MILP	Avg.
SteerLM + QA-SFT ExecuteLM , no GuideLM	57.7	55.1	87.6	28.5	67.4	59.3
AlgoPilot	72.9	75.8	91.5	64.6	59.6	72.9

Appendix

Table 9: Comparison with direct QA-based supervised fine-tuning. All values are accuracy (%).

Method	Code	CoT	PDDL	SMT	MILP	Avg.
Paradigm-specific GuideLM s	72.9	75.8	91.5	64.6	59.6	72.9
Unified GuideLM	67.8	72.5	69.0	30.7	62.4	60.5

Appendix

Table 10: Comparison of specialist and unified GuideLM s. All values are accuracy (%).

Model	Mode	Code	CoT	PDDL	SMT	MILP	Avg.
GLM-5	Self-routing	91.1	86.6	46.0	82.9	47.5	70.8
	Heuristic routing	90.0	88.2	64.1	55.3	59.8	71.5
	Template-direct	85.2	88.1	61.7	41.0	46.9	64.6
Qwen3-Coder-Next	Self-routing	83.1	66.7	6.6	78.1	45.9	56.1
	Heuristic routing	93.5	90.7	28.6	67.0	63.0	68.6
	Template-direct	85.6	90.5	7.9	49.8	58.8	58.5

Appendix

Table 11: Comparison with additional prompting and routing baselines. All values are accuracy (%). Avg. is the mean across the five paradigms.

Breakdown	Category	Top-1 agreement (%)
Problem category	Code	96.04
	CoT	96.60
	MILP	98.94
	PDDL	100.00
	SMT	100.00
Paraphrase style	Lexical reformulation	98.33

Appendix

Table 12: Top-1 agreement across problem categories and paraphrase styles.

Breakdown	Category	Original accuracy (%)	Paraphrased accuracy (%)
Problem category	Code	72.86	69.74
	CoT	75.78	74.19
	MILP	59.58	57.97
	PDDL	91.46	90.47
	SMT	64.45	50.73
	Overall	74.10	68.62

Appendix

Table 13: End-to-end accuracy comparison of paraphrased versus original inputs.

Category	Domain	Source benchmark	Status	# data points	AlgoPilot	Qwen3 8B Direct	Qwen3 8B Reason
Code	Algorithms	BIG-Bench Hard	Seen benchmark; held-out domain	46	87.68	56.52	88.41
Code	Penguins in a Table	BIG-Bench Hard	Seen benchmark; held-out domain	50	82.67	91.33	95.33
CoT	Sports Understanding	BIG-Bench Hard	Seen benchmark; held-out domain	50	50.67	30.00	47.33
CoT	Temporal Sequences	BIG-Bench Hard	Seen benchmark; held-out domain	50	92.00	96.00	87.33
SMT	FOLIO	FOLIO	Unseen benchmark	50	62.67	55.33	68.00
SMT	SATBench	SATBench	Unseen benchmark	50	84.67	54.00	80.67

Appendix

Table 14: Generalization to held-out domains and unseen benchmarks. All accuracy values are percentages.

Large models
Model / Method	Code	CoT	PDDL	SMT	MILP
GLM-5 ${}_{\textsc{Direct}}$	91.6 $\pm$ 1.0	55.4 $\pm$ 0.4	54.6 $\pm$ 0.8	62.2 $\pm$ 0.6	42.1 $\pm$ 2.0
GLM-5 ${}_{\textsc{Reason}}$	92.5 $\pm$ 0.8	29.6 $\pm$ 0.8	46.0 $\pm$ 0.7	82.2 $\pm$ 0.6	47.7 $\pm$ 1.2
Qwen3-Coder-Next ${}_{\textsc{Direct}}$	90.5 $\pm$ 0.1	46.3 $\pm$ 0.5	1.9 $\pm$ 0.7	55.6 $\pm$ 1.6	32.7 $\pm$ 0.4
Qwen3-Coder-Next ${}_{\textsc{Reason}}$	83.3 $\pm$ 0.7	40.4 $\pm$ 0.7	7.9 $\pm$ 0.7	62.9 $\pm$ 2.3	43.9 $\pm$ 1.3
Small-model baselines

Appendix

Table 15: Accuracy (%) comparison of AlgoPilot with baselines across five categories, with mean ± standard deviation across three samples/runs.

Prompt	Model	Prompt (M)	Completion (M)	Total (M)
direct	GLM-5	3.14	3.17	6.31
direct	Qwen3-Coder	3.21	11.83	15.04
direct	Gemma-4-E4B-it	3.34	7.32	10.67
direct	Qwen3.5-9B	3.35	9.59	12.93
reason	GLM-5	3.35	5.93	9.28
reason	Qwen3-Coder	3.42	20.15	23.57

Appendix

Table 16: Total token usage (in millions) per model–prompt configuration.

Prompt	Model	Code	CoT	MILP	PDDL	SMT
direct	GLM-5	410	19	824	1,168	184
direct	Qwen3-Coder	1,241	466	2,642	4,517	741
direct	Gemma-4-E4B-it	1,832	50	1,494	1,523	520
direct	Qwen3.5-9B	892	57	2,601	4,340	195
reason	GLM-5	718	522	1,225	1,372	870
reason	Qwen3-Coder	2,382	792	6,655	6,309	1,217

Appendix

Table 17: Average completion tokens per LLM call, by paradigm and configuration.

Stage	Calls	Prompt (M)	Completion (M)	Total (M)
SteerLM prediction	2,211	0.78	0.03	0.81
GuideLM generation	5,616	1.97	3.90	5.88
ExecuteLM runs	6,633	6.69	2.80	9.49
Total	14,460	9.44	6.73	16.17

Appendix

Table 18: Evaluation-time token usage of AlgoPilot across all five test categories.

Model	Code	CoT	MILP	PDDL	SMT
AlgoPilot ${}_{\textsc{SFT+GRPO}}$	724	450	1049	2470	926

Appendix

Table 19: Average completion tokens per LLM call of AlgoPilot

{}_{\textsc{SFT+GRPO}}

, by paradigm.

	Wrap with code block		Accuracy (%)
Model	direct	reason	Text-only	Code block
GLM-5	5.3%	71.2%	83.9	5.5
Qwen3-Coder	76.4%	94.1%	67.9	36.7

Appendix

Table 20: Fraction of CoT runs that produce a Python code block instead of a direct textual answer, and accuracy conditioned on output format ( reason prompt).

direct
Model	No domain/problem	Planner: no plan	Wrong plan	Total failures
GLM-5	218	389	0	607
Qwen3-Coder	941	371	0	1,312
reason
GLM-5	511	211	0	722
Qwen3-Coder	667	565	0	1,232

Appendix

Table 21: PDDL failure decomposition (per run, both prompts pooled for Qwen3-Coder).

Failure mode	Count	Fraction of failures
Format mismatch	11	0.6%
Execution error	157	9.1%
Execution timeout	15	0.9%
Generation mismatch	0	0.0%
Incorrect answer	1,535	89.3%

Appendix

Table 22: AlgoPilot failure decomposition over all failed execution runs.

Adaptive Reasoning in Language Models

See all 49 papers

Route-and-Generate: KL-Regularized Router Improvement for Reasoning Routing in LLMs
Oct 21, 2026Tien-Phat Nguyen, Truong Nguyen, Thanh-Thien Le +4Adaptive Reasoning in Language Models Efficient Language Model Reasoning
Hanoi University of Science and Technology, Vietnam · Monash University, Australia · New York University, USA +1
Heuristic Classification of Thoughts Prompting (HCoT): Integrating Expert System Heuristics for Structured Reasoning into Large Language Models
Apr 14, 2026Lei Lin, Jizhao Zhu, Yong Liu +3Structured Reasoning in Language Models Constraint-Guided Reasoning in Language Models
Computer Network Information Center, Chinese Academy of Sciences, Beijing, China · Shenyang Aerospace University · ZGC LAB, Beijing,China +1
Select-then-Solve: Paradigm Routing as Inference-Time Optimization for LLM Agents
Apr 8, 2026Heng Zhou, Zelin Tan, Zhemeng Zhang +15LLM Agent Workflow Optimization LLM Agents

Oct 21, 2026J/K move · Enter open · S save

Route-and-Generate: KL-Regularized Router Improvement for Reasoning Routing in LLMs

Tien-Phat Nguyen, Truong Nguyen, Thanh-Thien Le, Hoang Phan+3

Hanoi University of Science and Technology, Vietnam · Monash University, Australia · New York University, USA+1

Adaptive Reasoning in Language Models Efficient Language Model Reasoning LLM Routing RL for Language Model Reasoning

Open paper AskPDF arXiv

Large language models often apply complex reasoning indiscriminately, leading to overthinking on simple tasks and underthinking on harder ones. We present Route-and-Generate (Rouge), a hierarchical training framework that enables LLMs to adaptively decide when and how much reasoning is required for each input. Our approach decouples reasoning effort allocation from answer generation by introducing a routing policy that allocates a fixed sampling budget across multiple reasoning formats, while the base model is optimized with GRPO using per-sample rewards. To promote efficient reasoning, we employ a rank-based reward that favors shorter correct solutions, and update the routing policy via a closed-form KL-regularized policy improvement step distilled into a neural router. This design encourages concise answers on easy problems while reserving longer reasoning for harder ones, achieving improved accuracy-efficiency trade-offs across multiple reasoning benchmarks.

Abstract

Adaptive Reasoning in Language Models

Explore similar work

Route-and-Generate: KL-Regularized Router Improvement for Reasoning Routing in LLMs

Heuristic Classification of Thoughts Prompting (HCoT): Integrating Expert System Heuristics for Structured Reasoning into Large Language Models

Select-then-Solve: Paradigm Routing as Inference-Time Optimization for LLM Agents