When an LLM-based agent improves on a task, is the gain from the model itself or from the reasoning paradigm wrapped around it? We study this question by comparing six inference-time paradigms, namely Direct, CoT, ReAct, Plan-Execute, Reflection, and ReCode, across four frontier LLMs and ten benchmarks, yielding roughly 18,000 runs. We find that reasoning structure helps dramatically on some tasks but hurts on others: ReAct improves over Direct by 44pp on GAIA, while CoT degrades performance by 15pp on HumanEval. No single paradigm dominates, and oracle per-task selection beats the best fixed paradigm by 17.1pp on average. Motivated by this complementarity, we propose a select-then-solve approach: before answering each task, a lightweight embedding-based router selects the most suitable paradigm. Across four models, the router improves average accuracy from 47.6% to 53.1%, outperforming the best fixed paradigm at 50.3% by 2.8pp and recovering up to 37% of the oracle gap. In contrast, zero-shot self-routing only works for GPT-5 at 67.1% and fails for weaker models, all trailing the learned router. Our results argue that reasoning paradigm selection should be a per-task decision made by a learned router, not a fixed architectural choice.
Figures & tables
Figure 1: Direct prompting (gray), best single paradigm per dataset (colored), and oracle per-task selection (blue) for GPT-5. The best paradigm differs across tasks, and the oracle substantially exceeds any fixed choice, motivating our select-then-solve approach.
Paradigm
Key Idea
Control
Tools
LM Calls
Budget
Direct
Free-form answer
None
No
1
—
CoT
Step-by-step reasoning
Instructed
No
1
—
ReAct
Thought–action loop
Orchestrated
Yes
1–15
Tmax=15
Plan-Execute
Plan, then act
Orchestrated
Yes
2–16
no replanning
Reflection
Critique and revise
Orchestrated
Yes
3–45
Mmax=3
ReCode
Code as reasoning
Substrate
Yes
2–10
hierarchical
Table 1: Reasoning paradigms studied in the main paper, organized by the degree of reasoning control and tool access.
Model
Paradigm
HEval
M500
AIME
HPQA
NQ
MMLU
HLE
GAIA
SEAL
τ -b
Avg
GPT-5
Direct
85.0
86.0
90.0
62.0
37.0
88.0
20.0
28.0
16.0
5.9
51.8
CoT
70.0
84.0
86.7
58.0
26.0
89.0
16.0
40.0
34.0
9.8
51.3
ReAct
86.0
80.0
86.7
70.0
27.0
89.0
26.0
72.0
28.0
5.9
57.1
Plan-Exec
35.0
75.0
90.0
69.0
27.0
83.0
22.0
68.0
48.0
7.8
52.5
Reflect.
85.0
69.0
75.0
67.0
29.0
85.0
22.0
58.0
46.0
5.9
54.2
ReCode
95.0
86.0
88.3
61.0
35.0
84.0
20.0
38.0
10.0
7.8
52.5
Table 2: Complete success rates (%) across all models, paradigms, and datasets. Best and second best paradigm per (model, dataset) are highlighted. Avg is the unweighted mean across all ten datasets.
Model
Direct
Best-Single
Oracle
Oracle Gap
GPT-5
60.3%
61.8% (ReAct)
74.1%
+12.4pp
Gemini-Flash
48.0%
56.1% (CoT)
75.2%
+19.1pp
Qwen3-Max
41.3%
46.5% (ReAct)
72.5%
+21.6pp
Qwen3-30B
21.9%
31.7% (ReAct)
55.7%
+24.1pp
Average
42.9%
49.0%
69.4%
+17.1pp
Table 3: Oracle analysis: per-task best paradigm selection achieves substantially higher accuracy than any single paradigm. Oracle Gap = Oracle − Best-single.
Figure 2: The select-then-solve pipeline. A Paradigm Selector encodes, classifies, and routes each task to one of six paradigms. Only the selected paradigm is executed.
Figure 3: Router comparison across four models. The embedding router (green) consistently outperforms Direct and Best-single baselines. Self-routing (red) shows mixed results. The oracle (blue) shows substantial remaining headroom.
Model
Direct
CoT
ReAct
Plan-Exec
ReCode
Reflect.
Router
GPT-5
65.9
9.6
14.4
7.9
1.3
0.9
Gemini
73.4
13.5
6.5
0.4
6.1
0.0
Qwen3-Max
58.9
19.7
2.6
7.0
10.9
0.9
Qwen3-30B
47.2
26.2
15.7
3.1
6.6
1.3
Self
GPT-5
65.1
18.8
15.3
0.4
0.4
0.0
Gemini
72.1
10.9
16.6
0.0
0.4
0.0
Table 4: Paradigm distribution (%) predicted by the learned router vs. zero-shot self-routing on the test set. Self-routing shows degenerate biases; the learned router produces diverse, model-adapted distributions.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Benchmark
Domain
Eval Instances
Metric
Notes
HumanEval
Code
100
Execution
function completion
MATH500
Math
100
Numeric
multi-step derivation
AIME
Math
60
Numeric
olympiad-style problems
HotpotQA
QA
100
Text match
multi-hop QA
NQ
QA
100
Text match
factoid QA
MMLU
Knowledge
100
Letter match
broad subject coverage
Appendix
Table 5: Evaluation sets used in the paper. “Eval instances” refers to the number of tasks actually run per model-paradigm pair.
Paradigm
Avg LM Calls
Avg Tool Calls
Tools?
Multi-turn?
Key Mechanism
Direct
1.0
0
No
No
Single-shot generation
CoT
1.0
0
No
No
Step-by-step verbal reasoning
ReAct
1–15
0–14
Yes
Yes
Thought-action-observation loop
Plan-Exec
2–16
0–15
Yes
Yes
Plan first, then execute with tools
Reflection
3–45
0–42
Yes
Yes
Solve, critique, revise (up to 3 rounds)
ReCode
2–10
0–8
Yes
Yes
Recursive code decomposition
Appendix
Table 6: Detailed comparison of the six paradigms. LM calls and tool calls are per-task averages on GPT-5.
Method
GPT-5
Gemini
Qwen3-Max
Qwen3-30B
Avg
Base
Direct
60.3
55.5
49.8
24.9
47.6
Best-single
62.4
55.5
50.7
32.8
50.3
HC
Handcrafted (LR)
59.4
58.5
51.5
34.9
51.1
Handcrafted (MLP)
62.4
55.5
53.3
37.1
52.1
Emb
Embedding (LR)
64.2
61.9
53.3
32.3
52.9
Combined (LR)
64.2
61.0
54.6
32.8
53.1
Appendix
Table 7: Downstream task accuracy (%) on the held-out test set. Each router selects a paradigm per task (including Direct); accuracy is the fraction of tasks solved correctly under the selected paradigm. Best router per model is bolded .
Figure 4: Cost-effectiveness scatter plot: success rate vs. average tokens per task. Points closer to the top-left corner represent better cost-efficiency.
Figure 5: Router comparison across four models. The embedding router (green) consistently outperforms Direct and Best-single baselines. Self-routing (red) fails to improve over Direct on most models. The oracle (blue) shows substantial remaining headroom.
Figure 6: Oracle gap recovery across models. Each group shows the progression from Direct (gray) through handcrafted router (orange) and embedding router (green) to the Oracle upper bound (blue). The embedding router consistently narrows the gap, with the largest recovery on Gemini.
Figure 7: Paradigm distribution comparison: learned router (left) vs. zero-shot self-routing (right). The learned router produces diverse, model-adapted distributions, while self-routing shows degenerate biases (GPT-5 selects Direct 100%; Qwen3-30B over-selects ReAct at 48%).
Figure 8: Success rate heatmap for GPT-5 across paradigms (rows) and datasets (columns). Clear task-paradigm interactions are visible: no single row dominates all columns.
Figure 9: Jaccard similarity between paradigm success sets (aggregated across all models). Lower values indicate greater complementarity; paradigms solve different subsets of tasks.