Optional reasoning, retrieval, web search, and tool use can improve large language model responses but incur unnecessary computation when activated indiscriminately. This study investigates whether prompt optimization can compensate for limited router capacity and whether classification gains translate into downstream efficiency. The proposed framework uses prompt-driven small language models to make four independent capability decisions, allowing multiple capabilities to be selected per query. Qwen2.5 routers ranging from 1.5B to 14B parameters are compared across five prompting strategies, followed by an evaluation of capability-specific optimized configurations using a fixed GPT-OSS-120B downstream model on 127 questions. Under minimal prompting, increasing router capacity does not consistently improve classification performance. Prompt optimization narrows the capacity gap for reasoning and calculator routing, whereas RAG and web search retain larger benefits from higher capacity. Similar classification F1 scores do not necessarily imply equivalent downstream behavior: false-positive activations increase unnecessary computation, while missed capabilities can reduce answer quality. The 14B router achieves a mean BERTScore F1 of 0.8408, compared with 0.8418 when all capabilities are enabled, while reducing average downstream token consumption by 68.7% (8,837 to 2,764 tokens) and median downstream latency by 80.5% (9.79 to 1.91 seconds). These savings exclude router overhead, which is measured separately in a different inference environment and increases with router capacity. The findings demonstrate that the relationship between router capacity, prompt design, and downstream efficiency is capability-dependent. Consequently, router selection must consider error profiles and routing overhead alongside classification performance.
Figures & tables
Figure 1. Capability-routing workflow. Four binary predictions control downstream reasoning effort, document retrieval, web search, and calculator availability. Retrieved context and available tools are supplied to a fixed downstream LLM. RQ3 evaluates reference-answer similarity, downstream token consumption, and latency, with router overhead reported separately. An input query passes through four capability classifiers, producing binary decisions for reasoning, document retrieval, web search, and calculator access. These decisions configure the inputs and settings of a fixed downstream language model. The resulting response is evaluated for reference-answer similarity, token consumption, and latency, while router overhead is measured separately.
Capability
Total
Train
Validation
Test
Reasoning
875
94
29
752
RAG
1,003
100
30
873
Web Search
642
100
30
512
Calculator
995
100
30
865
Total
3,515
394
119
3,002
Table 1. Final RQ1/RQ2 capability-classification dataset and splits.
Capability
Overlap
Matching
Different
Agreement (%)
Calculator
83
78
5
93.98
RAG
87
87
0
100.00
Reasoning
80
53
27
66.25
Web Search
87
61
26
70.11
Table 2. Agreement between overlapping RQ2 and RQ3 capability labels before harmonization.
Capability
Required
Not Required
Reasoning
83
44
RAG
97
30
Web Search
10
117
Calculator
29
98
Table 3. Final corrected RQ3 ground-truth capability distribution.
Figure 2. RQ1 and RQ2 capability-classification workflow. Each input question is evaluated across router-model and prompt-strategy configurations for reasoning, RAG, web search, and calculator use, and predictions are compared with the corresponding ground-truth labels. An input question is classified for four capabilities: reasoning, retrieval-augmented generation, web search, and calculator use. Router model sizes and prompting strategies are varied, and the predicted capability labels are compared with reference annotations to evaluate classification performance.
Size
Reasoning
RAG
Web
Calculator
1.5B
Few-Shot
GEPA
GEPA
SME
3B
Few-Shot
GEPA
GEPA
Minimal
7B
Few-Shot
GEPA
GEPA
SME
14B
Few-Shot
GEPA
GEPA
Few-Shot
Table 4. Prompting strategies used for each router size and capability in RQ3.
Figure 3. RQ3 end-to-end routing workflow. The selected configuration determines whether reasoning, RAG, web search, and calculator capabilities are enabled before the resulting context is passed to the fixed downstream LLM. An input question is processed under a baseline, oracle, or SLM-routed capability configuration. The configuration controls reasoning effort, document retrieval, web search, and calculator access for a fixed downstream language model. Generated responses are evaluated for reference-answer similarity, token consumption, and latency.
Capability
Metric
1.5B
3B
7B
14B
Calculator
F1
0.644
0.928
0.917
0.888
Balanced Accuracy
0.644
0.936
0.924
0.903
RAG
F1
0.678
0.602
0.196
0.579
Balanced Accuracy
0.509
0.604
0.484
0.624
Reasoning
F1
0.745
0.771
0.809
0.768
Balanced Accuracy
0.500
0.569
0.663
0.559
Table 5. RQ1 performance under the Minimal Baseline prompt.
Capability
1.5B
3B
7B
14B
Reasoning
0
0
0
0
RAG
0
0
0
0
Web Search
0
0
0
0
Calculator
0
0
0
24
Table 6. Unknown predictions under the Minimal Baseline prompt.
Capability
1.5B
3B
7B
14B
Calculator
0.905
0.928
0.934
0.941
RAG
0.712
0.657
0.870
0.955
Reasoning
0.871
0.865
0.855
0.913
Web Search
0.651
0.639
0.834
0.868
Table 7. Best RQ2 F1 score for each capability and router size.
Method
1.5B
3B
7B
14B
Minimal Baseline
0
0
0
24
Descriptive SME
0
0
0
6
DSPy Baseline
358
6
0
0
DSPy Few-Shot
53
6
0
1
GEPA
32
6
0
1
Table 8. Unknown predictions by prompting strategy and router model.
Figure 4. Best F1 score and median router latency for the highest-performing prompt configuration at each capability and model size.
Capability
Metric
1.5B
3B
7B
14B
Reasoning
F1
0.607
0.676
0.740
0.716
BA
0.556
0.698
0.684
0.755
RAG
F1
0.580
0.328
0.715
0.953
BA
0.661
0.598
0.778
0.908
Web Search
F1
0.243
0.143
0.118
0.194
BA
0.715
0.537
0.524
0.573
Table 9. Router classification performance on the corrected RQ3 benchmark.
Figure 5. Mean downstream BERTScore F1 across routing conditions.
Figure 6. Average downstream token consumption across routing conditions.
Figure 7. Median downstream latency across routing conditions.
Router
Capability
TP
TN
FP
FN
Unknown
Errors
1.5B
Reasoning
44
25
18
39
0
1
RAG
40
27
3
55
2
0
Web Search
9
62
55
1
0
0
Calculator
27
31
65
2
2
0
3B
Reasoning
46
37
7
37
0
0
RAG
19
30
0
78
0
0
Table 10. Confusion counts for capability predictions on the corrected RQ3 benchmark. Unknown and error counts account for predictions excluded from the confusion matrix.
Figure 8. Median inference latency of the four SLM router configurations.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Component
Configuration
Router models
qwen2.5:1.5b , qwen2.5:3b , qwen2.5:7b , and qwen2.5:14b ( Yang et al., 2024 ) , served through Ollama.
Ollama
Version 0.40.4; local API endpoint 127.0.0.1:11434 .
DSPy
Version 3.4.1 ( Khattab et al., 2024 ) .
Raw router inference
Temperature = 0.0; maximum generated tokens ( num_predict ) = 10.
DSPy router inference
Temperature = 0.0; maximum generated tokens = 64.
Few-Shot construction
Thirty examples were drawn from the 100-example training set to form the Few-Shot candidate pool. The DSPy configuration permitted up to 6 labeled demonstrations, up to 4 bootstrapped demonstrations, and 4 candidate programs.
Appendix
Table 11. Experimental software, model, and inference configuration.
Component
Configuration
Web Search
Tavily search with search_depth="advanced" , maximum 5 results, and include_answer=False .
Calculator
Structured calculator tool exposed to the downstream model through a JSON tool schema requiring a single arithmetic expression string. Expressions were evaluated with a restricted Python AST evaluator.
Downstream model
openai.gpt-oss-120b-1:0 through AWS Bedrock.
Downstream generation
Temperature = 0.0; maximum generated tokens = 2048 for the standard downstream configuration.
BERTScore
BERTScore ( Zhang et al., 2020 ) with roberta-large ; final scoring was executed on GPU. Additional options not explicitly retained in the experiment metadata are reported as unavailable.
Router hardware
Google Colab with an NVIDIA T4 GPU. Other Colab hardware characteristics were not fixed and are therefore not reported.
Appendix
Table 12. Experimental software, model, and inference configuration (continued).
Large language model routing selects models, tools, experts, or inference strategies to balance answer quality, cost, latency, and deployment constraints.