HAPS: Hierarchical LLM Routing with Joint Architecture and Parameter Search
Organizations: Renmin University of China · Wireless Technology Lab, Huawei Technologies Co., Ltd.
Abstract
Large language model (LLM) routing aims to exploit the specialized strengths of different LLMs for diverse tasks. However, existing approaches typically focus only on selecting LLM architectures, treating candidate models as fixed while overlooking parameter adaptation, which can substantially affect task performance. In this paper, we introduce HAPS, a hierarchical LLM routing framework that jointly searches over model architectures and parameters. Specifically, a high-level router selects among candidate LLM architectures, while a low-level router generates input-conditioned LoRA parameters for the selected architecture, expanding each candidate from a single fixed behavior into a region of reachable behaviors. The low-level router is built on the backbone of the high-level router, so that architecture selection and parameter generation share a common representation of the prompt. We further optimize the whole framework with a reinforcement learning objective. Extensive experiments show that HAPS consistently outperforms strong routing baselines across reasoning- and knowledge-intensive benchmarks, with analyses of its components, design choices, scalability, cost, and mixed open- and closed-source deployment further demonstrating the robustness of these gains. Our code is available at https://github.com/zihangtian/HAPS.
Figures & tables
| Method | HotpotQA (F1) | MMLU (Acc) | ||||
| L-Q | M-Q | L-M | L-Q | M-Q | L-M | |
| Random | 38.94 | 11.44 | 23.26 | 64.48 | 60.00 | 58.97 |
| RouteLLM | 40.67 | 36.10 | 38.95 | 66.55 | 65.86 | 62.76 |
| GraphRouter | 41.31 | 23.07 | 26.75 | 66.90 | 65.52 | 63.10 |
| IRT-Router | 37.34 | 34.09 | 38.92 | 65.52 | 65.17 | 63.79 |
| HAPS | 43.16 | 39.70 | 40.58 | 68.97 | 67.24 | 65.86 |
| Variant | HotpotQA (F1) | MMLU (Acc) | ||||
| L-Q | M-Q | L-M | L-Q | M-Q | L-M | |
| High-Level Router Ablation | ||||||
| w/o High-Level (Random) | 40.13 | 26.37 | 25.44 | 64.83 | 61.03 | 58.62 |
| w/o High-Level (Fixed): | ||||||
| Teacher=Left, Student=Left | 38.48 | 11.55 | 37.84 | 62.07 | 56.21 | 64.48 |
| Teacher=Right, Student=Right | 34.23 | 35.51 | 7.10 | 68.28 | 66.90 | 54.48 |
| Method | Scalability | Training Time | Inference | |||
| F1@K=5 | F1@K=6 | K=5 | K=6 | K=5 (1k) | ||
| Random | 23.28 | 19.43 | -3.85 | – | – | – |
| RouteLLM | 34.09 | 28.05 | -6.04 | 6m 36s | 6m 32s | 52.24s |
| GraphRouter | 35.35 | 33.98 | -1.37 | 6m 04s | 7m 33s | 14.71s |
| HAPS | 37.14 | 41.41 | +4.27 | 5m 19s + 32m 28s | 4m 21s + 21m 37s | 25.60s |
| Method | |||||||||||
| Perf | Cost | Reward | Perf | Cost | Reward | Perf | Cost | Reward | |||
| Static | |||||||||||
| All Weak | 0.3458 | 0.0923 | 0.2582 | 0.3458 | 0.0923 | 0.1267 | 0.3458 | 0.0923 | -0.0047 | ||
| All Strong | 0.3697 | 0.3362 | 0.2285 | 0.3697 | 0.3362 | 0.0168 | 0.3697 | 0.3362 | -0.1950 | ||
| Baselines | |||||||||||
| Random | 0.3433 | 0.2160 | 0.2314 | 0.3433 | 0.2160 | 0.0636 | 0.3433 | 0.2160 | -0.1042 | ||
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | L-Q | M-Q | L-Q-M |
| Random | 45.87 0.52 | 39.53 1.05 | 42.93 0.09 |
| RouteLLM | 46.20 0.75 | 44.93 0.52 | 46.87 0.96 |
| GraphRouter | 47.00 0.71 | 45.33 0.19 | 47.47 0.09 |
| IRT-Router | 46.40 0.28 | 45.80 0.16 | 47.00 0.00 |
| HAPS | 47.40 0.90 | 46.73 0.75 | 49.33 0.74 |
| Question. What was the name of the lawyer who took the main role in the report that helped land Pete Rose a lifetime suspension from baseball? Answer: John M. Dowd | ||
| Round | Frozen base models | With generated parameters |
| 1 | Search[Pete Rose lifetime suspension lawyer] a television film about Pete Rose; no lawyer named Teacher: [Rethink] | Search[Pete Rose lifetime suspension lawyer] same paragraph Teacher: [Rethink] |
| 2 | Search[Pete Rose lifetime suspension lawyer] identical query repeated same paragraph Teacher: [Rethink] | Search[lawyer in Pete Rose report] query rephrased “The Dowd Report … was prepared by Special Counsel to the Commissioner, John M. Dowd …” Teacher: [Continue] |
| 3 | Search[Pete Rose lifetime suspension lawyer] identical query repeated same paragraph Teacher: [Rethink] | Finish[John M. Dowd] |
| 4 | Finish[Judge Kenesaw Mountain Landis] name absent from all retrieved text | – |
| Final | wrong answer F1 | John M. Dowd (correct) F1 |
LLM Routing
Large language model routing selects models, tools, experts, or inference strategies to balance answer quality, cost, latency, and deployment constraints.
See all 380 papers