ADAPT: Adaptive Decision and Abstention with Pool-Size-Aware Thresholds for Service Routing
Organizations: Open Source Institute, Faculty of Science and Technology, University of Canberra, Canberra, Australia.
Abstract
Retrieval-based classifiers efficiently route queries to specialised services, but they typically select a candidate even when no suitable service exists. We introduce Adaptive Decision and Abstention with Pool-Size-Aware Thresholds for Service Routing (ADAPT), a lightweight routing architecture with an explicit fallback option that enables queries to abstain from specialist selection. ADAPT uses the separation between the highest relevance score and the candidate score distribution as routing confidence. Experiments across retrieval models and increasing numbers of services reveal a consistent empirical behaviour: the confidence distribution changes systematically with candidate-pool size K, making fixed abstention thresholds increasingly miscalibrated. ADAPT addresses this effect with a K-aware threshold that adapts as services are added without classifier retraining. Evaluation across two specialist datasets shows that this simple mechanism provides an effective accuracy-cost trade-off compared with generative LLM routing, supporting lightweight and dynamically extensible service-routing architectures.
Figures & tables
| Pool size | Specialists |
| 3 | Abstract algebra, anatomy, astronomy |
| 5 | Pool of 3 plus business ethics, college computer science |
| 8 | Pool of 5 plus high school chemistry, high school geography, jurisprudence |
| 10 | Pool of 8 plus world religions, formal logic |
| 12 | Pool of 10 plus marketing, sociology |
| 14 | Pool of 12 plus high school psychology, prehistory |
| Family | Router | ||||||
| Bi-encoder | bge-large-en-v1.5 | 91.67 | 89.00 | 82.82 | 80.77 | 71.27 | 69.00 |
| Qwen3-Embedding-0.6B | 83.00 | 78.75 | 71.27 | 69.77 | 64.67 | 62.18 | |
| Qwen3-Embedding-4B | 89.67 | 75.12 | 65.64 | 64.46 | 57.53 | 54.76 | |
| Qwen3-Embedding-8B | 88.67 | 83.00 | 74.55 | 71.69 | 66.13 | 63.06 | |
| all-MiniLM-L6-v2 | 88.83 | 80.50 | 74.73 | 73.62 | 69.53 | 66.82 | |
| ColBERT | LFM2-ColBERT-350M | 93.33 | 85.12 | 81.45 | 77.38 | 71.60 | 69.59 |
| Acc. | Spec. Recall | Gen. Recall | |||||||
| Bi-encoder | bge-large-en-v1.5 | 72.71 | 67.50 | 61.39 | 58.57 | 33.89 | 35.93 | 96.00 | 99.33 |
| Qwen3-Embedding-8B | 65.83 | 65.17 | 61.81 | 61.43 | 47.22 | 42.78 | 77.00 | 95.00 | |
| ColBERT | Reason-ModernColBERT | 63.33 | 64.33 | 61.67 | 62.50 | 52.22 | 55.93 | 70.00 | 74.33 |
| LFM2-ColBERT-350M | 72.92 | 70.67 | 63.33 | 62.50 | 58.33 | 54.26 | 81.67 | 77.33 | |
| Cross-encoder | mxbai-rerank-large-v2 | 80.42 | 80.00 | 74.17 | 73.33 | 65.56 | 62.78 | 89.33 | 92.33 |
| Spec. | Gen. | Spec. | Gen. | |||||||||
| Model | Med. | MMLU | Med. | MMLU | Med. | MMLU | Med. | MMLU | ||||
| bge-large-en-v1.5 | 47.78 | 33.89 | 87.00 | 96.00 | 48.33 | 35.93 | 91.00 | 99.33 | ||||
| Qwen3-Embedding-8B | 47.22 | 47.22 | 77.00 | 77.00 | 39.26 | 42.78 | 96.67 | 95.00 | ||||
| Reason-ModernColBERT | 52.22 | 52.22 | 68.00 | 70.00 | 54.26 | 55.93 | 81.00 | 74.33 | ||||
| LFM2-ColBERT-350M | 58.89 | 58.33 | 81.33 | 81.67 | 55.19 | 54.26 | 70.67 | 77.33 | ||||
| Fixed | -aware | ||||||||||
| Family | Router | Acc. | Spec. | Gen. | Acc. | Spec. | Gen. | Acc. | Spec. | Gen. | |
| Bi-encoder | bge-large-en-v1.5 | 3 | 91.83 | 89.33 | 94.33 | 91.67 | 89.33 | 94.00 | |||
| 5 | 88.50 | 88.60 | 88.33 | 89.00 | 86.80 | 92.67 | |||||
| 8 | 80.00 | 83.12 | 71.67 | 82.82 | 80.38 | 89.33 | |||||
| 10 | 78.15 | 83.10 | 61.67 | 80.77 | 78.20 | 89.33 | |||||
| 12 | 68.93 | 76.25 | 39.67 | 71.27 | 70.75 | 73.33 | |||||
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Specialist recall | General recall | Balanced accuracy | |
| 85 | 0.868 | 0.863 | 0.865 |
| 90 | 0.852 | 0.921 | 0.886 |
| 95 | 0.821 | 0.968 | 0.894 |
| 99 | 0.748 | 0.991 | 0.870 |
Adaptive Model Routing
Adaptive model routing selects models or specialized experts per input to balance prediction quality, computational cost, latency, and resource constraints.
See all 102 papers