OrchSLM: Probing the Dynamics of Small Language Model Orchestration

Chengxi Zhang; Yu Yao

cs.AISep 11, 2026

OrchSLM: Probing the Dynamics of Small Language Model Orchestration

Authors: Chengxi Zhang, Yu Yao

Organizations: MIT

Abstract

Although large language models (LLMs) have demonstrated remarkable capabilities, their reliance on cloud-scale infrastructure poses fundamental challenges for deployment in agentic pipelines, including latency, privacy, connectivity, and substantial computational cost. Small language models (SLMs) offer a compelling alternative: recent studies suggest that many repetitive and narrowly scoped subtasks in agentic workloads may be better served by specialized SLMs than by monolithic LLMs. However, the limited capacity and context windows of SLMs can constrain long-horizon reasoning and interaction-heavy orchestration strategies such as iterative verification and debate. This motivates a complementary, non-interactive paradigm in which heterogeneous SLMs independently generate candidate solutions and a router orchestrates their cached samples without further model interaction. To further understand the mechanisms of such orchestration, we introduce OrchSLM, a routing framework that unifies existing non-interactive orchestration methods and exposes their underlying design choices as controllable parameters. Using OrchSLM as a systematic probe, we reveal how orchestration behavior emerges from diverse knobs, including the task structure, model-pool composition, and multi-agent consensus.

Figures & tables

Figure 1: Different routers can pick different answers from the same cached outputs.

Method	Examples	Trust	Aggregation	Position in family
Expert selection	Query routing, cascades, model routers	Expert	Single	$(1,\infty,-)$
Majority voting	Self-consistency voting and multi-agent majority	Average	Pooled sum	$(0,0,-)$
Weighted voting	Cross-validation weighted probabilistic ensemble	Expert	Pooled sum	$(0,\kappa>0,-)$
Self-consistency	SLM-MUX-style answer-level routing	Average	Max	$(1,0,\mathrm{Max})$
Weighted consistency	Validation-aware consistency routing	Expert	Max	$(1,\kappa>0,\mathrm{Max})$
Cross consistency	Answer-level cross-confidence and support variants	Average	Max + Pooled	$(0<\alpha<1,0,\mathrm{Max})$

Table 2: Common routers mapped to

(\alpha,\kappa,\mathrm{Agg})

in the generalized family.

Figure 2: Existing routers as fixed corners of the

(\alpha,\kappa,\mathrm{Agg})

generalized family; axes are defined in Section 2.1 .

Figure 3: Accuracy heatmaps over $(\alpha,\kappa)$ for Sum and Max aggregation on the full pool. Dashed contour: cells within $1$ pp of the best score; stars: best per task.

Task	Equal	Expert	Rel. $\Delta$
MATH-500	0.7338	0.7938	+8.2%
GSM8K	0.9052	0.9166	+1.3%
GPQA	0.3909	0.4328	+10.7%
MMLU-Pro	0.6304	0.6402	+1.6%

Table 4: Experts vs average models. Rel.

\Delta

is the relative change from the best equal-weight cell (

\kappa=0

) to the best full-sweep cell.

Task	Sum	Max	Rel. $\Delta$
MATH-500	0.7938	0.7936	0.0%
GSM8K	0.9089	0.9166	+0.8%
GPQA	0.4318	0.4328	+0.2%
MMLU-Pro	0.6398	0.6402	+0.1%

Table 5: Majority vs coalition support, vs strongest supporter. Rel.

\Delta

is the relative change from Sum to Max aggregation.

Figure 6: Pool-size scan. Left: routed accuracy vs. size (gray: each subset; lines: oracle, single-best, best generalized). Right: best-cell

(\alpha,\kappa)

shifts with pool size.

Figure 7: Per-model Atr (heard) vs. Ctr (helpful) under each canonical router.

Appendix figures & tables62 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Best acc.	$\alpha$	$\kappa$	Agg.	Interpretation
MATH-500	0.7938	0.6842	7.0171	Sum	single-best-like expert routing
GSM8K	0.9166	1.0000	7.0171	Max	expert self-consistency
GPQA	0.4328	0.3684	8.0000	Max	expert + support-aware routing
MMLU-Pro	0.6402	0.1053	5.3617	Max	expert-weighted pooled support

Appendix

Table 8: Task-specific routing regimes from the full-pool generalized sweep. The table summarizes broad preferred regions; near-best plateau statistics are reported separately.

Benchmark	Best acc.	Best params	Cells @0.25pp	Area @0.25pp	Cells @0.5pp	Area @0.5pp	Cells @1pp	Area @1pp
GPQA	0.4328	Max a=0.3684 k=8.0	19	0.0238	78	0.0975	158	0.1975
GSM8K	0.9166	Max a=1.0 k=7.0171	15	0.0187	39	0.0488	210	0.2625
MATH-500	0.7938	Sum a=0.6842 k=7.0171	137	0.1713	199	0.2487	273	0.3412
MMLU-Pro	0.6402	Max a=0.1053 k=5.3617	38	0.0475	213	0.2662	566	0.7075

Appendix

Table 9: Full-pool plateau summary. Broad near-best regions support interpreting best cells as regimes with stable neighboring cells.

Benchmark	Best gen.	Single-best	Gap	Best $\kappa$
MATH-500	0.7938	0.7924	0.0014	7.0171
GSM8K	0.9166	0.9089	0.0077	7.0171
GPQA	0.4328	0.4015	0.0313	8.0000
MMLU-Pro	0.6402	0.6158	0.0244	5.3617

Appendix

Table 10: Expert-prior axis: gap between the best generalized full-pool cell and the single-best limit.

Benchmark	Best $\kappa{=}0$	Std	Agg.	$\alpha$	Full-family best	Gap
MATH-500	0.7338	0.0055	Max	0.7368	0.7938	-0.0600
GSM8K	0.9052	0.0013	Max	0.8947	0.9166	-0.0114
GPQA	0.3909	0.0120	Sum	0.6316	0.4328	-0.0419
MMLU-Pro	0.6304	0.0069	Sum	0.2632	0.6402	-0.0098

Appendix

Table 11: No-expert-prior comparison on the full seven-model pool, read from the

\kappa=0

best cell in the full-pool phase diagram.

Benchmark	Best $\alpha{=}0$	Agg.	$\kappa$	Full-family best	Gap
GPQA	0.4318	Sum	7.0171	0.4328	-0.0010
GSM8K	0.9086	Sum	8.0000	0.9166	-0.0080
MATH-500	0.7934	Sum	8.0000	0.7938	-0.0004
MMLU-Pro	0.6398	Sum	5.3617	0.6402	-0.0004

Appendix

Table 12: Top-answer-term removal ablation on the full seven-model pool. Setting

\alpha=0

keeps only pooled answer support.

Benchmark	Best $\alpha{=}1$	Agg.	$\kappa$	Full-family best	Gap
GPQA	0.4308	Sum	6.1416	0.4328	-0.0020
GSM8K	0.9166	Max	7.0171	0.9166	0.0000
MATH-500	0.7936	Sum	6.1416	0.7938	-0.0002
MMLU-Pro	0.6350	Sum	5.3617	0.6402	-0.0052

Appendix

Table 13: Pooled-term removal ablation on the full seven-model pool. Setting

\alpha=1

removes pooled mutual support.

benchmark	benchmark_label	pool_size	mean_sum_minus_max	frac_sum_wins
gpqa	GPQA	1.0000	0.0000	0.0000
gpqa	GPQA	2.0000	0.0000	0.0000
gpqa	GPQA	3.0000	-0.0002	0.2857
gpqa	GPQA	4.0000	0.0004	0.5143
gpqa	GPQA	5.0000	-0.0002	0.4286
gpqa	GPQA	6.0000	-0.0006	0.2857

Appendix

Table 14: Sum/Max summary. The aggregation switch measures whether support is distributed across models or concentrated in the strongest supporter.

Benchmark	Disagreement bin	$N$	Accuracy
GPQA	(-0.01, 0.2]	80	0.5250
GPQA	(0.2, 0.4]	214	0.4346
GPQA	(0.4, 0.6]	587	0.4889
GPQA	(0.6, 0.8]	795	0.4050
GPQA	(0.8, 1.01]	304	0.3717
GSM8K	(-0.01, 0.2]	8882	0.9895

Appendix

Table 15: Full-pool best generalized accuracy by pair-disagreement bin.

Benchmark	Target	Pool	Size	Best gen.	Std	$\Delta$ vs SLM	$\Delta$ vs SB	Params	Regime
GPQA	best acc.	qwen+llama+gemma+ministral	4	0.4429	0.0093	0.0212	0.0414	Sum a=0.1053 k=8.0	high-kappa expert/single-best-like
GPQA	best gain	qwen+tulu+dsmath+phi	4	0.4333	0.0182	0.1101	0.0318	Sum a=1.0 k=3.4968	mixed support-aware
GSM8K	best acc.	qwen+llama+dsmath	3	0.9327	0.0018	0.0060	0.0238	Sum a=0.4211 k=4.6669	high-kappa expert/single-best-like
GSM8K	best gain	qwen+gemma+ministral	3	0.9111	0.0021	0.0227	0.0022	Max a=0.9474 k=8.0	no stable gain
MATH-500	best acc.	qwen+llama+dsmath+gemma	4	0.7952	0.0053	0.0558	0.0028	Max a=0.0526 k=6.1416	no stable gain
MATH-500	best gain	qwen+llama+tulu+dsmath+gemma+phi+ministral	7	0.7938	0.0049	0.0696	0.0014	Sum a=0.6842 k=7.0171	no stable gain

Appendix

Table 16: Pool-composition targets. For each benchmark we report the pool with highest absolute accuracy and the pool with largest gain over the self-consistency router.

Benchmark	Best absolute pool	Size	Best acc.	Full-pool best
MATH-500	qwen+llama+dsmath+gemma	4	0.7952	0.7938
GSM8K	qwen+llama+dsmath	3	0.9327	0.9166
GPQA	qwen+llama+gemma+ministral	4	0.4429	0.4328
MMLU-Pro	qwen+llama+dsmath+phi	4	0.6526	0.6402

Appendix

Table 17: Best absolute pool from the all-subset pool-composition scan. The companion figure shows how accuracy changes with pool size, while this table records the selected model names.

Benchmark	Regime	Pools	Frac.
GPQA	high-alpha Max self-consistency-like	8	0.063
GPQA	high-kappa expert/single-best-like	72	0.567
GPQA	low/moderate-alpha Sum coalition/Majority-like	12	0.094
GPQA	mixed support-aware	14	0.110
GPQA	no stable gain	21	0.165
GSM8K	high-alpha Max self-consistency-like	3	0.024

Appendix

Table 18: Regime classification over all scanned pools. Fractions are within benchmark.

benchmark_label	predictor	response	pearson_r
GPQA	pool_size	delta_best_vs_slm	0.6762
GPQA	pool_size	delta_best_vs_single_best	0.5491
GPQA	pool_size	delta_best_vs_majority	0.3595
GPQA	pool_size	best_gen_acc_mean	0.5517
GPQA	oracle_gap_mean	delta_best_vs_slm	0.6899
GPQA	oracle_gap_mean	delta_best_vs_single_best	0.5402

Appendix

Table 19: Structure-predictor correlations between pool metrics and generalized-routing gains.

Benchmark	Variant	Eval acc.	Eval std	Pool size	Delta vs SC	Delta vs single	Delta vs majority
GPQA	A_full_pool_select_scorer	0.4120	0.0309	7.0000	0.0760	0.0054	0.0253
GPQA	B_predefined_curated_select_pool_scorer	0.4167	0.0281	3.6200	0.0807	0.0101	0.0300
GPQA	C_all_subset_select_pool_scorer	0.4075	0.0319	3.9800	0.0715	0.0009	0.0208
GSM8K	A_full_pool_select_scorer	0.9149	0.0071	7.0000	0.0114	0.0047	0.0112
GSM8K	B_predefined_curated_select_pool_scorer	0.9311	0.0066	3.0000	0.0277	0.0210	0.0275
GSM8K	C_all_subset_select_pool_scorer	0.9306	0.0070	3.0600	0.0271	0.0204	0.0269

Appendix

Table 20: Split-selected evaluation. Variants differ in whether they select only scorer parameters on the full pool, choose from predefined curated pools, or choose from all subsets.

Benchmark	Pool	Comparison	$N$	Fix	Break	Target acc.	Base acc.	Net
GPQA	full	vs slm	1980	341	151	0.4328	0.3369	0.0960
GPQA	full	vs single	1980	155	93	0.4328	0.4015	0.0313
GPQA	full	vs Majority	1980	199	107	0.4328	0.3864	0.0465
GPQA	best_abs	vs slm	1980	139	97	0.4429	0.4217	0.0212
GPQA	best_abs	vs single	1980	163	81	0.4429	0.4015	0.0414
GPQA	best_abs	vs Majority	1980	108	59	0.4429	0.4182	0.0247

Appendix

Table 21: Fix/break summary for full and best-absolute pools. Counts are over all K20R10 question-replicate rows.

Benchmark	Pool role	Baseline	$\Delta$	Corrected $p$
GPQA	best_abs	SLM-MUX_best_abs	0.0212	0.009
GPQA	best_abs	single-best_best_abs	0.0414	0.000
GPQA	best_abs	Majority_best_abs	0.0247	0.001
GPQA	best_abs	cross slice_best_abs	0.0268	0.000
GPQA	best_gap	SLM-MUX_best_gap	0.1101	0.000
GPQA	best_gap	single-best_best_gap	0.0318	0.003

Appendix

Table 22: Selected-pool paired tests with max-stat correction across baselines within each benchmark/pool role.

Benchmark	Pool role	Method	Mean acc.	Std
GPQA	best_abs	Majority	0.4182	0.0144
GPQA	best_abs	Single-Best	0.4015	0.0170
GPQA	best_abs	SLM-MUX	0.4217	0.0122
GPQA	best_gap	Majority	0.3581	0.0155
GPQA	best_gap	Single-Best	0.4015	0.0170
GPQA	best_gap	SLM-MUX	0.3232	0.0203

Appendix

Table 23: Replicate stability. Means and standard deviations are over the 10 K20R10 subsampling replicates.

Benchmark	Split-selected acc.	$\Delta$ vs self-consistency	K=50 best gen.	K=50 self-consistency
MATH-500	0.7859	+0.0668	0.8000	0.7240
GSM8K	0.9311	+0.0277	0.9189	0.9052
GPQA	0.4167	+0.0807	0.4394	0.3030
MMLU-Pro	0.6375	+0.0653	0.6480	0.5580

Appendix

Table 24: Robustness checks. Split-selected results use held-out evaluation with predefined curated pools; K=50 uses all cached samples once. The direction of the regimes persists under both checks.

benchmark	benchmark_label	pool_role	pool_id	prior_variant	best_acc	best_agg	best_kappa	best_alpha
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	softmax_log_accuracy	0.7938	Sum	7.0171	0.6842
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	power_accuracy	0.7938	Sum	7.0171	0.6842
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	centered_logit	0.7938	Sum	2.1786	0.7368
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	rank_based	0.7936	Sum	5.3617	0.9474
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	uniform	0.7338	Max	0.0000	0.7368
gsm8k	GSM8K	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	softmax_log_accuracy	0.9166	Max	7.0171	1.0000

Appendix

Table 25: Expert-prior transform variants. These checks test whether the qualitative regimes depend on a single prior transformation.

benchmark	benchmark_label	pool_role	pool_id	tie_break	accuracy_mean	accuracy_std
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	pooled_lex	0.7938	0.0049
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	lex	0.7938	0.0049
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	fixed_model_order	0.7938	0.0049
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	random	0.7938	0.0049
gsm8k	GSM8K	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	pooled_lex	0.9166	0.0017
gsm8k	GSM8K	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	lex	0.9166	0.0017

Appendix

Table 26: Tie-break sensitivity. Tie-breaking changes small details while preserving the main regime interpretation.

benchmark	benchmark_label	pool_role	pool_id	lambda	accuracy_mean	accuracy_std
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	0.0	0.7296	0.0043
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	0.16666666666666666	0.7304	0.0042
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	0.25	0.7334	0.0059
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	0.5	0.7280	0.0044
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	1.0	0.7236	0.0071
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	2.0	0.7202	0.0070

Appendix

Table 27: Exact cross-support bridge sweep. This sweep studies how the bridge varies with cross-support strength.

Benchmark	Best acc.	Agg.	$\alpha$	$\kappa$	Majority	SC	Single-best
MATH-500	0.8000	Sum	0.6316	7.0171	0.7200	0.7240	0.7980
GSM8K	0.9189	Max	1.0000	8.0000	0.9045	0.9052	0.9105
GPQA	0.4394	Max	0.6842	4.0481	0.3737	0.3030	0.4091
MMLU-Pro	0.6480	Sum	0.8947	0.0000	0.6280	0.5580	0.6180

Appendix

Table 28: K=50 distribution-level phase diagram. K=50 uses all cached samples once and serves as a robustness check for the main protocol.

benchmark	benchmark_label	pool_role	pool_id	K_eff	R	method	accuracy_mean	accuracy_std	accuracy_min	accuracy_max
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	1.0000	50	majority	0.6561	0.0105	0.6340	0.6780
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	1.0000	50	slm_mux	0.6561	0.0105	0.6340	0.6780
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	1.0000	50	cc_like	0.6561	0.0105	0.6340	0.6780
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	1.0000	50	single_best	0.7312	0.0111	0.7020	0.7540
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	1.0000	50	best_gen_step3_cell	0.7315	0.0111	0.7020	0.7540
math500	MATH-500	full	qwen+llama+tulu+dsmath+gemma+phi+ministral	1.0000	50	best_sum_step3_cell	0.7315	0.0111	0.7020	0.7540

Appendix

Table 29: Effective sample-budget scaling under K20R-style subsampling.

benchmark	method_a	method_b	prediction_agreement	correctness_corr
gpqa	single_best_full	majority_full	0.5955	0.4504
gpqa	single_best_full	slm_mux_full	0.5247	0.3339
gpqa	single_best_full	cc_like_full	0.5732	0.3978
gpqa	single_best_full	best_gen_full	0.7813	0.7442
gpqa	single_best_full	oracle_full	0.4015	0.4282
gpqa	majority_full	slm_mux_full	0.7869	0.7271

Appendix

Table 30: Method-correlation analysis. High correlation indicates that two methods draw on overlapping evidence.

removed_model	GPQA	GSM8K	MATH-500	MMLU-Pro
dsmath	-0.0015	-0.0004	0.0002	0.0002
gemma	0.0076	-0.0003	-0.0002	-0.0100
llama	-0.0035	0.0036	0.0008	0.0054
ministral	-0.0035	-0.0008	0.0000	-0.0006
phi	-0.0066	-0.0002	-0.0006	0.0110
qwen	0.0485	0.0099	0.1118	0.0176

Appendix

Table 31: Leave-one-out model contribution summary.

Benchmark	Comparison	Pool	Size	Best gen.	$\Delta$ vs SLM
GPQA	contains_llama	qwen+llama+gemma+ministral	4	0.4429	0.0212
GPQA	no_llama	qwen+gemma+phi	3	0.4429	0.0152
GSM8K	contains_llama	qwen+llama+dsmath	3	0.9327	0.0060
GSM8K	no_llama	qwen+dsmath	2	0.9231	0.0016
MATH-500	contains_llama	qwen+llama+dsmath+gemma	4	0.7952	0.0558
MATH-500	no_llama	qwen+dsmath+gemma	3	0.7944	0.0546

Appendix

Table 32: Llama inclusion analysis comparing the best pool containing Llama with the best pool excluding Llama.

Benchmark	Setting	Model	Atr	Ctr
MATH-500	Full	qwen	0.8283	0.1220
MATH-500	Full	llama	0.5022	0.0014
MATH-500	Full	tulu	0.4817	0.0002
MATH-500	Full	dsmath	0.4392	0.0002
MATH-500	Full	gemma	0.4951	0.0002
MATH-500	Full	phi	0.4637	-0.0004

Appendix

Table 33: Per-model Atr/Ctr diagnostics for the full pool and the selected curated pool in each benchmark. Atr is support assigned to the routed answer; Ctr is the leave-one-out counterfactual contribution defined in § 2 .

Benchmark	Pool	Model	Atr	Correct part	Wrong part	Balance	Precision
GPQA	best_abs	qwen	0.6093	0.2862	0.3231	-0.0368	0.4698
GPQA	best_abs	gemma	0.5304	0.2480	0.2824	-0.0343	0.4676
GPQA	best_abs	llama	0.4682	0.2105	0.2578	-0.0473	0.4496
GPQA	full	qwen	0.6139	0.2814	0.3326	-0.0512	0.4583
GPQA	full	gemma	0.5169	0.2419	0.2750	-0.0332	0.4679
GPQA	full	phi	0.4828	0.2148	0.2680	-0.0532	0.4449

Appendix

Table 34: Accepted-support decomposition. For each benchmark and pool role, we show the three models with highest Atr under best generalized routing. Balance is correct accepted support minus wrong accepted support.

Benchmark	Best single	Best single acc.	Majority	SLM-MUX	Cross-support	Single-Best
MATH-500	Qwen	0.7660	0.7220	0.7200	0.7080	0.7680
GSM8K	Qwen	0.9022	0.8999	0.9158	0.9022	0.9037
GPQA	Gemma	0.4091	0.3636	0.3434	0.3535	0.3788

Appendix

Table 35: Six-model exploratory baseline summary.

Benchmark	Majority	SC	Cross-support	Single-best	Best gen.	Best $\alpha$	Best $\kappa$	Agg.
MATH-500	0.7060	0.7000	0.7040	0.7680	0.7760	0.3000	4.0000	Max
GSM8K	0.8999	0.9007	0.9014	0.9037	0.9105	1.0000	0.5000	Max
GPQA	0.3737	0.3535	0.3687	0.3990	0.4343	0.6000	4.0000	Sum

Appendix

Table 36: Six-model fixed-prior sweep summary.

Benchmark	Rank	Pool	Size	Best gen.	Single-best	$\Delta$ vs SB	$\Delta$ vs SLM	Oracle gap	Regime
MATH-500	1	qwen+phi	2	0.7780	0.7680	0.0100	0.0280	0.0140	low/moderate-alpha Sum coalition/Majority-like
MATH-500	2	qwen+tulu+dsmath+phi+ministral	5	0.7780	0.7680	0.0100	0.0660	0.0560	high-kappa expert/single-best-like
MATH-500	3	qwen+tulu+ministral	3	0.7760	0.7680	0.0080	0.0340	0.0340	high-kappa expert/single-best-like
GSM8K	1	qwen+tulu+dsmath	3	0.9219	0.9037	0.0182	0.0045	0.0569	low/moderate-alpha Sum coalition/Majority-like
GSM8K	2	qwen+dsmath+phi	3	0.9189	0.9037	0.0152	0.0099	0.0584	high-kappa expert/single-best-like
GSM8K	3	qwen+dsmath	2	0.9174	0.9037	0.0136	0.0038	0.0478	low/moderate-alpha Sum coalition/Majority-like

Appendix

Table 37: Six-model top pool-composition results by best generalized accuracy.

Benchmark	Pool	Strategy	Majority	SC	Cross-support	Single-best	Best gen.	Std	Modal best params
GPQA	six-model	K20 once	0.3932	0.3422	0.3730	0.4020	0.4444	0.0087	Sum a=0.9 k=2 (2/20)
GPQA	six-model	K20R10 mean	0.3821	0.3172	0.3750	0.4078	0.4467	0.0081	Max a=0.9 k=2 (6/20)
GPQA	six-model	K=50 full	0.3838	0.3081	0.3788	0.4091	0.4495	0.0000	Max a=0.9 k=2 (1/1)
GPQA	seven-model	K20 once	0.3929	0.3409	0.3727	0.4020	0.4394	0.0118	Sum a=0.0 k=8 (3/20)
GPQA	seven-model	K20R10 mean	0.3798	0.3096	0.3674	0.4078	0.4379	0.0058	Max a=0.9 k=2 (4/20)
GPQA	seven-model	K=50 full	0.3737	0.3030	0.3535	0.4091	0.4343	0.0000	Sum a=0.2 k=8 (1/1)

Appendix

Table 38: Sampling-strategy comparison. K20R10 preserves replicate variability while using the larger cache.

Benchmark	Best single	Best single	Majority	SLM-MUX	Cross slice	Oracle
MATH-500	qwen	0.7924 $\pm$ 0.0048	0.7246 $\pm$ 0.0073	0.7242 $\pm$ 0.0053	0.7302 $\pm$ 0.0040	0.8536 $\pm$ 0.0067
GSM8K	qwen	0.9089 $\pm$ 0.0024	0.9032 $\pm$ 0.0018	0.9031 $\pm$ 0.0018	0.9050 $\pm$ 0.0015	0.9760 $\pm$ 0.0011
GPQA	qwen	0.4015 $\pm$ 0.0162	0.3864 $\pm$ 0.0069	0.3369 $\pm$ 0.0155	0.3727 $\pm$ 0.0087	0.7854 $\pm$ 0.0116
MMLU-Pro	qwen	0.6158 $\pm$ 0.0058	0.6282 $\pm$ 0.0055	0.5738 $\pm$ 0.0085	0.6130 $\pm$ 0.0045	0.8420 $\pm$ 0.0042

Appendix

Table 39: Seven-model K20R10 baseline summary. Standard deviations are across the 10 subsampling replicates.

Benchmark	Majority	SC	Cross-support	Single-best	Best gen.	Best $\alpha$	Best $\kappa$	Agg.
MATH-500	0.7246	0.7242	0.7302	0.7924	0.7938	0.6842	7.0171	Sum
GSM8K	0.9032	0.9031	0.9050	0.9089	0.9166	1.0000	7.0171	Max
GPQA	0.3864	0.3369	0.3727	0.4015	0.4328	0.3684	8.0000	Max
MMLU-Pro	0.6282	0.5738	0.6130	0.6158	0.6402	0.1053	5.3617	Max

Appendix

Table 40: Seven-model K20R10 fixed-prior sweep summary.

Benchmark	Pool	Size	Best gen.	Single-best	SC	Oracle	Agg.
MATH-500	qwen+llama+tulu+gemma	4	0.7952	0.7924	0.7452	0.8366	Max
MATH-500	qwen+llama+dsmath+gemma	4	0.7952	0.7924	0.7394	0.8380	Max
MATH-500	qwen+llama+gemma	3	0.7950	0.7924	0.7504	0.8294	Sum
MATH-500	qwen+llama+tulu+dsmath+gemma	5	0.7948	0.7924	0.7360	0.8434	Sum
MATH-500	qwen+llama	2	0.7944	0.7924	0.7860	0.8174	Sum
GSM8K	qwen+llama+dsmath	3	0.9327	0.9089	0.9267	0.9660	Sum

Appendix

Table 41: Seven-model top pool-composition results, with five best pools per benchmark by generalized accuracy.

Benchmark	Rows	AUC $(c)$	AUC $(s)$	AUC $(c{+}s)$	$\rho(c,y)$	$\rho(s,y)$
MATH-500	29780	0.820	0.904	0.901	0.574	0.706
GSM8K	79140	0.754	0.945	0.936	0.438	0.581
GPQA	11280	0.525	0.600	0.574	0.043	0.164
GPQA ( $K{=}3$ )	11088	0.531	0.605	0.584	0.055	0.172

Appendix

Table 42: Row-level correctness prediction quality for isolated self-consistency

c

, cross-support

s

, and

c{+}s

Setting	SLM	Cross real	Cross q-shuf	Cross m-shuf	$\Delta$ real
GPQA / 6-model / K=5	0.343	0.354	0.345	0.320	0.010
GPQA / 6-model / K=3	0.332	0.363	0.341	0.317	0.031
GPQA / qwen+phi+gemma / K=5	0.394	0.429	0.404	0.406	0.035
MATH-500 / 6-model / K=5	0.720	0.708	0.704	0.692	-0.012
GSM8K / 6-model / K=5	0.916	0.902	0.908	0.892	-0.014

Appendix

Table 43: Placebo tests that corrupt cross-support while leaving self-consistency fixed.

Setting	Best variant	Acc.	$\Delta$ vs SLM
GPQA / 6-model / K=3	s_only	0.388	0.056
GPQA / 6-model / K=5	s_only	0.379	0.035
GPQA / qwen+phi+gemma / K=5	rank_sum	0.439	0.045
MATH-500 / 6-model / K=5	c_times_s	0.722	0.002
GSM8K / 6-model / K=5	c_only	0.916	0.000

Appendix

Table 44: Best score form in each setting from the score-form ablation.

Setting	Support definition	$s$ -only	$\Delta$ vs SLM
GPQA / 6-model / K=3	top2_strong	0.396	0.064
GPQA / 6-model / K=5	top2_strong	0.419	0.076
GPQA / qwen+phi+gemma / K=5	majority_fraction	0.414	0.020

Appendix

Table 45: Best support definition by

s

-only accuracy.

Setting	Best configuration	Acc.	$\Delta$ vs SLM
GPQA / 6-model / K=5	s_only (rank)	0.384	0.040
GPQA / qwen+phi+gemma / K=5	c+s (zscore,rank)	0.449	0.056
MATH-500 / 6-model / K=5	c+s (minmax,minmax)	0.720	0.000

Appendix

Table 46: Best normalisation / calibration configuration in each setting.

Benchmark	$K$	AUC $(c)$	AUC $(s)$	AUC $(c{+}s)$	Entropy	Stable-wrong SLM
GPQA	1	0.500	0.599	0.599	1.218	64.6
GPQA	2	0.522	0.608	0.582	1.401	64.9
GPQA	3	0.531	0.605	0.584	1.460	66.8
GPQA	4	0.535	0.602	0.583	1.485	66.7
GPQA	5	0.525	0.600	0.574	1.506	65.7
MATH-500	1	0.500	0.944	0.944	1.272	27.6

Appendix

Table 47: How the routing signals change with the sample budget

K

Benchmark	$K$	SLM	Cross	$\Delta$	Fix	Break
MATH-500	1	0.724	0.703	-0.022	116	224
MATH-500	2	0.708	0.694	-0.014	94	166
MATH-500	3	0.711	0.703	-0.009	81	124
MATH-500	4	0.713	0.708	-0.005	76	101
MATH-500	5	0.720	0.708	-0.012	60	120
GSM8K	1	0.886	0.897	0.011	405	254

Appendix

Table 48: Accuracy scaling with the per-model sample budget

K

on the full 6-model pool.

Setting	Routed correct	No oracle	Stable-but-wrong	No support	Other
GPQA / 6-model / K=5	34.3	24.2	31.8	4.5	5.1
GPQA / qwen+phi+gemma / K=5	39.4	34.8	16.7	5.1	4.0
MATH-500 / 6-model / K=5	72.0	16.6	9.8	0.6	1.0
GSM8K / 6-model / K=5	91.6	2.9	2.4	2.2	1.0

Appendix

Table 49: Oracle-gap decomposition of routing outcomes.

Pair	Pair oracle	Inc. gain	Error overlap	Wrong-answer agr.
qwen + gemma	0.566	0.167	0.434	0.477
tulu + gemma	0.561	0.162	0.439	0.471
gemma + phi	0.556	0.157	0.444	0.409
qwen + tulu	0.545	0.167	0.455	0.544
qwen + ministral	0.535	0.157	0.465	0.533

Appendix

Table 50: Top GPQA model pairs by pairwise oracle. All top pairs contain Gemma.

Setting	Bin	Views	SLM	Cross	$s$ -only	$\Delta$ Cross-SLM
GPQA / 6-model / K=3	q1	426	0.434	0.467	0.460	0.033
GPQA / 6-model / K=3	q2	323	0.359	0.399	0.424	0.040
GPQA / 6-model / K=3	q3	317	0.341	0.404	0.423	0.063
GPQA / 6-model / K=3	q4	914	0.271	0.287	0.329	0.015
GPQA / 6-model / K=5	q1	430	0.442	0.465	0.442	0.023
GPQA / 6-model / K=5	q2	380	0.368	0.474	0.500	0.105

Appendix

Table 51: Question-slice analysis on GPQA by model disagreement.

Benchmark	$\beta_{\text{dom}}$	$\beta_{\text{oracle}}$	$\beta_{\text{size}}$	$R^{2}$	$n$
MATH-500	0.015	0.223	-0.005	0.669	57
GSM8K	0.125	0.270	-0.004	0.655	57
GPQA	0.072	0.212	-0.013	0.106	57

Appendix

Table 52: OLS coefficients for predicting

\Delta=\text{Cross}-\text{SLM}

from subset-level pool statistics.

Pool	$n$	Cross	$\Delta$ vs core	Oracle	$\Delta$ oracle
qwen_phi_gemma	3	0.439	0.000	0.652	0.000
qwen_tulu_phi_gemma	4	0.384	-0.056	0.712	0.061
qwen_phi_ministral_gemma	4	0.429	-0.010	0.702	0.051
qwen_phi_gemma_dsmath	4	0.374	-0.066	0.682	0.030
qwen_tulu_phi_gemma	4	0.384	-0.056	0.712	0.061
qwen_tulu_phi_ministral_gemma	5	0.364	-0.076	0.747	0.096

Appendix

Table 53: GPQA contamination analysis anchored at the curated core pool “qwen_phi_gemma“.

Comparison	$N$	$\Delta$	95% CI	$p$
GPQA / 6-model / K=5 : cross-support vs slm_mux	198	0.010	[-0.035, 0.056]	0.822
GPQA / 6-model / K=3 : cross-support vs slm_mux	198	0.031	[-0.008, 0.068]	0.115
GPQA / qwen+phi+gemma / K=5: cross-support vs slm_mux	198	0.005	[-0.030, 0.045]	1.000
GPQA / qwen+phi+gemma / K=5: cross-support vs single_best_sc	198	0.040	[-0.020, 0.101]	0.249
MATH-500 / 6-model / K=5: cross-support vs slm_mux	500	-0.012	[-0.028, 0.004]	0.238
GSM8K / 6-model / K=5: cross-support vs slm_mux	1319	-0.014	[-0.024, -0.004]	0.011

Appendix

Table 54: Question-level paired bootstrap confidence intervals and permutation tests.

Setting	Tie-break	Acc source	SLM	Cross	$\Delta$
GPQA / 6-model / K=5	model_order	ignore	0.343	0.354	0.010
GPQA / 6-model / K=5	validation_accuracy	auto	0.354	0.359	0.005
GPQA / qwen+phi+gemma / K=5	model_order	ignore	0.394	0.429	0.035
GPQA / qwen+phi+gemma / K=5	validation_accuracy	auto	0.434	0.439	0.005
MATH-500 / 6-model / K=5	model_order	ignore	0.720	0.708	-0.012
MATH-500 / 6-model / K=5	validation_accuracy	auto	0.722	0.710	-0.012

Appendix

Table 55: Tie-break robustness under model-order and validation-accuracy protocols.

Method	Held-out mean
Selected small-pool cross-support	0.415
Full-pool cross-support	0.364
Full-pool SLM	0.345
Full-pool Single-Best	0.388

Appendix

Table 56: Held-out GPQA pool selection over 20 random 50/50 splits.

Setting	SLM	Cross	$\Delta$	Wins / ties / losses
GPQA / 6-model / K=3	0.340 $\pm$ 0.006	0.360 $\pm$ 0.006	0.020 $\pm$ 0.008	19/0/1
GPQA / 6-model / K=5	0.343 $\pm$ 0.000	0.354 $\pm$ 0.000	0.010 $\pm$ 0.000	20/0/0
GPQA / qwen+phi+gemma / K=5	0.394 $\pm$ 0.000	0.429 $\pm$ 0.000	0.035 $\pm$ 0.000	20/0/0
MATH-500 / 6-model / K=5	0.720 $\pm$ 0.000	0.708 $\pm$ 0.000	-0.012 $\pm$ 0.000	0/0/20

Appendix

Table 57: Seed robustness over 20 offline subsampling seeds.

Configuration	Budget	SLM	Cross	Single-Best	Oracle
6 models $\times$ K=1 (budget=6)	6	0.354	0.364	0.353	0.766
4 models $\times$ K=2 (budget=8) qwen+phi+ministral+gemma	8	0.350	0.372	0.360	0.684
3 models $\times$ K=3 (budget=9) qwen+phi+gemma	9	0.376	0.400	0.371	0.641
2 models $\times$ K=5 (budget=10) qwen+gemma	10	0.409	0.414	0.379	0.566
6 models $\times$ K=5 (budget=30) full pool reference	30	0.343	0.354	0.379	0.758

Appendix

Table 58: Budget-fair comparison on GPQA. Curated pools with deeper sampling dominate the full 6-model reference.

Benchmark	Order SB	Val SB	SLM	Cross	$s$ -only	Composite
MATH-500	0.768	0.768	0.720	0.708	0.712	0.768
GSM8K	0.904	0.904	0.916	0.902	0.899	0.902
GPQA	0.379	0.399	0.343	0.354	0.379	0.354

Appendix

Table 59: With- vs without-validation comparison. Validation-selected single-best is the strongest deployable baseline.

Rule	Mean test cross-support	Std	Mean test $s$ -only	Most selected pool
max_calib_cross	0.417	0.022	0.392	qwen_phi_gemma
max_calib_s_only	0.408	0.028	0.381	qwen_gemma
min_pool_size	0.420	0.022	0.393	qwen_gemma
smallest_top_cross	0.414	0.023	0.394	qwen_gemma

Appendix

Table 60: Held-out GPQA pool selection across four selection rules.

Category	Rows	Fraction
Both wrong, oracle hit	710	61.2
Cross-support fixes SLM	110	9.5
Cross-support breaks SLM	90	7.8
Contamination	250	21.6

Appendix

Table 61: Case taxonomy for the GPQA qualitative analysis.

Benchmark / $K$	majority $\delta$	top-coal $\delta$	SLM-MUX ${}_{\text{ans}}$ $\delta$	cross slice $\delta$	single-best $\delta$
GPQA / $K{=}5$ / 6m	$+0.000$	–	$+0.010$	–	$+0.000$
MATH-500 / $K{=}5$ / 6m	$-0.004$	–	$-0.020$	–	$+0.000$
GSM8K / $K{=}5$ / 6m	$+0.000$	–	$-0.015$	–	$+0.000$

Appendix

Table 62: Implementation-level recovery of the canonical baselines as limits inside the generalized family. All limits are recovered exactly modulo the deterministic

(W,\text{pooled},\text{lex})

tie-break specified for the family; the small MATH-500 majority gap is from a handful of questions on which one model has fewer valid extracted answers than the others.

Setting	majority	top-coal	SLM-MUX ${}_{\text{ans}}$	cross slice	best Sum	best Max	best $(\alpha,\kappa,\text{Agg})$	best acc
	( $\alpha{=}0$ )	( $\alpha{=}1$ , Sum)	( $\alpha{=}1$ , Max)	( $\alpha{=}1{-}\tfrac{1}{N-1}$ )	( $\kappa{=}0$ )	( $\kappa{=}0$ )
GPQA / 6m / $K{=}3$	0.384	0.372	0.364	0.365	0.384	0.384	( $0.0,\ 0,\ \text{Sum}$ )	0.384
GPQA / 6m / $K{=}5$	0.374	0.369	0.354	0.369	0.379	0.374	( $1.0,\ 4,\ \text{Max}$ )	0.394
GPQA / qwen+phi+gemma / $K{=}3$	0.410	0.394	0.405	0.404	0.410	0.410	( $0.0,\ 0,\ \text{Sum}$ )	0.410
GPQA / qwen+phi+gemma / $K{=}5$	0.419	0.424	0.434	0.429	0.450	0.439	( $0.6,\ 0,\ \text{Sum}$ )	0.450
GPQA / qwen+gemma / $K{=}5$	0.429	0.419	0.419	0.429	0.429	0.424	( $0.0,\ 0,\ \text{Sum}$ )	0.429

Appendix

Table 63: Generalized two-axis family on the full 6-model and curated GPQA pools. All canonical points are evaluated on the same cached generations as the rest of the paper.

\alpha

controls majority-vs-top-answer aggregation;

\kappa

controls uniform-vs-expert weighting (calibration-half acc i , held-out evaluation for

\kappa{>}0

); Agg selects coalition (Sum) vs strongest-supporter (Max) aggregation. Best generalized point in bold per row.

Pool	$K$	majority	SLM-MUX ${}_{\text{ans}}$	cross slice	single-best (val)	best gen $(\alpha,\kappa,\text{Agg})$	best acc
full 6m	1	0.370	0.370	0.370	0.338	( $0.0,0,\text{Sum}$ )	0.370
full 6m	2	0.372	0.372	0.372	0.300	–	–
full 6m	3	0.384	0.364	0.365	0.342	( $0.0,0,\text{Sum}$ )	0.384
full 6m	4	0.365	0.349	0.358	0.369	–	–
full 6m	5	0.374	0.354	0.369	0.364	( $1.0,4,\text{Max}$ )	0.394
qwen+phi+gemma	1	0.375	0.375	0.375	0.334	( $0.0,0,\text{Sum}$ )	0.375

Appendix

Table 64: K interaction on GPQA for the generalized family. Best validation-free

\kappa{=}0

generalized point reported per

K

, alongside the canonical points. “–” marks aggregations not swept at

K{=}2

and

K{=}4

. The full pool peaks at

K{=}3

in the

\kappa{=}0

region; the curated pool is monotonic in

K

Setting	majority	SLM-MUX ${}_{\text{ans}}$	cross slice	single-best (val)	best Sum	best Max
GPQA / 6m / $K{=}3$	0.384	0.362	0.364	0.360	0.391 @( $0.0,1.0$ )	0.391@( $0.0,1.0$ )
GPQA / 6m / $K{=}5$	0.380	0.354	0.370	0.374	0.403 @( $0.5,4.0$ )	0.402@( $0.3,4.0$ )
GPQA / qwen+phi+gemma / $K{=}5$	0.422	0.441	0.434	0.368	0.455 @( $0.6,0.0$ )	0.446@( $0.7,0.5$ )
MATH-500 / 6m / $K{=}5$	0.699	0.691	0.695	0.761	0.765@( $0.8,4.0$ )	0.766@( $0.3,4.0$ )
GSM8K / 6m / $K{=}5$	0.897	0.898	0.899	0.903	0.902@( $0.0,8.0$ )	0.907 @( $1.0,0.5$ )

Appendix

Table 65: Validation-aware evaluation of the generalized family. Per setting we draw a fresh 50/50 random split over questions for each of 20 seeds, fit per-model calibration accuracies

\text{acc}_{i}

on one half, and evaluate every

(\alpha,\kappa,\text{Agg})

cell on the held-out half. All numbers are held-out accuracy averaged across the 20 seeds. “Single-best (val)” is the

\kappa{=}\infty

limit.

Setting	$N$	maj	top-coal	SLM-MUX ${}_{\text{ans}}$	cross slice	best Sum $\alpha^{\star}$	best Sum	best Max $\alpha^{\star}$	best Max	single-best	$s$ -only	rank-sum	top2_strong
GPQA / 6m / $K{=}3$	6	0.384	0.372	0.364	0.365	0.0	0.384	0.0	0.384	0.342	0.386	0.381	0.367
GPQA / 6m / $K{=}5$	6	0.374	0.369	0.354	0.369	0.7	0.379	0.0	0.374	0.364	0.384	0.364	0.379
GPQA / qwen+phi+gemma / $K{=}3$	3	0.410	0.394	0.405	0.404	0.0	0.410	0.0	0.410	0.337	0.391	0.403	0.392
GPQA / qwen+phi+gemma / $K{=}5$	3	0.419	0.424	0.434	0.429	0.6	0.450	0.6	0.439	0.364	0.394	0.439	0.404
MATH-500 / 6m / $K{=}5$	6	0.706	0.702	0.700	0.704	0.3	0.708	0.5	0.712	0.788	0.710	0.712	0.716
GSM8K / 6m / $K{=}5$	6	0.900	0.897	0.901	0.901	0.0	0.900	0.4	0.901	0.914	0.899	0.901	0.904

Appendix

Table 66: Region summary. For each (benchmark,

K

, pool) we report the best validation-free

\kappa{=}0

generalized point on each of the Sum and Max planes, alongside the canonical points and external references (

s

-only, rank-sum, trust-aware top2_strong, oracle). This compressed view summarizes the best-region map per regime.

Question	Answer from the generalized sweep
RQ1: majority vs top-answer ( $\alpha$ )	Balanced GPQA prefers low to moderate $\alpha$ (full-pool $K{=}3$ : $\alpha{=}0$ ; curated $K{=}5$ : $\alpha{=}0.6$ ). Dominated pools accept high $\alpha$ only when paired with high $\kappa$ , i.e. single-best in disguise.
RQ2: uniform vs expert ( $\kappa$ )	Under leakage-free held-out evaluation, $\kappa{>}0$ adds $+0.02$ to $+0.09$ on GPQA above every canonical point including $\kappa{=}\infty$ single-best. On MATH-500 and GSM8K the best $\kappa{>}0$ cell lies within $0.005$ of single-best – the family recovers single-best routing rather than a new mode.
RQ3: Sum vs Max	Sum wins on 5 of 6 GPQA settings (the curated $K{=}5$ peak is at Sum $=0.450$ vs Max $=0.439$ ). Max is preferred at $K{=}5$ on the full GPQA pool and on GSM8K, where one model has a consistently strong $c_{i}$ .
RQ4: regime	The best routing region shifts with pool curation: low- $\alpha$ , $\kappa$ near zero, Sum aggregation on balanced pools; high- $\alpha$ , $\kappa{\geq}4$ , Max aggregation near the single-best limit on dominated pools. $K{=}3$ remains the GPQA sweet spot when $\kappa$ is held to zero.

Appendix

Table 67: What the generalized family tells us about the four routing research questions on this benchmark set. “balanced” = curated GPQA pools (qwen+phi+gemma, qwen+gemma, phi+gemma); “dominated” = full 6-model pool on MATH-500 / GSM8K (Qwen

\geq 10

pts above the rest).

Setting	best $(\alpha,\kappa,\text{Agg})$	gen acc	$\Delta$ vs SLM-MUX ${}_{\text{ans}}$	$\Delta$ vs $s$ -only	$\Delta$ vs rank-sum
GPQA / 6m / $K{=}3$	( $0.0,0,\text{Sum}$ )	0.384	+0.053 [+0.006, +0.099] $p{=}0.024$	$-0.002\ p{=}0.73$	$+0.004\ p{=}0.51$
GPQA / 6m / $K{=}5$	( $0.7,0,\text{Sum}$ )	0.379	$+0.035$ [ $-0.030,+0.101$ ] $p{=}0.37$	$-0.005\ p{=}1.0$	$+0.015\ p{=}0.59$
GPQA / qwen+phi+gemma / $K{=}5$	( $0.6,0,\text{Sum}$ )	0.450	+0.056 [+0.015, +0.096] $p{=}0.018$	+0.056 [+0.020, +0.096] $p{=}0.007$	$+0.010\ p{=}0.69$
MATH-500 / 6m / $K{=}5$	( $0.5,0,\text{Max}$ )	0.712	$-0.008\ p{=}0.57$	$+0.002\ p{=}1.0$	$\phantom{-}0.000\ p{=}1.0$
GSM8K / 6m / $K{=}5$	( $0.4,0,\text{Max}$ )	0.901	$\mathbf{-0.014\ [-0.024,-0.005]\ p{=}0.007}$	$+0.002\ p{=}0.45$	$+0.001\ p{=}1.0$

Appendix

Table 68: Question-level paired tests for the strongest validation-free generalized point (

\kappa{=}0

) against each of the SLM-MUX

{}_{\text{ans}}

s

-only, and rank-sum baselines.

\Delta

is the per-question mean correctness difference (positive = generalized wins);

p

is the two-sided paired-permutation

p

-value with

B{=}5000

replicates; 95% CIs from paired bootstrap. Statistically significant gains in bold (

p<0.05

Configuration	budget	majority	SLM-MUX ${}_{\text{ans}}$	cross slice	best $(\alpha,\text{Agg})$	best gen
6 models $\times$ $K{=}1$	6	0.355	0.355	0.355	( $0,\text{Sum}$ )	0.355
4 models $\times$ $K{=}2$ (qwen+phi+ministral+gemma)	8	0.386	0.378	0.378	( $0,\text{Sum}$ )	0.386
3 models $\times$ $K{=}3$ (qwen+phi+gemma)	9	0.410	0.405	0.404	( $0,\text{Sum}$ )	0.410
2 models $\times$ $K{=}5$ (qwen+gemma)	10	0.429	0.419	0.429	( $0,\text{Sum}$ )	0.429
6 models $\times$ $K{=}5$ (full pool reference)	30	0.369	0.348	0.364	( $0.7,\text{Sum}$ )	0.379

Appendix

Table 69: Budget-fair generalized comparison on GPQA. Each row is a fixed model

\times K

budget. “best gen” is the strongest

\kappa{=}0

generalized point at that budget. Curated pools with deeper sampling cleanly dominate the full

6{\times}5

reference at

3\times

less compute.

LLM Agent Orchestration

Large language model agent orchestration coordinates models, tools, and workflows to complete complex tasks reliably and efficiently.

See all 454 papers

Jun 2, 2025

Small Language Models are the Future of Agentic AI

Large language models (LLMs) are often praised for exhibiting near-human performance on a wide range of tasks and valued for their ability to hold a general conversation. The rise of agentic AI systems is, however, ushering in a mass of applications in which language models perform a small number of specialized tasks repetitively and with little variation. Here we lay out the position that small language models (SLMs) are sufficiently powerful, inherently more suitable, and necessarily more economical for many invocations in agentic systems, and are therefore the future of agentic AI. Our argumentation is grounded in the current level of capabilities exhibited by SLMs, the common architectures of agentic systems, and the economy of LM deployment. We further argue that in situations where general-purpose conversational abilities are essential, heterogeneous agentic systems (i.e., agents invoking multiple different models) are the natural choice. We discuss the potential barriers for the adoption of SLMs in agentic systems and outline a general LLM-to-SLM agent conversion algorithm. Our position, formulated as a value statement, highlights the significance of the operational and economic impact even a partial shift from LLMs to SLMs is to have on the AI agent industry. We aim to stimulate the discussion on the effective use of AI resources and hope to advance the efforts to lower the costs of AI of the present day. Calling for both contributions to and critique of our position, we commit to publishing all such correspondence at https://research.nvidia.com/labs/lpr/slm-agents.

Peter Belcak, Greg Heinrich, Shizhe Diao +5

NVIDIA Research · Georgia Institute of Technology

Small Language Models LLM Agents

Ask PDF

Feb 18, 2026

AdaptOrch: Task-Adaptive Multi-Agent Orchestration in the Era of LLM Performance Convergence

As large language models from diverse providers converge toward comparable benchmark performance, the traditional paradigm of selecting a single best model per task yields diminishing returns. We argue that orchestration topology -- the structural composition of how multiple agents are coordinated, parallelized, and synthesized -- now dominates system-level performance over individual model capability. We present AdaptOrch, a formal framework for task-adaptive multi-agent orchestration that dynamically selects among four canonical topologies (parallel, sequential, hierarchical, and hybrid) based on task dependency graphs and empirically derived domain characteristics. Our framework introduces three key contributions: (1) a Performance Convergence Scaling Law, formalizing conditions under which orchestration selection outweighs model selection; (2) a Topology Routing Algorithm that maps task decomposition DAGs to optimal orchestration patterns in O(|V| + |E|) time; and (3) an Adaptive Synthesis Protocol with provable termination guarantees and heuristic consistency scoring for parallel agent outputs. We validate AdaptOrch across coding (SWE-bench), reasoning (GPQA), and retrieval-augmented generation tasks, demonstrating that topology-aware orchestration achieves 12-23% improvement over static single-topology baselines, even when using identical underlying models. Our results establish orchestration design as a first-class optimization target independent of model scaling.

Geunbin Yu

Department of Artificial Intelligence, Korea National Open University

Multi-Agent Orchestration LLM Agent Orchestration Multi-Agent System Optimization Multi-Agent Systems

Ask PDF

Oct 9, 2026

Proactive Agentic Orchestration

Agentic orchestration delegates tasks to sub-agents with independent contexts, providing a key path to extending the effective execution horizon of large language model agents. However, existing orchestration methods face two limitations: in execution, current methods fix sub-agents' goals and harnesses at creation time, limiting their adaptability to dynamic real-world environments; in optimization, sparse and delayed feedback constrains effective learning of the orchestration policy. To address these limitations, we introduce POrchestra, an on-the-fly proactive agentic orchestration framework. It allows sub-agents to proactively communicate with the orchestrator at any logical point to request harness adjustments while maintaining continuity of local execution. This mechanism not only enables sub-agents to adapt seamlessly to dynamic environments but also naturally inspires On-Policy Orchestration Distillation (OPOD) based on hindsight feedback. OPOD attributes sub-agents' proactive revision requests back to preceding orchestration decisions to derive targeted improvement signals, and leverages these signals as process supervision alongside the final reward to optimize the orchestration policy. We evaluate POrchestra on three long-horizon agent benchmarks spanning deep research, dynamic digital assistance, and software engineering. POrchestra improves task success by an average of 9.1% relative to the strongest baseline; combined with OPOD, it improves small-orchestrator success by an average of 10.0% relative to existing training baselines. POrchestra also achieves a more favorable cost-performance trade-off. Our code is publicly available at https://github.com/XueyangFeng/POrchestra.

Xueyang Feng, Chaozhuo Li, Xiaohe Bo +4

Gaoling School of Artificial Intelligence, Renmin University of China · Beijing Academy of Artificial Intelligence · The Chinese University of Hong Kong Shenzhen

LLM Agent Orchestration LLM Agent Workflow Optimization Long-Horizon Agent Tasks Agentic RL Process Supervision

Ask PDF

Abstract

LLM Agent Orchestration

Explore similar work

Small Language Models are the Future of Agentic AI

AdaptOrch: Task-Adaptive Multi-Agent Orchestration in the Era of LLM Performance Convergence

Proactive Agentic Orchestration