ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inference

Siyuan Ma; Bo Gao; Xiaojun Jia; Simeng Qin; Tianlin Li; Ke Ma; Xiaoshuang Jia; Wenqi Ren

cs.AIFeb 27, 2026

ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inference

Authors: Siyuan Ma, Bo Gao, Xiaojun Jia, Simeng Qin, Tianlin Li, Ke Ma, Xiaoshuang Jia, Wenqi Ren, +1 more

Organizations: Nanyang Technological University · Carnegie Mellon University · Northeast University (Qinhuangdao Campus) · Beihang University · University of the Chinese Academy of Sciences · Renmin University of China · Sun Yat-sen University

Abstract

The paradigm of large language model (LLM) reasoning is shifting from parameter scaling to test-time compute scaling, yet many existing approaches still rely on uniform brute-force sampling (for example, fixed best-of-N or self-consistency) that is costly, hard to attribute, and can trigger overthinking with diminishing returns. We propose ODAR-Expert, an adaptive routing framework that optimizes the accuracy-efficiency trade-off via principled resource allocation. ODAR uses a difficulty estimator grounded in amortized active inference to dynamically route queries between a heuristic Fast Agent and a deliberative Slow Agent. We further introduce a free-energy-principled, risk-sensitive fusion mechanism that selects answers by minimizing a variational free energy objective, balancing log-likelihood with epistemic uncertainty (varentropy) as a principled alternative to ad hoc voting over heterogeneous candidates. Extensive evaluation across 23 benchmarks shows strong and consistent gains, including 98.2% accuracy on MATH and 54.8% on Humanity's Last Exam (HLE), while improving the compute-accuracy frontier under compute-matched settings. We also validate reproducibility on a fully open-source stack (Llama 4 + DeepSeek), where ODAR surpasses homogeneous sampling strategies while reducing computational costs by 82%. Overall, our results suggest that thinking-optimal scaling requires adaptive resource allocation with free-energy-based decision-making rather than simply increasing test-time compute.

Figures & tables

Figure 1: Multi-dimensional performance comparison across 23 benchmarks. The radar chart visualizes ODAR’s average accuracy across 8 task categories relative to frontier models (GPT-5.1, Claude-4.5) and the strongest inference-time baseline (Self-Consistency). ODAR significantly expands the performance envelope, particularly in Mathematics (+20.2% on IMO 2025) and Advanced Cognition (+12.0% on HLE), while maintaining superior cost-efficiency.

Figure 2: ODAR System Architecture (Overview). Given input $x$ , a rule-based dispatch layer (ER/MR/SS; fixed heuristics and priority rules) extracts coarse task features and selects a base model identifier for dispatch. In parallel, the Difficulty Estimator predicts $d\in[0,1]$ and the Strategy Selector routes the query using fixed thresholds $\tau_{1}{=}0.3$ and $\tau_{2}{=}0.7$ : Simple ( $d{<}\tau_{1}$ ): Fast-only ( $c{=}1$ ); Medium ( $\tau_{1}{\leq}d{<}\tau_{2}$ ): Fast + Slow verification ( $c{=}2$ ); Hard ( $d{\geq}\tau_{2}$ ): Best-of- $N$ with $N{=}5$ Slow candidates plus FEP fusion ( $c{=}6$ ). Our evaluation emphasizes difficulty-based routing and FEP fusion; ER/MR provide system-level dispatch/model selection.

Property	Fast Agent ( $A_{\theta}$ )	Slow Agent ( $A_{\gamma}$ )
Base Model	GPT-5.1	Claude-4.5 Sonnet
Temperature	0.2 (Deterministic)	0.3 (Exploratory)
Token Limit	512	1024
Avg. Latency	1.8s	6.4s / 32.0s (Seq.)
Primary Role	Hypothesis generation	Verification / Exploration

Table 1: Fast vs. Slow Agent Configuration

Category	Dataset	GPT-5.1	Claude-4.5	Len-Adapt.	Self-Consist.	ODAR (Ours)	Gain
Mathematics	MATH Hendrycks et al. (2021)	91.8	89.2	94.0	94.5	98.2	+6.4
	GSM8K Cobbe et al. (2021)	95.9	94.5	97.0	97.2	99.1	+3.2
	IMO 2025 Mathematical Association of America (2025)	48.5	45.2	55.6	55.4	68.7	+20.2
	MathVista Lu et al. (2023)	89.1	87.4	91.7	92.3	96.5	+7.4
Commonsense	ARC-Challenge Clark et al. (2018)	92.5	90.8	94.6	94.2	98.5	+6.0
	OpenBookQA Mihaylov et al. (2018)	91.2	89.5	93.5	93.8	97.8	+6.6

Table 2: Main Results: Performance Across 23 Diverse Benchmarks. Evaluation conducted under a unified protocol with compute-matched budgets. ODAR sets the new performance ceiling for inference-time compute scaling across 22 out of 23 benchmarks.

Category	Dataset	(A) Llama 4	(B) DeepSeek	(C) DeepSeek	(D) Open-ODAR	Gain
		(Fast Agent)	(CoT)	(SC $k=5$ )	(Llama+DeepSeek)	(vs. SC)
Math	MATH Hendrycks et al. (2021)	76.5	87.8	90.2	92.4	+2.2%
Math	GSM8K Cobbe et al. (2021)	88.4	96.3	97.1	97.8	+0.7%
Coding	LiveBench White et al. (2024)	58.2	71.4	73.5	76.1	+2.6%
Coding	SWE-bench Jimenez et al. (2023)	45.5	62.5	65.8	68.2	+2.4%
Reasoning	GPQA Rein et al. (2024)	35.8	68.4	70.5	72.8	+2.3%

Table 3: Open-ODAR Performance (Fully Reproducible Stack). Comparison between single-agent baselines and Open-ODAR using strictly open-source models (Llama 4 + DeepSeek V3.2). ODAR achieves superior accuracy to Self-Consistency (SC) at a fraction of the compute cost (

\sim 4.5\times

25.0\times

Single-Agent Baselines
Method	MATH	BBH	MMLU-Pro	Cost	Eff. †
GPT-5.1 Base	91.8	88.0	86.2	1.0 $\times$	0.0
Claude-4.5 Base	89.2	86.5	85.1	3.0 $\times$	-
Multi-Candidate Strategies
Self-Consistency ( Wang et al., 2022 )	94.5	91.2	89.5	5.0 $\times$	0.6
Best-of-N ( $N$ =5) ( Aminian et al., 2025 )	95.0	92.0	90.0	5.0 $\times$	0.7

Table 4: Comparison with SOTA Methods. ODAR achieves highest accuracy with best efficiency score.

Figure 3: Multi-dimensional Ablation Matrix. Columns denote accuracy drop (

\Delta

Acc %) relative to Full ODAR across 16 representative benchmarks. The red line indicates normalized inference cost. Results confirm that the DE prevents cost explosions while the Slow Agent ensures reasoning depth.

Appendix figures & tables40 assets

Supplementary material from the paper’s appendix.

Appendix

Variables & Distributions
Symbol	Description
$x$	Input query (text or multimodal)
$y,\hat{y}$	Candidate answer sequence / Final selected answer
$s$	Latent cognitive state (reasoning path)
$q(s\|y)$	Variational belief distribution over hidden states
$\mathcal{F}(y\|x)$	Variational Free Energy (Accuracy + Complexity)

Appendix

Table 5: Nomenclature and key variables used throughout the ODAR framework.

Biological Component	ODAR Abstraction	Computational Role
Theta Rhythm (4-8Hz)	Difficulty Estimator	Gating Signal: Assesses demand/load
Gamma Bursts (30-100Hz)	Slow Agent Execution	Intensive Compute: Expensive processing
Phase-Amplitude Coupling	Dynamic Routing ( $\tau$ )	Conditional Logic: If High Load $\to$ Activate
Metabolic Constraints	Token Cost Budget	Constraint: Maximize accuracy per unit cost

Appendix

Table 6: Functional Correspondence between TG-PAC and ODAR.

Framework	Core Mechanism	Router Training	Inference Overhead	Primary Goal	Reasoning Focus	Bio-Inspired
HierRouter Gupta et al. [2025]	Reinforcement Learning (PPO)	High (Policy Training)	Medium	General Efficiency	Partial	No
RTR Pan et al. [2025]	Supervised Regression	Medium (Scalar Predictor)	Low	Strategy Selection	Yes	No
MoMA Guo et al. [2025]	Intent Classification	Medium (Capability Profiling)	Low	General Orchestration	No	No
CP-Router Su et al. [2025]	Conformal Prediction	Low (Calibration Only)	Medium (Sampling)	Uncertainty/Cost	Yes	No
InferenceDynamics Shi et al. [2025]	Capability Knowledge Graph	Medium (Profiling)	Low	Knowledge Routing	No	No
ODAR (Ours)	Active Inference & FEP	Low (Lightweight DE)	Low	Test-Time Scaling	Strong (System 2)	Yes (TG-PAC)

Appendix

Table 7: Qualitative Comparison of Adaptive Routing Frameworks. ODAR distinguishes itself by integrating principled difficulty estimation (Active Inference) with a training-free fusion mechanism, avoiding the high training overhead of RL-based methods while targeting reasoning depth specifically.

Figure 4: Mapping from theoretical principles to computational implementation. Each theoretical framework contributes a distinct component: FEP enables principled fusion, Active Inference guides adaptive routing, and theta-gamma coupling motivates the dual-agent architecture.

Figure 5: Theoretical framework overview. Left panel: neuroscientific principles including gamma oscillations, theta rhythms, and the Free Energy Principle. Middle panel: corresponding computational principles of fast pattern matching, slow verification, and uncertainty-driven resource allocation. Right panel: ODAR implementation comprising the Fast Agent, Slow Agent, and Difficulty Estimator.

ID	Feature	Definition	Imp.
Lexical Features ( $k_{1}=8$ )
$f_{1}$	Character length	Total number of characters in query	0.012
$f_{2}$	Word count	Number of whitespace-separated tokens	0.018
$f_{3}$	Token count	BPE tokens via tiktoken (cl100k_base)	0.024
$f_{4}$	Avg. word length	Mean character count per word	0.008
$f_{5}$	Max word length	Maximum character count across words	0.006

Appendix

Table 8: Complete feature specification for the Difficulty Estimator. Importance scores represent mean decrease in Pearson

r

when feature is permuted, averaged over 10 runs.

Domain	Representative Vocabulary ( $\|V_{d}\|$ )
Mathematics (87)	theorem, derivative, integral, matrix, eigenvalue, convergence, polynomial
Coding (92)	algorithm, function, complexity, implement, recursive, runtime, optimize
Logic (73)	valid, satisfiable, contradiction, premise, inference, deduction, entailment

Appendix

Table 9: Representative vocabulary by domain used in lexical feature extraction.

Rank	Feature	Category	Importance
1	Domain: math ( $f_{22}$ )	Semantic	0.056
2	Difficulty density ( $f_{18}$ )	Semantic	0.052
3	TF-IDF math ( $f_{15}$ )	Semantic	0.048
4	Domain: code ( $f_{23}$ )	Semantic	0.047
5	Type: proof ( $f_{21}$ )	Semantic	0.044
6	Math symbol ( $f_{8}$ )	Lexical	0.042

Appendix

Table 10: Top-10 features by permutation importance. Semantic features contribute 56% of total importance.

Configuration	Pearson $r$	Routing Acc.	Task Acc.
Full model (24 features)	0.79	87.3%	83.5%
Without lexical ( $k_{1}=8$ )	0.76	85.1%	82.8%
Without structural ( $k_{2}=6$ )	0.74	83.8%	82.1%
Without semantic ( $k_{3}=10$ )	0.61	74.2%	79.3%
Random baseline	0.00	33.3%	75.4%

Appendix

Table 11: Feature category ablation. Removing semantic features causes the largest performance degradation.

Configuration	Macro Avg.	MMLU-Pro	MATH	HLE	Subsample CI
Base (Single Model)	77.6%	89.5%	91.8%	42.8%	$\pm$ 0.32%
ODAR (Rules Only, No DE Gating)	79.2%	90.1%	93.4%	44.1%	$\pm$ 0.41%
ODAR (ER/MR Off, DE Active)	88.4%	93.1%	96.9%	53.2%	$\pm$ 0.55%
Full ODAR (Final)	89.6%	94.2%	98.2%	54.8%	$\pm$ 0.45%
Randomized (Matched Call-Budget)	81.3%	91.2%	94.1%	46.5%	$\pm$ 0.85%

Appendix

Table 12: Step-wise Ablation and De-confounding Analysis. All results are averaged over 3 seeds; subsample uncertainty is reported as t-based 95% CI over

K=5

disjoint folds. The "Rules Only" baseline uses fixed routing (all queries to Medium path via overriding

d\equiv 0.5

) with ER/MR active, thus bypassing Hard-path Best-of-N expansion.

Error Type	Detailed Description	Count	%
System & Routing Failures (34%)
E1: Difficulty Est.	Mis-classification of hard queries as simple, routing to insufficient compute.	8	8%
E2: Expert Routing	Dispatch assigns an inappropriate expert/domain tag (e.g., Math vs. Code), leading to suboptimal model/prompt/strategy.	5	5%
E3: Verification Fail.	False Acceptance : Slow Agent incorrectly verifies a wrong Fast answer.	12	12%
E4: FEP Selection	Missed Oracle : a correct candidate exists in $\mathcal{C}$ , but FEP ranks an incorrect candidate as best.	9	9%
Model Capability Limitations (66%)

Appendix

Table 13: Error Taxonomy and Distribution (

N=100

). Errors are categorized to distinguish between system-level routing failures (E1–E4) and underlying model capability limitations (E5–E6).

ID	Source	Reasoning Quality	Answer	Energy ( $\ell_{i}$ )	Risk ( $\sigma_{i}^{2}$ )	$\mathcal{F}_{\text{raw}}$
$y_{\theta}$	Fast	Intuitive Guess	$x=2$	$0.18$	$0.025$	$0.1825$
$y_{\gamma}^{(1)}$	Slow	Standard Derivation	$x=2$	$0.09$	$0.012$	$0.0912$
$y_{\gamma}^{(2)}$	Slow	Rigorous Calculus	$\mathbf{x=2}$	$\mathbf{0.06}$	$\mathbf{0.008}$	0.0608 ∗
$y_{\gamma}^{(3)}$	Slow	Sign Error (Logic)	$x=1^{{\color[rgb]{1,0,0}\times}}$	$0.45$	$0.120$	$0.4620$
$y_{\gamma}^{(4)}$	Slow	Vertex Formula	$x=2$	$0.08$	$0.015$	$0.0815$
$y_{\gamma}^{(5)}$	Slow	Circular Reasoning	$x=2$	$0.21$	$0.032$	$0.2132$

Appendix

Table 14: FEP Fusion Case Study. ODAR generates multiple candidate paths. While a hallucinated path (

y_{\gamma}^{(3)}

) produces an incorrect answer, FEP Fusion correctly filters it out. Note: Values represent Character-Normalized Free Energy (

\mathcal{F}_{\text{raw}}

) as defined in Eq. 10.

Fusion Method	MATH	BBH	TruthfulQA	Avg.
Max Confidence	94.5	89.2	79.1	87.6
Majority Voting (SC)	95.1	90.4	80.3	88.6
Average Log-Prob	95.8	90.7	81.2	89.2
FEP Fusion (Ours)	96.7	91.7	82.7	90.4
Gain vs. Best Baseline	+0.9	+1.0	+1.5	+1.2

Appendix

Table 15: Fusion Method Ablation (Accuracy %)

Component	Parameters	Values
Difficulty Estimator	Feature Dimension ( $k$ )	24
Difficulty Estimator	$L_{2}$ Regularization ( $\lambda_{\text{reg}}$ )	0.01
	Training Epochs	50
	Learning Rate	$10^{-3}$
	Batch Size	32
	Optimizer	Adam

Appendix

Table 16: ODAR Hyperparameters and Configuration

Path	Difficulty Range	Distribution	Cost
Simple ( $\pi_{\text{S}}$ )	$d<0.3$	41%	$1.0\times$
Medium ( $\pi_{\text{M}}$ )	$0.3\leq d<0.7$	35%	$4.2\times$
Hard ( $\pi_{\text{H}}$ )	$d\geq 0.7$	24%	$16.0\times$
Weighted Average		100%	$2.55\times$

Appendix

Table 17: Path Distribution Across Benchmarks

[SYSTEM]
You are a rapid reasoning assistant optimized for speed and pattern recognition.
Behavioral Constraints:
1. Limit reasoning to $\leq$ 3 intermediate steps
2. Prioritize retrieval and pattern matching over deliberation
3. Avoid excessive justification or self-correction
Output Format: Answer: <response>

Appendix

Table 18: Fast Agent System Prompt

System Instruction
You are a careful verification assistant. Your task is to rigorously validate a proposed solution.
Verification Protocol:
1. Restate : Summarize the proposed answer and key reasoning steps
2. Validate : Check each logical step for correctness
3. Judge : Output CORRECT or INCORRECT
4. Correct : If incorrect, provide the corrected solution with justification

Appendix

Table 19: Slow Agent Verification Prompt (Medium Path)

[SYSTEM]
You are a thorough reasoning assistant. Solve the problem using rigorous, step-by-step analysis.
Solution Protocol:
1. Analyze : Identify key concepts, constraints, and edge cases
2. Strategize : Select an appropriate solution approach
3. Execute : Implement the solution with explicit intermediate steps
4. Verify : Perform self-check before finalizing

Appendix

Table 20: Slow Agent Independent Reasoning Prompt (Hard Path)

Configuration	MATH	BBH
Full Prompt (Ours)	98.2	95.4
Fast Agent Ablations
w/o step limit ( $\leq$ 3)	97.4 (-0.8)	94.9 (-0.5)
w/o “avoid self-correction”	97.6 (-0.6)	94.7 (-0.7)
Slow Agent Ablations
w/o verification protocol	96.9 (-1.3)	94.2 (-1.2)

Appendix

Table 21: Prompt Design Ablation. Each row removes one design element from the full prompt.

Path	Distribution	Calls per Query	Contribution
Simple ( $d<0.3$ )	41%	1 (Fast)	0.41
Medium ( $0.3\leq d<0.7$ )	35%	2 (Fast + Slow)	0.70
Hard ( $d\geq 0.7$ )	24%	6 (Fast + 5 $\times$ Slow)	1.44
Weighted Average	100%	2.55	2.55

Appendix

Table 22: Breakdown of inference calls by routing path. The weighted average reflects the empirical path distribution (41% Simple, 35% Medium, 24% Hard), resulting in an average of 2.55 calls per query.

Path	P50 Latency	P95 Latency	Parallelizable
Simple	1.8s	3.2s	N/A
Medium	8.2s	14.5s	No
Hard	12.4s	28.7s	Yes (5 Slow calls)
Overall	6.1s	22.3s	Partial

Appendix

Table 23: Latency breakdown by routing path. P50 and P95 denote median and 95th percentile latencies.

Parameter	Fast Agent ( $A_{\theta}$ )	Slow Agent ( $A_{\gamma}$ )	Open-ODAR (Ref.)
Model ID	gpt-5.1	claude-4.5-sonnet	Llama-4-Scout / DeepSeek-V3.2
Deployment	OpenAI API	Anthropic API	vLLM (Local, FP8 Quant)
System Prompt	MD5: 7a9c… (See prompts/fast.json)	MD5: b4e1… (See prompts/slow.json)	MD5: … (Identical to API)
Temperature	0.2 (Fixed)	0.3 (Fixed)	0.6 (Calibrated for Open Weights)
Top-p (Nucleus)	0.9	0.95	0.95
Stop Tokens	["\n\n", "Q:"]	["Wait", "User:"]	$<\|$ eot_id $\|>$ only

Appendix

Table 24: Exact Inference Snapshot. System prompts are version-controlled via MD5 hashes. Parameters align with Table 16 to ensure consistency.

Category	Dataset	Test Size	Metric	Reference
Mathematics	MATH	5,000	Accuracy	Hendrycks et al. [2021]
	GSM8K	1,319	Accuracy	Cobbe et al. [2021]
	IMO 2025	100 (Shortlist)	Proof Score	IMO Comp. (2025)
	MathVista	1,000	Accuracy	Lu et al. [2023]
Commonsense	ARC-Challenge	1,172	Accuracy	Clark et al. [2018]
	OpenBookQA	500	Accuracy	Mihaylov et al. [2018]

Appendix

Table 25: Dataset details and statistics for all 23 evaluation benchmarks. Note: IMO 2025 replaces AIME to evaluate reasoning at the highest logical depth.

Figure 6: Performance breakdown by task category. Left: absolute accuracy by category. Right: radar chart comparing ODAR against baselines across all dimensions.

Category	Acc.	Simple	Medium	Hard	Cost
Commonsense	93.7%	68%	24%	8%	1.3 $\times$
Instruction	97.4%	82%	14%	4%	1.1 $\times$
Knowledge QA	83.7%	35%	41%	24%	2.6 $\times$
Multimodal	91.4%	39%	38%	23%	2.5 $\times$
Multi-Hop	70.9%	22%	43%	35%	3.4 $\times$
Mathematics	90.1%	31%	22%	47%	4.1 $\times$

Appendix

Table 26: Path distribution by task category. Hard Path allocation correlates strongly with category difficulty (

r=0.84

Difficulty	# Datasets	Baseline	ODAR	Gain
Simple ( $\geq 90\%$ )	6	92.6%	98.2%	+5.6%
Medium (70–90%)	9	84.0%	92.5%	+8.5%
Hard ( $<70\%$ )	8	51.5%	63.8%	+12.3%
Overall	23	79.8%	89.6%	+9.8%

Appendix

Table 27: Performance stratified by task difficulty. ODAR provides the largest gains on hard tasks (+12.3%). Note: Overall accuracy (89.6%) aligns with the main results in Table 2.

Figure 7: Pareto frontier analysis. The ODAR curve (red) strictly dominates fixed-budget strategies. ODAR surpasses o3-mini’s accuracy ( 98.2% vs 97.8%) while reducing cost by approximately 65% .

Model	Acc.	Avg. Cost	Max Cost	Risk Profile
DeepSeek-R1	92.4%	1,884	$\sim$ 3,500	Underthinking
OpenAI o3-mini	97.8%	7,150	$>$ 8,192	Unbounded
ODAR (Ours)	98.2%	2,550	6,900	Deterministic

Appendix

Table 28: Cost stability analysis (

N=500

). ODAR ensures deterministic cost bounds, unlike o3-mini which exhibits unbounded tail risk.

Performance (Accuracy)
Metric	MATH	HLE	BoolQ	Avg.
Oracle (Upper Bound)	99.1%	62.5%	99.8%	95.2%
ODAR (Ours)	98.2%	54.8%	98.1%	89.6%
Optimality Ratio	99.1%	87.7%	98.3%	94.1%
Efficiency (Inference Calls)
Oracle (Minimum)	2.10 $\times$	12.5 $\times$	1.00 $\times$	1.95 $\times$

Appendix

Table 29: Oracle analysis. ODAR achieves 94.1% average optimality ratio. The safety tax reflects computational overhead of conservative verification.

Figure 8: Latency distribution analysis. While mean latency is 26.6s due to Hard Path demands, median latency is 9.9s, and 41% of queries receive instant response (

<

3s).

Method	Model	Pearson $r$	Latency	Throughput
LLM-Judge	GPT-4o-mini	0.78	$\sim$ 450ms	$\sim$ 2 req/s
Deep Learning	DeBERTa-v3	0.81	15ms (GPU)	$\sim$ 60 req/s
ODAR DE	Linear	0.79	$<$ 1ms (CPU)	$>$ 10k req/s

Appendix

Table 30: Difficulty Estimator design comparison. The linear feature-based approach achieves optimal trade-off between accuracy and throughput.

Figure 9: Best-of-N sample size analysis. Accuracy improves sharply from $n=1$ to $n=5$ ( +6.4% on MATH), with diminishing returns beyond $n=5$ .

$n$	MATH	BBH	HLE	Relative Cost
1	91.8	88.0	42.8	1.0 $\times$
3	96.5	93.5	50.5	3.0 $\times$
5	98.2	95.4	54.8	5.0 $\times$
7	98.4	95.7	55.2	7.0 $\times$
9	98.5	95.9	55.4	9.0 $\times$

Appendix

Table 31: Best-of-N detailed results. The setting

n=5

achieves optimal efficiency-accuracy trade-off.

Query Type	Query Content	Intrinsic Diff.	ODAR DE (Ours)	DeBERTa-v3
Fact Retrieval	“Who is considered the father of Quantum Mechanics ?”	Simple	Simple (✓)	Hard (Overfit)
Simple Code	“Print ’Hello World’ in C++ .”	Simple	Simple (✓)	Medium (Overfit)
Derivation	“Derive the time-dependent Schrödinger equation.”	Hard	Hard (✓)	Hard (✓)

Appendix

Table 32: The “Keyword Trap” Analysis. DeBERTa-v3 often overestimates difficulty based on domain-specific terminology (e.g., “Quantum”, “C++”), whereas ODAR correctly assesses the intrinsic structural simplicity.

Method	Training	GPU-h	MATH	HLE
Self-Consistency	None	0	86.5%	39.5%
Supervised PRM	800k samples	$\sim$ 500	97.1%	45.8%
ODAR FEP	None	0	96.7%	47.3%

Appendix

Table 33: Comparison with supervised PRM. ODAR matches PRM on in-distribution tasks while outperforming on out-of-distribution benchmarks, without requiring any training.

Simple Queries (GSM8K)
Scenario	Orig.	Adv.	Cost $\Delta$	Behavior
Standard	95.5%	—	1.10 $\times$	Correct
Complexity Attack	—	95.3%	+154%	Overthinking
Hard Queries (MATH)
Standard	52.4%	—	4.50 $\times$	Correct
Shortcut Attack	—	46.0%	$-$ 38%	Underestimation

Appendix

Table 34: Adversarial robustness analysis. ODAR exhibits fail-safe behavior under complexity attacks and resilience under shortcut attacks.

Figure 10: Simple Path execution for a factual query. The Difficulty Estimator ( $d=0.12$ ) correctly routes to the Fast Agent only.

Figure 11: Medium Path workflow for an algebraic equation. FEP Fusion selects the Slow Agent’s rigorous solution ( $\mathcal{F}=2.08$ ) over the Fast Agent’s answer ( $\mathcal{F}=2.67$ ).

Figure 12: Hard Path execution for IMO-level combinatorics. FEP Fusion identifies the unique complete proof ( $y_{\gamma}^{(1)}$ , $\mathcal{F}=1.85$ ) among six candidates.

Dataset	Mean Acc. ( $\mu$ )	Subsample CI	Within MoE	Gain $\Delta$	95% CI of $\Delta$
MATH	98.2%	$\pm$ 0.38%	$\pm$ 0.82%	+6.4%	[5.8%, 7.0%]
GSM8K	99.1%	$\pm$ 0.21%	$\pm$ 0.58%	+3.2%	[2.9%, 3.5%]
ARC-Challenge	98.5%	$\pm$ 0.41%	$\pm$ 0.85%	+6.0%	[5.2%, 6.8%]
MMLU-Pro	94.2%	$\pm$ 0.62%	$\pm$ 1.45%	+4.7%	[3.8%, 5.6%]
BBH	95.4%	$\pm$ 0.55%	$\pm$ 1.31%	+4.2%	[3.5%, 4.9%]
HLE	54.8%	$\pm$ 0.95%	–	+12.0%	[10.2%, 13.8%]

Appendix

Table 35: Rigorous Statistical Analysis: Mean accuracy and between-subsample CI (

K=5

disjoint folds,

t

-based).

\Delta

denotes the paired improvement (ODAR vs. Baseline) with its 95% bootstrap CI. Within MoE uses a binomial SRS approximation (

N=1,000

) as a reference scale. Global Avg. is the macro-average across 23 benchmarks; uncertainty is estimated via bootstrap over datasets.

Explore similar work

Aug 4, 2026

Interpretable Adaptive Sampling for LLM Test-Time Scaling

Test-time scaling improves LLM reasoning by generating and aggregating multiple candidate answers, yet many pipelines use fixed per-query budgets that spend the same compute on easy and difficult prompts. These fixed budgets are also difficult to inspect because they do not explain why a given prompt receives a particular number of samples. We propose adaptive} test-time scaling with a lightweight fuzzy controller that maps interpretable signals, including estimated prompt complexity and model confidence, to a per-query sampling budget. The controller assigns fewer samples to easier or more confident prompts and more samples to harder or less certain prompts, making inference-time compute inspectable rather than fixed or opaque. We evaluate under a fair-alignment protocol with matched decoding settings and controlled answer selection, and compare against best-of-

N

, compute-aware scaling, and self-certainty-based baselines on question-answering and mathematical reasoning tasks. Across models and datasets, adaptive fuzzy control improves over several standard baselines and remains close to a selector-matched full-budget control while reducing the average number of samples. These findings suggest that interpretable adaptive sampling is a practical direction for more efficient test-time reasoning in large language models.

Mobina Kashaniyan, Ali Jannesari

Iowa State University Ames, IA, USA

LLM Inference Efficiency Test-Time Scaling Efficient Language Model Reasoning Adaptive Sampling

Ask PDF

May 11, 2026

Reasoning Is Not Free: Robust Adaptive Cost-Efficient Routing for LLM-as-a-Judge

Reasoning-capable large language models (LLMs) have recently been adopted as automated judges, but their benefits and costs in LLM-as-a-Judge settings remain unclear. Through controlled comparisons between reasoning and non-reasoning judges, we show that explicit reasoning substantially improves judgment accuracy on tasks requiring structured verification (e.g., math and coding), while offering limited or even negative gains on simpler evaluations and incurring significantly higher computational cost. These findings motivate that reasoning should be used selectively rather than universally, with awareness of possible distribution shift. We propose a Robust Adaptive Cost-Efficient Routing (RACER), which dynamically selects between reasoning and non-reasoning judges under a fixed budget by formulating routing as a constrained distributionally robust optimization problem. RACER explicitly accounts for distribution shift via a KL-divergence uncertainty set, admits an efficient primal--dual algorithm, and enjoys theoretical guarantees including uniqueness of the optimal policy and linear convergence. Extensive experiments show that RACER achieves superior accuracy--cost trade-offs under distribution shift.

Wenbo Zhang, Lijinghua Zhang, Liner Xiang +1

Department of Statistics, University of California, Irvine, USA.

LLM-as-a-Judge Cost-Aware Inference LLM Routing

Ask PDF

May 11, 2026

Exploration-Driven Optimization for Test-Time Large Language Model Reasoning

Post-training techniques combined with inference-time scaling significantly enhance the reasoning and alignment capabilities of large language models (LLMs). However, a fundamental tension arises: inference-time methods benefit from diverse sampling from a relatively flattened probability distribution, whereas reinforcement learning (RL)-based post-training inherently sharpens these distributions. To address this, we propose Exploration-Driven Optimization (EDO), which extends reward-biasing style exploration objectives to iterative post-training and integrates them into standard RL objectives, encouraging greater diversity in sampled solutions while facilitating more effective inference-time computation. We incorporate EDO into iterative Direct Preference Optimization (iDPO) and Group Relative Policy Optimization (GRPO), resulting in two variants: ED-iDPO and ED-GRPO. Extensive experiments demonstrate that both ED-iDPO and ED-GRPO exhibit greater solution diversity and improved reasoning abilities, particularly when combined with test-time computation techniques like self-consistency. Across three in-distribution reasoning benchmarks, EDO achieves a 1.0-1.3% improvement over the strongest baselines, and delivers an additional 1.5% average gain on five out-of-distribution tasks. Beyond accuracy, EDO preserves model entropy and stabilizes RL training dynamics, highlighting its effectiveness in preventing over-optimization collapse. Taken together, these results establish EDO as a practical framework for balancing exploration and exploitation in LLM reasoning, especially in settings that rely on test-time scaling.

Changhao Li, Yuchen Zhuang, Chenxiao Gao +4

Georgia Institute of Technology

RL for Language Models Inference-Time Optimization RL for Language Model Reasoning RL Post-Training LLM Reasoning

Ask PDF