Better Reasoning, Not Just Bigger Models: A Structure-Grounded Framework for Efficient and Reliable Inference

Amirhossein Aghaei; Liangyan Li; Shiyi Yang; Shaohua Wang; Yan Wang

cs.AIOct 22, 2026

Better Reasoning, Not Just Bigger Models: A Structure-Grounded Framework for Efficient and Reliable Inference

Authors: Amirhossein Aghaei, Liangyan Li, Shiyi Yang, Shaohua Wang, Yan Wang

Organizations: Huawei Research Canada, Markham, Canada · Huawei Research Canada, Waterloo, Canada · Huawei Technologies Co., Ltd., Beijing, China · Central University of Finance and Economics, Beijing, China

Abstract

Scaling large language models has substantially improved linguistic fluency, yet reliable autonomous agents require more than fluent generation. In multi-turn inference, current agentic systems often produce semantically plausible but structurally ungrounded reasoning, especially under incomplete observations. We propose Structure-Grounded Uncertainty Reduction (SGUR), a training-free framework that complements token-level generation with active uncertainty reduction over structured hypothesis spaces. SGUR represents incomplete observations as a query graph and aligns them with candidate hypothesis graphs through a prior-weighted partial graph matching mechanism, where database-derived empirical priors assign greater importance to decision-relevant relations rather than treating all graph edges uniformly. The resulting inference state is iteratively refined by an active information-gain strategy that selects the next most informative node to query, systematically pruning the candidate space. We instantiate SGUR on a challenging PrimeKG-derived sequential medical diagnosis benchmark, demonstrating that structural grounding can partially substitute for model scale: starting from only two observed symptoms, SGUR achieves 92.5% Hit@3, outperforming the strongest Graph-RAG baseline by 9.4 percentage points while using approximately 5x fewer tokens. Notably, SGUR enables smaller backbone models to match or exceed the performance of models up to 10x larger, suggesting that structural grounding provides a more robust and efficient paradigm for scaling agent performance than relying exclusively on increasing model size.

Figures & tables

Figure 1: Overview of the SGUR framework. The offline phase constructs a hierarchical expert graph database $G_{\mathrm{db}}$ from expert data under a shared schema. The online phase performs closed-loop multi-turn inference through four stages: (a) query graph construction, which grounds observations $\mathcal{O}_{t}$ into $G_{q}^{(t)}$ with latent routing; (b) candidate pruning, which applies path filtering and specificity-aware structural priors to obtain ${G}_{\mathrm{cand}}^{(t)}$ ; (c) prior-weighted partial FGW alignment, which ranks candidate graphs by structural consistency with the query; and (d) active acquisition, which selects informative unobserved nodes to obtain $\mathcal{O}_{t+1}$ . Across rounds, SGUR progressively contracts the candidate space under partial observation.

Method	Complex regime after R3					Complex Hit@3			Efficiency Avg.
Method	MTTC $\downarrow$	MRR $\uparrow$	Hit@1 $\uparrow$	Hit@2 $\uparrow$	Hit@3 $\uparrow$	R1	R2	R3	Tokens $\downarrow$	Latency $\downarrow$
GraphRAG	1.74	0.71	61.20	77.60	83.10	59.60	76.90	83.10	10454.00	34.90
NodeRAG	1.90	0.71	60.20	75.20	80.10	54.20	75.00	80.10	8461.10	15.20
HippoRAG2	1.83	0.68	59.50	74.50	81.70	60.40	77.40	81.70	2352.80	11.40
ArchRAG	2.01	0.65	56.70	70.40	76.10	56.70	70.40	76.10	2683.40	10.40
SGUR (Ours)	1.62	0.77	64.90	83.40	92.50	61.40	84.70	92.50	1845.40	8.90

Table 1: Main accuracy, convergence, and efficiency results. Final retrieval metrics are measured after round 3 in the complex two-symptom regime; convergence reports complex-regime Hit@3 across rounds; efficiency reports three-round average total tokens and latency. All Hit@

\!K

values are percentages. SGUR achieves the best final ranking quality, lowest MTTC, and lowest inference cost.

Method	Backbone	Hit@3 $\uparrow$	MRR $\uparrow$	MTTC $\downarrow$
GraphRAG	GPT-5.2 (L)	83.10	0.71	1.74
	GPT-5.4-nano (S)	74.70	0.62	2.02
	Retention (%)	90.30	87.10	86.10
NodeRAG	GPT-5.2 (L)	80.10	0.70	1.90
	GPT-5.4-nano (S)	73.60	0.60	2.12
	Retention (%)	91.90	84.50	89.60

Table 2: Model-scaling stress test in the complex regime. Each method is evaluated with large and small backbones; retention reports the small/large ratio. SGUR preserves over

99\%

Hit@3 after backbone scaling and remains above all large-backbone graph-RAG baselines.

Figure 2: Accuracy–efficiency Pareto frontier in the complex regime. The x-axis reports average total tokens over three interaction rounds, and the y-axis reports final Hit@3 after round 3. Higher is better on the y-axis and lower is better on the x-axis. SGUR achieves the highest retrieval accuracy with the lowest inference cost.

Variant	Final complex-regime performance				Evidence sufficiency: Hit@3
Variant	Hit@3 $\uparrow$	$\Delta$ Hit@3	MRR $\uparrow$	MTTC $\downarrow$	2 Obs.	4 Obs.	6 Obs.
Full SGUR	92.50	–	0.77	1.62	61.40	85.70	92.50
Semantic-only Alignment	91.90	$-0.60$	0.75	1.64	59.00	85.30	91.90
Structure-only Alignment	91.60	$-0.90$	0.76	1.67	56.20	84.40	91.60
Uniform Importance	83.40	$-9.10$	0.64	2.01	38.20	71.80	83.40
Full FGW	73.30	$-19.20$	0.53	2.45	15.30	59.50	73.40

Table 3: Combined ablation study in the complex regime. The left block isolates each SGUR component at the final round, while the right block reports Hit@3 as observed evidence increases. All Hit@3 and

\Delta

Hit@3 values are percentages.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Description	Value
$T$	Turn budget	3
$b$	Per-turn question batch size (complex regime)	2
$b$	Per-turn question batch size (simple regime)	4
$k$	Top- $k$ return list size	3
$\kappa$	Top- $\kappa$ candidate pool for evidence acquisition	5
$\tau_{\mathrm{sim}}$	Entity-matching cosine-similarity threshold	0.85

Appendix

Table 4: Hyperparameter settings used in all SGUR experiments unless stated otherwise.

Method	Final complex-regime performance (R3)					Simple Hit@3			Complex Hit@3
Method	MTTC $\downarrow$	MRR@R3 $\uparrow$	Hit@1 $\uparrow$	Hit@2 $\uparrow$	Hit@3 $\uparrow$	R1	R2	R3	R1	R2	R3
GraphRAG	1.74	0.71	61.20	77.60	83.10	64.20	81.30	84.50	59.60	76.90	83.10
NodeRAG	1.90	0.71	60.20	75.20	80.10	63.00	79.00	86.00	54.20	75.00	80.10
HippoRAG2	1.83	0.68	59.50	74.50	81.70	68.30	82.90	87.70	60.40	77.40	81.70
ArchRAG	2.01	0.65	56.70	70.40	76.10	58.20	74.10	80.90	56.70	70.40	76.10
SGUR (Ours)	1.62	0.77	64.90	83.40	92.50	67.10	86.70	93.40	61.40	84.70	92.50

Appendix

Table 5: Full retrieval performance and multi-round convergence. Final metrics are measured after round 3 in the complex two-symptom regime, while the simple and complex columns report Hit@3 across interaction rounds. All Hit@

\!K

values are percentages. SGUR achieves the best final ranking quality and the lowest MTTC, and its advantage becomes more pronounced as evidence accumulates.

Method	Input Tokens $\downarrow$			Output Tokens $\downarrow$			Total Tokens $\downarrow$			Latency (s) $\downarrow$
Method	R1	R2	R3	R1	R2	R3	R1	R2	R3	R1	R2	R3
GraphRAG	9886.00	10076.00	10080.00	376.00	452.00	491.00	10263.00	10529.00	10570.00	34.10	35.00	35.50
NodeRAG	7609.00	8118.40	8394.70	337.40	427.90	496.00	7946.40	8546.30	8890.70	14.10	15.10	16.40
HippoRAG2	2014.20	2058.90	2116.10	249.40	293.50	326.20	2263.70	2352.40	2442.40	10.10	11.00	13.00
ArchRAG	2223.40	2283.10	2384.40	326.10	402.30	431.00	2549.50	2685.40	2815.40	9.50	10.40	11.20
SGUR (Ours)	1755.50	1759.70	1765.50	78.60	79.40	85.40	1838.00	1843.20	1854.90	8.10	9.10	9.70

Appendix

Table 6: Full per-round inference efficiency breakdown. We report input tokens, output tokens, total tokens, and end-to-end latency across three interaction rounds. SGUR keeps total token usage nearly flat from R1 to R3 and achieves the lowest latency in every round, confirming that structured query-state updates avoid the growing cost of re-injecting accumulated dialogue history.

Method	Backbone	Hit@1	Hit@2	Hit@3	MRR	MTTC
GraphRAG	GPT-5.2 (L)	61.20	77.60	83.10	0.71	1.74
	GPT-5.4-nano (S)	52.10	64.40	74.70	0.62	2.02
	Retention (%)	85.10	83.00	90.30	87.10	86.10
NodeRAG	GPT-5.2 (L)	60.20	75.20	80.10	0.70	1.90
	GPT-5.4-nano (S)	48.50	65.40	73.60	0.60	2.12
	Retention (%)	80.60	87.00	91.90	84.50	89.60

Appendix

Table 7: Full model-scaling stress test in the complex regime. Each method is evaluated under a Large (L) and Small (S) backbone, with retention reported in the third row of each method. Hit@

k

and retention values are percentages. Retention is computed as Small/Large for Hit@

k

and MRR; for MTTC, where lower is better, retention is computed as

\min(\mathrm{L},\mathrm{S})/\max(\mathrm{L},\mathrm{S})

. All metrics are computed after round 3.

LLM Reasoning with Graphs

Large language model reasoning with graphs uses structured relationships to support multi-step inference, evidence retrieval, and planning.

See all 211 papers

Improving reasoning at inference time via uncertainty minimisation
Mar 7, 2026Nicolas Legrand, Kenneth Enevoldsen, Márton Kardos +1Efficient Language Model Reasoning Inference-Time Scaling
Center for Humanities Computing Aarhus University, Denmark
Pause or Fabricate? Training Language Models for Grounded Reasoning
Apr 21, 2026Yiwen Qiu, Linjuan Wu, Yizhou Liu +9LLM Grounding RL for Language Model Reasoning
1Zhejiang University · 2Tencent · 3Xiaohongshu Inc.
Eigen-Agent: Adaptive Multi-Agent Scientific Reasoning with Monitor-Based RAG
Sep 25, 2025Xiangru Tang, Wanghan Xu, Yujie Wang +13Scientific Reasoning in Language Models Multi-Agent Reasoning
Yale University · Shanghai Jiao Tong University · Fudan University +4

Mar 7, 2026J/K move · Enter open · S save

Improving reasoning at inference time via uncertainty minimisation

Nicolas Legrand, Kenneth Enevoldsen, Márton Kardos, Kristoffer Nielbo

Center for Humanities Computing Aarhus University, Denmark

Efficient Language Model Reasoning Inference-Time Scaling Language Model Decoding

Open paper AskPDF arXiv

Large language models (LLMs) now exhibit strong multi-step reasoning abilities, but existing inference-time scaling methods remain computationally expensive, often relying on extensive sampling or external evaluators. We propose a principled strategy that frames reasoning as uncertainty minimisation and operates at the level of individual thoughts rather than tokens. Our method selects, at each reasoning step, the continuation that maximizes the model's self-certainty, a metric computed from its internal predictive distribution. This approach achieves significant improvement with a small number of samples, relies exclusively on model-internal signals, and applies to open-ended questions as opposed to methods like majority voting. Experiments on MATH500 and GSM8K across multiple model sizes demonstrate that thought-level self-certainty maximization consistently outperforms greedy decoding and matches or exceeds self-consistency under comparable token budgets. Cross-linguistic evaluations further indicate that the method transfers robustly beyond high-resource languages. Furthermore, analysis of self-certainty dynamics reveals that correct reasoning trajectories converge early to stable paths, suggesting that early decisions, likely associated with the planning of the reasoning process, are predictive of final accuracy. Building on this result, we show that self-certainty maximisation applied to the early steps can explain most of the performance gain and provide a simple yet efficient inference-time scaling method.

Abstract

LLM Reasoning with Graphs

Explore similar work

Improving reasoning at inference time via uncertainty minimisation

Pause or Fabricate? Training Language Models for Grounded Reasoning

Eigen-Agent: Adaptive Multi-Agent Scientific Reasoning with Monitor-Based RAG