SAGE: Mitigating Long-Horizon Reasoning Biases via Topological Guidance

Xinyue Zeng; Jiawei Zhang; Yujun Yan; Dawei Zhou

cs.AISep 24, 2026

SAGE: Mitigating Long-Horizon Reasoning Biases via Topological Guidance

Authors: Xinyue Zeng, Jiawei Zhang, Yujun Yan, Dawei Zhou

Organizations: CS Department Virginia Tech · CS Department University of Wisconsin Madison · CS Department Dartmouth College

Abstract

Long-horizon reasoning remains a central challenge for large language models (LLMs) under sparse-reward regimes. We argue that this brittleness arises from two biases induced by complex reasoning spaces: an exploration bias, where models are drawn toward locally plausible but structurally unstable branches, and a compounding bias, where small local deviations accumulate across depth and suppress rare rewards. We introduce Symbolic Closure Analysis (SCA) as a theoretical lens characterizing how branching structures and sparse rewards induce these biases in long-horizon reasoning with local admissibility, and as a design principle for structural priors in less formal reasoning tasks. Motivated by this analysis, we propose SAGE (Structural Admissibility-Guided Exploration), a unified framework that injects structural guidance to alleviate exploration bias and compounding bias in long-horizon reasoning. SAGE combines two complementary structural guidance: algebraic sparsification, which projects locally admissible candidates onto operator-indexed algebraic subspaces to suppress spurious branching and mitigate exploration bias, and hyperbolic structural guidance, which embeds reasoning states into a negatively curved space to provide dense depth-wise signals and mitigate compounding bias. Across 12 benchmarks and 7 model families, SAGE outperforms competitive baselines. In particular, SAGE achieves up to an 8-fold improvement on the Andrews-Curtis problem, an open real-world long-horizon task. Code is available at: https://github.com/Susan571/SAGE-NeurIPS2026.

Figures & tables

Figure 1 : AC problem illustrates two long-horizon reasoning biases: exploration bias from many locally valid but low-promise branches, and compounding bias from early plausible deviations whose failures appear in later steps.

Figure 2 : Overview of SAGE workflow. Outcome-only training induces exploration and compounding biases; SAGE injects algebraic sparsification and hyperbolic structural guidance during RL, yielding focused search, stable reasoning, and no additional inference-time cost.

Flagship
Model	MATH	Minerva	Olympiad	AIME24	AMC23	GSM8K	Putnam	Avg.
Llama-3.3-70B-Instruct	66.08	33.61	32.94	17.31	29.02	80.47	9.91	38.48
2B models
Qwen3.5	50.64	11.62	24.58	9.41	43.36	47.38	3.66	27.24
Qwen3.5 w/SFT	60.41	26.52	27.96	3.74	37.88	55.06	4.63	30.89
Qwen3.5 w/GRPO	73.39	33.27	33.86	16.05	50.94	63.12	6.79	39.63

Table 1 : Accuracy (%) on mathematical reasoning benchmarks. Best in bold , second best underlined .

9B models
Model	STEM	MMLU-Pro				GPQA	BBH-H	ARC-C
		Humanity	Social	Other	Avg.
Qwen3.5	12.71	8.02	14.95	10.21	11.06	10.91	21.58	18.49
Qwen3.5 w/SFT	20.18	11.36	28.97	19.22	19.85	12.31	32.44	29.56
Qwen3.5 w/GRPO	33.38	28.31	50.41	39.26	39.33	18.21	41.69	37.82
Qwen3.5 w/EMPO	32.57	27.32	48.73	37.69	37.91	21.11	44.04	39.73

Table 2 : Accuracy (%) on free-form natural reasoning benchmarks. The best is in bold with second best in underline .

Figure 3 : Comparative performance of models with SAGE versus base models across two primary metrics: AC Validity (Left) and Lean-Verified Proofs (Right). Numbers annotated above bars indicate the absolute percentage point improvement.

Variant	Olympiad		BBH-H
	Acc	Rew./1k	Acc	Rew./1k
GRPO	38.82	16.08	41.69	20.06
EMPO	37.03	15.03	44.04	19.02
SAGE w/o $\Psi_{\mathcal{H}}$	39.84	18.47	39.06	22.36
SAGE w/o $\Psi_{\mathcal{P}}$	39.12	17.68	38.85	21.18
SAGE w/ Euclidean	38.01	16.92	38.02	20.89

Table 3: Core ablations on Qwen3.5-9B. All variants share the same entropy-filtered training subset, rollout budget, decoding constraints, optimization steps, and KL coefficient.

Method	Olym.	BBH-H	Lean
GRPO	65.31	63.29	14.64
GRPO-PRM	63.27	59.12	17.36
SAGE	68.26	69.07	23.69

Table 4 : Left: comparison with a learned process-reward baseline (Qwen3.5-35B). Right: component ablation on AC task (Qwen3.5-35B).

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4 : Hyperparameter ablation on the Olympiad dataset. Left: SAGE scales consistently across Pass@ $K$ budgets. Middle: SAGE remains robust under different decoding temperatures. Right: SAGE is less sensitive to the KL coefficient $\beta_{\mathrm{KL}}$ than GRPO.

9B models
Model	STEM	MMLU-Pro				GPQA	BBH-H	ARC-C
		Humanity	Social	Other	Avg.
Qwen3.5	12.71 $\pm$ 0.21	8.02 $\pm$ 0.18	14.95 $\pm$ 0.25	10.21 $\pm$ 0.20	11.06 $\pm$ 0.16	10.91 $\pm$ 0.31	21.58 $\pm$ 0.38	18.49 $\pm$ 0.35
Qwen3.5 w/SFT	20.18 $\pm$ 0.34	11.36 $\pm$ 0.27	28.97 $\pm$ 0.43	19.22 $\pm$ 0.36	19.85 $\pm$ 0.31	12.31 $\pm$ 0.42	32.44 $\pm$ 0.55	29.56 $\pm$ 0.51
Qwen3.5 w/GRPO	33.38 $\pm$ 0.56	28.31 $\pm$ 0.49	50.41 $\pm$ 0.62	39.26 $\pm$ 0.55	39.33 $\pm$ 0.47	18.21 $\pm$ 0.73	41.69 $\pm$ 0.84	37.82 $\pm$ 0.79
Qwen3.5 w/EMPO	32.57 $\pm$ 0.53	27.32 $\pm$ 0.51	48.73 $\pm$ 0.66	37.69 $\pm$ 0.58	37.91 $\pm$ 0.50	21.11 $\pm$ 0.69	44.04 $\pm$ 0.82	39.73 $\pm$ 0.76

Appendix

Table 5 : Accuracy (%) on free-form natural reasoning benchmarks over 5 random seeds, reported as mean with standard deviation . The best mean is in bold with second best in underline .

Explore similar work

Stabilizing Efficient Reasoning with Step-Level Advantage Selection
Apr 27, 2026Han Wang, Xiaodong Yu, Jialian Wu +4LLM Reasoning Strategies Step-Level Advantage Estimation
UNC Chapel Hill · Advanced Micro Devices, Inc.
SAGE: Shaping Anchors for Guided Exploration in RLVR of LLMs
May 15, 2026Chanuk Lee, Minki Kang, Sung Ju HwangReinforcement Learning With Verifiable Reward LLM Reasoning Strategies
KAIST
SAM: State-Adaptive Memory for Long-Horizon Reasoning Agent
May 23, 2026Yuyang Hu, Hongjin Qian, Shuting Wang +5Long-Horizon Agents Long-Horizon Task Planning
GSAI, Renmin University of China · Beijing Academy of Artificial Intelligence

Apr 27, 2026, cs.CLJ/K move · Enter open · S save

Stabilizing Efficient Reasoning with Step-Level Advantage Selection

Han Wang, Xiaodong Yu, Jialian Wu, Jiang Liu+3

UNC Chapel Hill · Advanced Micro Devices, Inc.

LLM Reasoning Strategies Step-Level Advantage Estimation Post-Training

Open paper AskPDF arXiv

Large language models (LLMs) achieve strong reasoning performance by allocating substantial computation at inference time, often generating long and verbose reasoning traces. While recent work on efficient reasoning reduces this overhead through length-based rewards or pruning, many approaches are post-trained under a much shorter context window than base-model training, a factor whose effect has not been systematically isolated. We first show that short-context post-training alone, using standard GRPO without any length-aware objective, already induces substantial reasoning compression-but at the cost of increasingly unstable training dynamics and accuracy degradation. To address this, we propose Step-level Advantage Selection (SAS), which operates at the reasoning-step level and assigns a zero advantage to low-confidence steps in correct rollouts and to high-confidence steps in verifier-failed rollouts, where failures often arise from truncation or verifier issues rather than incorrect reasoning. Across diverse mathematical and general reasoning benchmarks, SAS improves average Pass@1 accuracy by 0.86 points over the strongest length-aware baseline while reducing average reasoning length by 16.3%, yielding a better accuracy-efficiency trade-off.

Abstract

Explore similar work

Stabilizing Efficient Reasoning with Step-Level Advantage Selection

SAGE: Shaping Anchors for Guided Exploration in RLVR of LLMs

SAM: State-Adaptive Memory for Long-Horizon Reasoning Agent