Decoupled Marginal Sharpening for Training-Free Inference-Time Scaling

Weixin Wang; Wei Deng; Anderson Schneider; Yuriy Nevmyvaka; Pan Xu; Andrew Bennett

cs.LGOct 14, 2026

Decoupled Marginal Sharpening for Training-Free Inference-Time Scaling

Authors: Weixin Wang, Wei Deng, Anderson Schneider, Yuriy Nevmyvaka, Pan Xu, Andrew Bennett

Organizations: Duke University · Morgan Stanley

Abstract

We study how to improve LLM reasoning at inference time without updating model weights or using external reward models. Recent reward-free power-sampling methods define the sequence-level power target $π_α(y\mid x)\propto p_0(y\mid x)^α$ , where $α>1$ , which increases the relative probability of completions that are already likely under the base model. Existing methods use Metropolis--Hastings or Sequential Monte Carlo (SMC) to approximately sample from this target. However, two challenges arise at different levels. At the target level, full-sequence sharpening can concentrate probability on a few exact reasoning--answer trajectories, suppressing alternative paths that support the same answer. At the sampler level, finite-particle SMC can suffer from path collapse when resampling repeatedly duplicates a few high-weight prefixes. To address these challenges, we propose Decoupled Marginal Sharpening (DMS), which separately controls how strongly the model favors particular reasoning paths and how strongly it favors final answers after aggregating support across those paths. At the target level, DMS first samples from a sharpened reasoning distribution, then marginalizes the model's answer probabilities over the resulting weighted reasoning population and sharpens the induced answer marginal. Separate exponents allow mild reasoning sharpening to encourage exploration of diverse paths, followed by stronger answer-marginal sharpening to concentrate probability on answers with greater aggregate support across those paths. At the sampler level, we develop Fully Adapted Annealed SMC, which combines an annealed sequence of power targets with locally fully adapted proposals and predictive ancestor resampling. Experiments on MATH500, GPQA-Diamond, HumanEval, and AIME24/25 across three base models show that DMS achieves the highest mean pass@1 among the compared methods in all tasks.

Figures & tables

Figure 1: Accuracy and inference latency across three models and four reasoning benchmarks. Our method DMS achieves the highest accuracy in all settings while requiring less inference time than the strong baseline Marginal Sharpening. Vertical error bars indicate

\pm 1

standard deviation in pass@1. In each subplot, higher accuracy and lower latency are better (toward the upper left).

Figure 2: Overview of DMS. FAA-SMC constructs

K

weighted reasoning paths that define an empirical answer marginal. We sample

M

candidates from this marginal and select the final answer according to their normalized answer-sharpening weights.

Model	Method	MATH500	GPQA-Diamond	HumanEval	AIME24/25
Qwen3-4B	Standard Decoding	$76.00\pm 0.53$	$50.00\pm 1.75$	$84.96\pm 0.35$	$59.44\pm 3.47$
	Low-Temperature Decoding	$74.07\pm 0.81$	$50.84\pm 2.59$	$84.55\pm 1.27$	$53.33\pm 7.64$
	Power-SMC	$77.67\pm 0.95$	$49.83\pm 2.92$	$84.15\pm 1.61$	$48.89\pm 6.31$
	Marginal Sharpening	$75.67\pm 0.23$	$54.21\pm 1.77$	$90.04\pm 0.93$	$71.67\pm 1.67$
	DMS (Ours)	$\mathbf{85.67\pm 0.58}$	$\mathbf{60.77\pm 0.77}$	$\mathbf{95.53\pm 0.70}$	$\mathbf{75.00\pm 1.67}$
Qwen3-8B	Standard Decoding	$71.73\pm 0.58$	$50.67\pm 3.36$	$83.74\pm 0.35$	$55.56\pm 2.55$

Table 1: Pass@1 (%) across models and benchmarks. Shaded rows denote DMS, and boldface indicates the highest mean within each model–benchmark setting. The DMS results on AIME24/25 use the Island-DMS variant.

Variant	$\alpha_{Z}$	$\alpha_{A}$	Expected	Pass@1	Oracle pass@32
Structured decoding only	1	1	66.48	67.78	82.78
Answer sharpening only	1	4	76.89	77.22	82.78
Reasoning sharpening only	1.005	1	71.77	72.78	78.33
Standard DMS	1.005	4	74.55	74.44	78.33
Island-DMS	1.005	4	76.83	77.22	81.67

Table 2: Component analysis on AIME24/25 using Qwen3-8B. Expected accuracy is the probability of selecting a correct answer under the normalized answer weights, averaged over problems. Pass@1 evaluates one sampled answer per problem, while oracle pass@32 is the fraction of problems with at least one correct answer among the 32 candidates. All accuracy metrics are percentages.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Model	MATH500	GPQA-Diamond	HumanEval	AIME24/25
Qwen3-4B	$(1.25,3)$	$(1.025,2)$	$(1.25,1.5)$	$(1.025,3.5)$
Qwen3-8B	$(1.025,2.5)$	$(1.025,2)$	$(1.25,1.5)$	$(1.005,4)$
DeepSeek-R1-Distill-Llama-8B	$(1.001,6)$	$(1.001,5)$	$(1.25,1.5)$	$(1.001,8)$

Appendix

Table 3: DMS reasoning and answer exponents

(\alpha_{Z},\alpha_{A})

used in the main results ( Table 1 ).

Figure 3: Expected accuracy (%) of standard DMS with Qwen3-8B on AIME24/25 across reasoning and answer exponents, averaged over three seeds.

Variant	Expected	Oracle pass@32	Final ESS $/K$	Resamples	Unique roots
standard DMS	74.55	78.33	0.765	1.39	8.17
No annealing	71.72	77.78	0.773	2.26	7.11
No resampling	72.09	80.00	0.252	0	16.00

Appendix

Table 4: Sampler ablations for non-island DMS with Qwen3-8B on AIME24/25, averaged over three seeds. Expected accuracy and oracle pass@32 are percentages. Final ESS is computed from the final reasoning weights and divided by

K=16

. Resamples and unique roots report the mean number of reasoning-stage resampling events and surviving initial-particle ancestors, respectively.

$\alpha_{Z}$	Expected accuracy ( $\alpha_{A}=4$ )	Oracle pass@32
Fine ladder
$1$	$95.06$	$98.17$
$1.025$	$96.17$	$97.97$
$1.05$	$96.59$	$97.97$
$1.1$	$96.39$	$97.97$
Extended ladder

Appendix

Table 5: Reasoning-exponent sensitivity with a higher-budget replica-exchange sampler using Qwen3-4B on HumanEval. Expected accuracy and oracle pass@32 are reported as percentages averaged over three seeds. The fine and extended ladders are run independently.

Model	Method	MATH500	GPQA-Diamond	HumanEval	AIME24/25
Qwen3-4B	Standard Decoding	50.57	124.41	43.06	249.27
	Low-Temperature Decoding	49.36	123.26	43.09	259.47
	Power-SMC	46.01	163.14	43.33	376.40
	Marginal Sharpening	140.51	710.42	103.05	1957.74
	DMS (Ours)	62.09	241.09	83.49	617.09
Qwen3-8B	Standard Decoding	57.09	133.99	43.92	261.37

Appendix

Table 6: Mean wall-clock inference time in seconds per problem, averaged over seeds.

Explore similar work

Explore Broadly, Reason Sharply: Push Small Models toward the Frontier via Sampling
Sep 29, 2026Panagiotis Theodoropoulos, Nan Jiang, Xintong Duan +4Efficient Language Model Reasoning Parallel Tempering
Georgia Institute of Technology · Morgan Stanley · University of Texas at El Paso +1
Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening
Jan 29, 2026Xiaotong Ji, Rasul Tutunov, Matthieu Zimmer +1Efficient Language Model Inference RL for Language Model Reasoning
Huawei Noah’s Ark Lab · UCL Centre for Artificial Intelligence
Self-Consistency via Marginal Sharpening
May 27, 2026Aleksei Arzhantsev, Otmane Sakhi, Nicolas ChopinEfficient Language Model Reasoning Reasoning Consistency in Language Models
Criteo AI Lab, CREST IP Paris · Criteo AI Lab · ENSAE, Institut Polytechnique de Paris

Sep 29, 2026J/K move · Enter open · S save

Explore Broadly, Reason Sharply: Push Small Models toward the Frontier via Sampling

Panagiotis Theodoropoulos, Nan Jiang, Xintong Duan, Ali Hasan+3

Georgia Institute of Technology · Morgan Stanley · University of Texas at El Paso+1

Efficient Language Model Reasoning Parallel Tempering Inference-Time Scaling

Open paper AskPDF arXiv

Power-sharpened sampling is an inference-time alternative to reinforcement-learning (RL) post-training for enhancing reasoning in large language models (LLMs). High-probability sequences are amplified under the base model without parameter updates or external rewards, avoiding the costly optimization and jagged generalization of RL. However, this approach faces a fundamental exploration--exploitation trade-off, as % strong sharpening restricts exploration, trapping samplers in plausible but incorrect reasoning trajectories, whereas weak sharpening leaves the answer distribution diffuse. To resolve this trade-off, we introduce Parallel Power Tempering (PPT), instantiating power-sharpened LLM sampling via parallel tempering. Running multiple interacting replicas in parallel at different sharpening levels allows lower-power replicas to explore diverse reasoning trajectories and higher-power chains to further exploit higher-likelihood responses favored by the sharpened target. Specifically, we tailor \method{} to inference-time sampling by mitigating a truncation bias, identified in prior power samplers, and investigate effective swap strategies under finite memory and compute budgets. Extensive experimentation shows that \method{} substantially improves single-chain power-sharpened sampling and outperforms RL-post-trained models, producing higher-quality reasoning traces and even achieving performance comparable to frontier models.

Abstract

Explore similar work

Explore Broadly, Reason Sharply: Push Small Models toward the Frontier via Sampling

Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening

Self-Consistency via Marginal Sharpening