Follow the Mixed Groups: Ridge-Tracking Gaussian Sampling for Zero-Variance Web Rollouts

Tomas Duncun; Jonah Ellison; Owen Pell; Mateo Ruiz; Hanae Fujita; Ingrid Solberg; Caleb Nguyen

cs.LGOct 18, 2026

Follow the Mixed Groups: Ridge-Tracking Gaussian Sampling for Zero-Variance Web Rollouts

Authors: Tomas Duncun, Jonah Ellison, Owen Pell, Mateo Ruiz, Hanae Fujita, Ingrid Solberg, Caleb Nguyen

Organizations: University of Michigan Ann Arbor, Michigan, USA · University of Edinburgh Edinburgh, United Kingdom

Abstract

Group relative policy optimization learns from a question only when the sampled responses disagree. A group that is entirely correct or entirely incorrect has zero reward variance and a zero advantage. On the Web, that group has already spent its searches and page fetches. Gaussian curricula aim the next batch at pass rates where a finite group can still mix. A Gaussian on the latest empirical rate treats every all-fail history as the same point, and a center stepped by the average signed change of those rates is pushed by prompts already stuck at 0 or 1. RIDGE keeps the Gaussian curriculum and changes the coordinate, the center, and the width. Each prompt carries a Beta posterior. The sampling weight is the marginal of a Gaussian target under a Gaussian approximation of that posterior, divided by a prediction of tool-call cost. The center and the width are an exponential smooth of the posterior means of groups that just came back mixed. All-correct and all-incorrect groups refresh the posterior and do not move the center. The GRPO update itself is unchanged. On two math backbones and a retrieval-augmented question answerer, RIDGE raises average accuracy by 3.4 and 2.9 points over GRPO and average exact match by 3.7 points, at 29.6% fewer math rollouts and 32.7% fewer tool calls than dynamic sampling.

Figures & tables

Figure 1. Why the Gaussian is not placed on the latest empirical rate. (a) After an all-fail history the empirical rate is

0

for every sample size, while the Jeffreys posterior mean falls as

1/(2(f{+}1))

. The dashed curve is the point-mass Gaussian weight at center

0.45

and width

0.12

, relative to zero failures. (b) For

G{=}8

the probability that a group mixes (black) stays above one half on most of

(0,1)

. The shaded bands are the two boundary layers. A Gaussian curriculum (red) selects a band inside that plateau; it is not a fit to the plateau. (c) If the pass rate does not appear in the stored value, the expected change of a stored

0

equals the current pass rate, and the expected change of a stored

1

equals that rate minus one. Mixed groups are excluded from the ridge average, so they contribute nothing to this boundary vote.

Figure 2. One GRPO step under Ridge . Prompts are scored by a Gaussian on the Beta posterior mean, divided by predicted tool-call cost. Mixed groups update the center and enter the unchanged GRPO update in Eq. ( 1 ). All-correct and all-incorrect groups refresh the Beta counts and do not move the center.

Qwen2.5-7B-Base
Method	GSM8K	MATH500	Minerva	Olympiad	AMC23	AIME24	Avg.	Roll.
GRPO	83.4	62.8	29.6	36.8	48.5	10.2	45.2	1.00
DS	83.9	64.1	30.4	37.6	49.8	11.6	46.2	1.62
GRESO	83.1	63.4	30.1	37.2	48.2	11.0	45.5	1.28
CurES	84.0	64.5	31.2	38.4	50.6	12.0	46.8	1.20
FG-ExPO	84.2	65.0	31.8	38.9	51.1	12.5	47.2	1.17

Table 1. Mathematical reasoning. Acc@8 is the mean accuracy of eight samples. Rollouts are cumulative generated responses, including groups that were not used in the update, divided by the GRPO count. Each entry is the mean of three seeds. Ridge is best on average and on the harder sets, and it is not best on GSM8K.

Method	HotpotQA	MuSiQue	2Wiki	Bamboogle	Avg.	Calls
GRPO	38.6	14.8	32.4	21.7	26.9	1.00
DS	39.4	15.6	33.1	22.6	27.7	1.71
GRESO	38.9	15.3	32.8	22.2	27.3	1.33
CurES	40.1	16.4	34.2	23.5	28.6	1.24
FG-ExPO	40.5	16.8	34.6	24.0	29.0	1.19
Ridge	41.8	18.2	36.1	26.4	30.6	1.15

Table 2. Multi-hop question answering over a Wikipedia index. EM is exact match. Tool calls count search and page-fetch actions, divided by the GRPO count. Means of three seeds.

Figure 3. Accuracy against cost, and the zero-variance fraction that remains in the update. (a) Six-benchmark Acc@8 on Qwen2.5-7B-Base against cumulative rollouts, relative to GRPO. Faint lines are the step values; bold lines are a trailing mean. The curves end at the averages in Table 1 . (b) Web exact match against cumulative tool calls. (c) Fraction of the GRPO batch whose reward variance is zero. Dynamic sampling is omitted because its retained batch is zero-variance-free by construction; its cost is the horizontal position in (a) and (b).

Method	Math avg.	Web avg.
Ridge	48.6	30.6
Empirical rate	47.4	29.2
Fixed center 0.5	46.8	28.7
Signed-change	47.2	29.0
No inflation	48.1	29.9
Uniform	45.4	27.1

Table 3. Ablations on Qwen2.5-7B-Base (math Acc@8) and Qwen2.5-7B-Instruct (web EM). Each row removes one piece of Ridge . “No cost weight” leaves the math run unchanged.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Quantity	Value
Posterior mean after 1 failure	0.250
Posterior mean after 8 failures	0.056
Posterior mean after 16 failures	0.029
Weight ratio, 16 failures over 1	0.009
$s_{8}(0.50)$	0.992
$s_{8}(0.20)$	0.832

Appendix

Table 4. Closed forms used in Figure 1 . The weight ratio is the point-mass Gaussian weight after 16 failures divided by the weight after one failure, with center

0.45

and width

0.12

. The mixed interval is where

s_{8}(p)\geq 1/2

Estimator	Mean absolute error
Ridge center	0.171
Signed-change center	0.336

Appendix

Table 5. Binomial simulation (600 prompts, group size 8, 40 steps, shared logit drift

0.06

per step). The target is the mean pass rate of prompts with mixed-group probability at least

1/2

. Errors are means over the last 20 steps.

Method	GSM8K	MATH500	Minerva	Olympiad	AMC23	AIME24	Avg.
GRPO	95.2	78.4	48.2	55.1	72.4	26.7	62.7
DS	95.4	79.6	49.5	56.4	73.8	28.3	63.8
GRESO	94.8	78.8	48.8	55.8	72.0	27.5	62.9
CurES	95.6	80.2	50.6	57.2	74.6	29.2	64.6
FG-ExPO	95.8	80.8	51.2	57.9	75.2	30.0	65.1
Ridge	95.5	82.1	53.4	60.2	77.5	33.3	67.0

Appendix

Table 6. Pass@8 on Qwen2.5-7B-Base, the fraction of problems with at least one correct sample among eight. Means of three seeds.

Method	GSM8K	MATH500	Minerva	Olympiad	AMC23	AIME24	Avg.
GRPO	97.4	88.6	61.2	68.4	86.1	38.3	73.3
DS	97.6	89.1	62.0	69.0	86.8	39.6	74.0
GRESO	97.2	88.8	61.5	68.6	86.0	39.1	73.5
CurES	97.8	89.6	62.8	69.7	87.4	40.8	74.7
FG-ExPO	98.0	90.0	63.4	70.2	87.9	41.5	75.2
Ridge	97.7	90.8	65.1	71.8	89.2	44.6	76.5

Appendix

Table 7. Pass@8 on Qwen3-8B-Base.

Method	HotpotQA	MuSiQue	2Wiki	Bamboogle	Avg.
GRPO	50.0	23.0	42.5	29.1	36.1
DS	50.6	23.6	43.0	29.8	36.8
GRESO	50.2	23.4	42.8	29.5	36.5
CurES	51.7	24.8	44.5	31.1	38.0
FG-ExPO	52.2	25.3	45.0	31.7	38.5
Ridge	53.8	27.0	46.9	34.5	40.5

Appendix

Table 8. Token F1 on the web suite, Qwen2.5-7B-Instruct.

Method	HotpotQA	MuSiQue	2Wiki	Bamboogle	Avg.	Calls
GRPO	31.2	11.4	26.8	17.5	21.7	1.00
DS	31.8	12.0	27.4	18.1	22.3	1.66
FG-ExPO	32.6	12.7	28.2	18.8	23.1	1.18
Ridge	34.0	14.1	29.6	21.2	24.7	1.16

Appendix

Table 9. Qwen3-4B-Instruct on the same Wikipedia index. EM, with tool-call multiples in the last column.

Method	GSM8K	MATH500	Minerva	Olympiad	AMC23	AIME24	Avg.
Ridge	83.8	66.4	33.5	40.6	52.8	14.7	48.6
Empirical rate	83.6	65.2	32.1	39.2	51.0	13.1	47.4
Fixed center 0.5	84.0	64.6	31.4	38.5	50.4	12.2	46.8
Signed-change	83.5	65.0	32.0	39.0	50.8	12.8	47.2
No inflation	83.9	65.8	32.8	40.0	52.0	13.9	48.1
Uniform	83.2	63.0	29.9	37.1	48.8	10.6	45.4

Appendix

Table 10. Per-benchmark ablations , Acc@8, Qwen2.5-7B-Base.

Setting	Avg.
$\lambda{=}0.05$	47.9
$\lambda{=}0.10$	48.3
$\lambda{=}0.20$ ( Ridge )	48.6
$\lambda{=}0.40$	48.4
$\lambda{=}0.80$	47.6
$G{=}4$	47.4

Appendix

Table 11. Sensitivity of math Acc@8 on Qwen2.5-7B-Base. One factor moves at a time.

Method	One page	Three or more
GRPO	52.4	22.8
FG-ExPO	53.1	24.6
Ridge	53.0	28.1

Appendix

Table 12. Where the web gain sits. Exact match on Qwen2.5-7B-Instruct, split by the number of gold Wikipedia pages.

Method	Math			Web
	1	2	3	1	2	3
GRPO	44.8	45.3	45.5	26.6	27.0	27.1
DS	45.8	46.3	46.5	27.4	27.8	27.9
GRESO	45.1	45.6	45.8	27.0	27.4	27.5
CurES	46.4	46.9	47.1	28.2	28.6	28.8
FG-ExPO	46.9	47.4	47.5	28.7	29.1	29.2

Appendix

Table 13. Seeds. Math Acc@8 and web EM, averaged over benchmarks, for three seeds. The main tables report these means.

Adaptive Sampling

Adaptive sampling allocates limited data, measurements, or computation where they are expected to be most useful.

See all 202 papers

Prompts Live on an Arc: Gaussian Curricula in Fisher--Rao Coordinates for Rollout-Efficient GRPO
Sep 29, 2026Mei Okonkwo, Pixel Nomand, Julian Berg +5Prompt Selection for RL Group Relative Policy Optimization
University of Wisconsin–Madison · University of Washington
Where Groups Split: A Probe Prior for Nonzero Within-Group Variance in RL Post-Training
Oct 18, 2026Jonah Ellison, Mateo Ruiz, Owen Pell +4RL for Language Model Reasoning RL Post-Training
University of Michigan · University of Edinburgh
WS-GRPO: Weakly-Supervised Group-Relative Policy Optimization for Rollout-Efficient Reasoning
Feb 19, 2026Gagan Mundada, Zihan Huang, Rohan Surana +8Efficient Language Model Reasoning RL for Language Model Reasoning
University of California, San Diego · University of Toronto · 3Adobe Research +2

Sep 29, 2026J/K move · Enter open · S save

Prompts Live on an Arc: Gaussian Curricula in Fisher--Rao Coordinates for Rollout-Efficient GRPO

Mei Okonkwo, Pixel Nomand, Julian Berg, Elena Voss+4

University of Wisconsin–Madison · University of Washington

Prompt Selection for RL Group Relative Policy Optimization RL for Language Model Reasoning Sample-Efficient RL Curriculum RL

Open paper AskPDF arXiv

Group relative policy optimization (GRPO) learns only from prompts whose sampled responses disagree: a group that is entirely correct or entirely incorrect has zero reward variance, contributes no gradient, and still consumes its rollouts. Prompt-selection methods reduce this waste by steering sampling toward intermediate pass rates, but they choose the target, its width, and the uncertainty model heuristically, in raw pass-rate or logit coordinates. We show that GRPO comes with a natural coordinate for pass rates: the arc length

ψ=\arcsin\sqrt{p}

on the Bernoulli Fisher--Rao manifold. In arc length, the expected GRPO update is uniform up to two boundary ramps; the probability of a zero-variance group is bounded by two Gaussian boundary layers of width

1/\sqrt{2G}

; pass-rate evidence has constant noise; and the gradients of the pass@

k

and pass

^k

objectives are Gaussians whose center and width follow from

k

in closed form. A prompt curriculum for GRPO is therefore a Gaussian in arc length, and choosing its center amounts to choosing the objective. We turn this observation into ARCUS, a drop-in sampler that tracks every prompt with a Kalman filter in arc length, scores prompts by an objective-matched Gaussian kernel times the predicted probability of an informative group, keeps only informative groups for the unchanged GRPO update, and paces the target toward the hardest objective whose predicted yield stays within a small slack of the best. Across six mathematical reasoning benchmarks and three backbones, ARCUS improves the average accuracy of GRPO by 2.8--2.9 points and that of dynamic sampling by 1.1--1.2 points, while generating 48--57% fewer rollouts than dynamic sampling.

Abstract

Adaptive Sampling

Explore similar work

Prompts Live on an Arc: Gaussian Curricula in Fisher--Rao Coordinates for Rollout-Efficient GRPO

Where Groups Split: A Probe Prior for Nonzero Within-Group Variance in RL Post-Training

WS-GRPO: Weakly-Supervised Group-Relative Policy Optimization for Rollout-Efficient Reasoning