Post-training can improve rewarded behavior while weakening capabilities users still need. We study two GRPO branches from a shared supervised checkpoint in a multilingual mixture-of-experts model. The released branch gains 36.07 points on MATH but loses 19.32 F1 points on XQuAD-Indic, while XQuAD-English improves under the same scoring rule. An independently trained sibling adds Indic instruction following and language-aware shaping. It recovers the regressing Indic task scores and preserves MATH, but does not restore script compliance and trades off competition mathematics and code. In a later round, judge training is followed by a sharp extraction decline from the Indic branch; reward-free distillation from its earlier checkpoint restores most of the lost F1. A MATH score decomposition associates most of the measured gain with increased boxed-answer emission; accuracy among boxed answers changes little. These results show why benchmark scores should be read alongside the response forms they reward.
Figures & tables
Figure 1: Checkpoint lineage across post-training rounds. Judge-B and Judge-I each have separate base and + configurations; + marks the auxiliary reward bundle. The dashed arrow marks RL-Indic as the distillation teacher.
Benchmark
Input
Passage
Recovered
Output script
Grader
ΔBase
language
supplied
from it
constrained
MATH
En
no
no
no
judge, final answer
+36.07
GPQA-Diamond
En
no
no
no
exact match, option
+26.84
IFBench
En
no
no
no
constraint checker
+22.66
IFEval
En
no
no
no
constraint checker
+16.45
MMLU-Pro
En
no
no
no
exact match, option
+10.59
Table 1: Benchmark-property matrix for comparison A. Extractive rows report F1. The matrix distinguishes source text supplied in a prompt from answers graded against a span of that text. XQuAD-English and XQuAD-Indic share these properties but move in opposite directions. Question marks identify unverified benchmark properties.
Benchmark
SFT-Init
RL-Base
RL-Indic
ΔB
Indic regressions
XQuAD-Indic (EM)
18.84
12.86
32.00
+19.14
IndicIFEval-ground
40.52
33.05
60.00
+26.95
IndicQA (EM)
12.77
12.22
23.46
+11.24
IFEval-Hindi
52.71
50.59
72.29
+21.70
Other reasoning and instruction-following tasks
Table 2: Indic recovery and its trade-offs. ΔB is the directly measured RL-Indic minus RL-Base difference from comparison B. SFT-Init scores come from comparison A, so comparisons with that column assume campaign comparability. Scores are percentages, and deltas are percentage points. Table 19 gives the complete comparison.
Measurement
SFT-Init
RL-Base
Boxed-answer emission
55.2
91.6
Accuracy given a boxed answer
91.1
92.6
Accuracy given an unboxed answer
1.5
11.4
Component of pooled gain
Points
Share
Boxed-answer emission rate
33.13
95.2%
Accuracy given a boxed answer
1.39
4.0%
Table 3: MATH performance and answer conventions in comparison A. Rates are percentages. The gain decomposition pools all 5,000 items, unlike the scorecard’s average over seven subtasks. Emission-rate change is weighted by the initial conditional accuracy. Conditional-accuracy change is weighted by the released branch’s emission rate. The pooled gain is 34.80 points, compared with 36.07 on the scorecard.
Checkpoint
Span
Script
EM
F1
RL-Base
14
–
12.65
17.17
RL-Indic
18
80.0
32.00
50.44
Judge-B , step 50
153
43.7
3.54
15.10
Judge-B+ , step 40
210
11.2
1.87
7.67
Judge-B+ , step 50
83
53.1
8.49
22.79
RL-OPD , step 20
26
83.2
33.52
47.13
Table 4: Answer form and score on XQuAD-Indic, measured in one harness with reasoning enabled. Span is the median length in characters of the text after the final reasoning block, since the harness recorded no token counts. Script is the share of answers that are at least half Indic characters, and was not measured for RL-Base . The RL-Base row is an independent scoring of the checkpoint in Table 8 , so it differs slightly from that column. A harness change made exact match more permissive after August 2026, which separates the first two rows from the rest; F1 is unaffected, and we read every comparison across that boundary on F1.
Round
Checkpoint
EM
F1
1, verifiable rewards
RL-Indic
32.00
50.44
2, judge reward
Judge-I+ , step 20
1.91
17.67
3, no reward
RL-OPD , step 20
33.52
47.13
Table 5: XQuAD-Indic across three post-training rounds on the RL-Indic branch, each run for twenty steps. RL-OPD has been scored on this benchmark, not on the full matched suite. Rounds two and three use a more permissive exact match than round one, so we read the recovery on F1.
Figure 2: Tool-use trade-offs under different comparison baselines. (A) Direct RL-Indic minus RL-Base changes in percentage points, with branch scores beneath each benchmark name. (B) BFCL category accuracies relative to SFT-Init . Parallel calls deteriorate while sequential interaction improves. Given a second turn, the sibling supplies the missing call in 95 of 96 examined cases.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Benchmark
Earlier delta
Corrected delta
MMLU
−0.78
+9.47
CrossSum-Indic (chrF)
−10.87
+5.81
HumanEval+
+2.26
−0.12
AIME 2025
+40.00
+1.46
Appendix
Table 6: Changes to reported campaign-A deltas after correcting the SFT baseline.
Setting
Value
Export point
partway through the final pass, verified from export metadata
Scheduled sequence positions
80.53B ( 2,400×256×131,072 ), including padding
Approximate loss-bearing positions
∼ 15B (18.2%, assistant-only mask)
Trainable-rate measurement
95,515 / 524,288 positions over four packed rows
Intra-pack packing efficiency
99.98%
Prepared packed capacity
92.12B positions across 702,824 sequences
Appendix
Table 7: Training record for SFT-Init , the export used to initialize both RL branches.
Benchmark
Metric
Final SFT
RL ckpt
Δ (RL − SFT)
ckpt
step 40
Math & Reasoning
MATH
exact match
48.69
84.76
+36.07
MATH-500
exact match
48.80
84.00
+35.20
GSM8K
exact match
85.75
94.31
+8.56
MGSM-English
exact match
91.60
98.40
+6.80
Appendix
Table 8: Matched evaluation of the final SFT checkpoint ( SFT-Init ) and the released RL-Base checkpoint. Both columns come from a single campaign on one harness, model mode, and decoding configuration. All 55 recorded metrics completed, displayed as 53 rows because the two IndicIFEval-ground rows each combine a prompt-level and an instruction-level score. Every row was scored within this campaign. Deltas are RL minus SFT. Accuracy-family metrics are percentages. chrF++, chrF, and BLEU are native sacreBLEU scores on the same 0–100 range and are not percentages. Rows marked ↓ are better when lower, so their delta sign is inverted relative to the rest of the table.
Setting
Value
Initialization
SFT-Init
Algorithm
GRPO with a leave-one-out baseline and per-group Bessel-corrected standard-deviation normalization, divisor σ+10−6 , no floor
Table 9: Released RL-Base configuration. No custom reward shaping is applied. Each environment returns its own binary or task-specific scalar.
Setting
Value
Pool
Rows
Share
Added environment
indic_if
Mathematics
6,192
40.5%
Reward
Rtask+0.3Rlang−0.0Rlen−0.3Rrep
Indic instruction following
3,179
20.8%
Malformed / abandoned
−0.25
Structured outputs
1,932
12.6%
Advantage std. floor
max(σ,0.05) , with 44% of indic_if groups below it
Workplace assistant
1,804
11.8%
Steps / rows / validation
40 / 15,300 / 256
English guardrail
1,509
9.9%
Recorded rollouts
251,906
MCQA
684
4.5%
Appendix
Table 10: RL-Indic configuration and input mixture after difficulty filtering and per-environment caps. Rounded shares sum to 100.1%, while row counts total 15,300. These are surviving row shares, not target weights.
Directive
Initial in-script rate
Rate after 20 steps
Explicit Hindi name
38.4%
90.5%
Implicit or referential wording
near zero
Appendix
Table 11: Language-directive diagnostic. Explicit Hindi directives improve in-script output over 20 training steps, while implicit and referential phrasings remain near zero.
Environment
we
dˉe
wedˉe
Script-free share
Structured outputs
43.4%
+0.2588
+0.1123
0.3%
Workplace assistant
19.3%
+0.2483
+0.0480
7.5%
MCQA
—
+0.1030
—
—
Mathematics with judge
—
+0.0006
—
10.1%
Aggregate dˉ
+0.1681 (SE 0.0095)
95% bootstrap CI
[+0.1489,+0.1871]
Appendix
Table 12: Reward differential and environment decomposition in the retained rollout sample. The final column separately reports script-free answers as a share of Indic-prompt rollouts, not of paired groups. Dashes indicate unreported values.
Retained file
Mean reward
Logged reference
Early-level file A
0.324
0.333 (steps 1–8)
Early-level file B
0.312
0.333 (steps 1–8)
Late-level file
0.717
0.698 (steps 24–49)
Appendix
Table 13: Locating retained files by reward level. Four files survive, each capped at 20,000 records, from a run spanning 49 steps. The available reward-level comparisons locate three files.
Sample
dˉ
Mixed-script groups
Early file, first chunk
+0.062
232
Early file, middle chunk
+0.150
224
Early file, final chunk
+0.165
175
Late-level retained file
—
2
Fresh sample, step 20
—
1.7%
Fresh sample, step 30
—
0.9%
Appendix
Table 14: Within-file differential and fresh-checkpoint mixed-script coverage. The final column is a count for retained-file rows and a percentage of prompt groups for checkpoint samples.
Variant
dˉ
Paired groups
Primary answer-span filter
+0.1681
1,148
Script-bearing-character filter
+0.1745
1,116
No length filter
+0.066
—
Second seed, truncated sample
+0.115
554
Change from dropping versus absorbing ambiguous spans
0.0009
Unfiltered mathematics differential
−0.082
Appendix
Table 15: Sensitivity to filtering and seed. The primary filter requires a 20-character answer span. Counting 20 script-bearing characters is a distinct variant.
Step
0
20
60
100
140
160
Pairwise accuracy
0.661
0.810
0.813
0.820
0.831
0.812
Format validity
0.996
0.999
0.998
1.000
–
–
Total variation distance ↓
0.229
0.305
0.108
0.083
0.173
0.156
Appendix
Table 16: Judge validation. Step 100 is the exported checkpoint; later steps lose on both axes.
RL-Indic
Judge-I 20
Judge-I+ 20
Judge-I+ 50
IndicIFEval-ground, strict
60.00
62.11
63.96
65.06
IndicIFEval-translated, strict
63.27
59.31
59.14
58.91
IFBench, strict
50.33
44.00
45.33
40.67
AIME 2025, pass@1
65.42
67.29
67.08
66.04
XQuAD-Indic, F1
50.44
19.34
17.67
19.08
XQuAD-Indic, EM
32.00
3.79
1.91
2.85
Appendix
Table 17: Round two from the RL-Indic branch. These campaigns scored a six-benchmark subset; IndicQA is absent from all of them, and six of the eight branch-by-step combinations were never scored. Exact match here is the more permissive post-August grader, so it is not comparable with the RL-Indic column.
Aggregation
RL-Base
RL-Indic
Difference
Shared metrics, QA exact match (33)
0.661188
0.661233
+0.000045
QA rows scored by F1 instead
—
—
+0.009
Non-instruction-following subset (27)
—
—
−0.026
Appendix
Table 18: Sensitivity of the pooled comparison. Differences are RL-Indic minus RL-Base on the 0–1 scale. Unreported branch means are shown as dashes.
Benchmark
SFT
RL-Base
ΔBase
RL-Indic
ΔIndic
Math & Reasoning
MATH
48.69
84.76
+36.07
86.76
+38.07
MATH-500
48.80
84.00
+35.20
87.60
+38.80
GSM8K
85.75
94.31
+8.56
91.58
+5.83
MGSM-English
91.60
98.40
+6.80
97.60
+6.00
MGSM-Indic
79.60
84.60
+5.00
86.40
+6.80
Appendix
Table 19: Three-checkpoint comparison across campaigns A and B. The shared RL-Base scores match Table 8 . Both deltas use SFT-Init as baseline, so ΔIndic is reconstructed under the campaign-comparability assumption (§ 3 ). One benchmark-native metric is shown per benchmark. Table 8 gives the complete campaign-A metric set.
Figure 3: Task scores and answer language are distinct measurements. (A) IndicIFEval measurements of the same RL-Base checkpoint across campaigns. NA denotes an unretained metric, not zero. The dedicated campaign also records a loose score of 65.62. (B) In-script output before and after sibling training. (C) English answers on XQuAD-Indic before and after the released run. A prose-tolerant regrade changes macro F1 by at most 0.24 points.
Reinforcement learning post-training adapts pretrained models with reward or feedback signals to improve reasoning, alignment, and interactive behavior.