cs.CLNov 11, 2026

Outside the Purview of the Reward: Capability Drift in Multilingual Post-Training

Authors: Sutanshu Saksena Raj, Purusharth Malik, Pasunuti Prasanjith, Daksh Maan, Parijat Chatterjee, Avinash Benki

Abstract

Post-training can improve rewarded behavior while weakening capabilities users still need. We study two GRPO branches from a shared supervised checkpoint in a multilingual mixture-of-experts model. The released branch gains 36.07 points on MATH but loses 19.32 F1 points on XQuAD-Indic, while XQuAD-English improves under the same scoring rule. An independently trained sibling adds Indic instruction following and language-aware shaping. It recovers the regressing Indic task scores and preserves MATH, but does not restore script compliance and trades off competition mathematics and code. In a later round, judge training is followed by a sharp extraction decline from the Indic branch; reward-free distillation from its earlier checkpoint restores most of the lost F1. A MATH score decomposition associates most of the measured gain with increased boxed-answer emission; accuracy among boxed answers changes little. These results show why benchmark scores should be read alongside the response forms they reward.

Figures & tables

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. What Does Post-Training Change in Multilingual Reasoning?

    Sep 29, 2026Hongyang Li, Xiao Li, Caesar Wu +2Multilingual Language Model EvaluationCross-Lingual Reasoning in Language Models