cs.LGNov 8, 2026

The Race Is Decided Early! A Kernel Law Predicts Lock-In in Majority-Vote Test-Time RL

Authors: Jaeho Hwang, Sehong Park, Daejin Choi, Uiwon Hwang

Organizations: Human-Centered Artificial Intelligence Research Institute, Ewha Womans University · Department of Artificial Intelligence, Ewha Womans University · Department of Computer Science and Engineering, Ewha Womans University

Abstract

Test-time reinforcement learning with majority-vote rewards trains a language model on unlabeled questions by rewarding the samples that agree with its most frequent answer. During training, each question locks into one answer, on hard questions often a wrong one, but which answer it locks into is known only after training. In this paper, we show that this outcome is decided early, in a short race driven by sampling noise between the answers that are close at the start, and we derive the probability of each outcome. With a separate logit for each answer, the runner-up wins with a normal tail probability of the initial gap, scaled by an explicit noise constant. In a network, however, the answer logits depend on shared parameters, and through the kernel of these parameters the updates of the trailing answers also change the gap of the two leaders. This spillover shifts the boundary between the starts won by each leader, and a closed form computes the shifted probability from a kernel probe and the start logits. In LoRA experiments with three models from two families, predictions made before training with the kernel frozen at its start value forecast which answer each question locks into better than a logistic model refitted on the outcomes, although they use no fitted constants. The closed form matches these predictions on the runs in which a check before training finds that the two leaders stay above the other answers, and it explains why a law that ignores the spillover needs a strongly inflated step.

Figures & tables

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. CoRE: Consensus Rewards via Equilibrium for Test-Time Reinforcement Learning

    Aug 10, 2026Ambuj Mehrish, Sebastiano VasconReward ModelingRL for Language Models

  2. Adapter Thickets: Splitting an RLVR Budget Beats Concentrating It

    Oct 1, 2026Jonathan Williams, Esin Tureci, Karthik R. NarasimhanRL for Language Model ReasoningLow-Rank Adaptation