cs.LGNov 9, 2026

Spectral Rank and Step-Direction Dynamics in Language-Model Post-Training

Authors: Khurram Khalil, Khaza Anuarul Hoque

Organizations: Department of Electrical Engineering & Computer Science, University of Missouri-Columbia, MO, USA

Abstract

Language-model post-training modifies weights through cumulative updates and successive displacements, yet existing studies report seemingly conflicting spectral properties: nearly full-rank updates and low stable-rank changes. We show these observations are compatible and characterize update directions during training. For singular values following σi∝i−ασ_i \propto i^{-α} with 0<α≤1/20 < α\leq 1/2, we prove that stable rank becomes negligible relative to fixed-energy rank as dimension increases. Full singular value decompositions of selected OLMo-2-7B updates confirm this separation: three proximal policy optimization (PPO) attention updates have stable ranks of 9.6-18.3 but require 532-1,222 singular components to capture 80% of their energy. We next compute consecutive checkpoint-displacement cosines from full-model inner products. Across nine publicly released reinforcement learning from verifiable rewards (RLVR) stages, all 131 adjacent pairs have negative cosines on their native checkpoint grids, including PPO and Group Relative Policy Optimization (GRPO) trajectories. Supervised fine-tuning comparisons across four model families show configuration-dependent alignment, including negative late-training cosines for Mistral and FP32 Llama. Finally, a controlled OLMo-2-1B study with three paired seeds and a 2,048-token response cap compares group-relative advantages against a learned critic with generalized advantage estimation (GAE). At matched 60-rollout windows, late-training mean cosines are +0.259 and +0.012, respectively; the paired critic-minus-group difference is -0.247 (seed-level 95% interval [-0.358, -0.137]). These findings distinguish spectral concentration, energy coverage, and trajectory direction, and demonstrate that the advantage system influences directional alignment under matched training conditions.

Figures & tables

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. When Rank Rises as LLMs Degrade

    Oct 7, 2026Zhaohui Geoffrey WangLanguage Model MonitoringLLM Post-Training

  2. Spectrally Targeted Muon

    Oct 7, 2026Vishrut Goyal, Rohan RamkumarSpectral OptimizationDeep Learning Optimization