Language Model Post-Training

Follow this topic Ask about this topic

Momentum

23 papers in the last four weeks, up 283% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 132

All topics

Beyond Mode-Seeking RL: Trajectory-Balance Post-Training for Diffusion Language Models
May 13, 2026Saba Ahmadi, Prasanna Parthasarathi, Yufei CuiRL for Language Model Reasoning RL for Diffusion Models
Noah’s Ark Lab
TokenRatio: Principled Token-Level Preference Optimization via Ratio Matching
May 12, 2026Truong Nguyen, Tien-Phat Nguyen, Linh Ngo Van +3Pairwise Preference Learning LLM Alignment
Hanoi University of Science and Technology · University of Stuttgart · German Research Center for Artificial Intelligence +3
On Predicting the Post-training Potential of Pre-trained LLMs
May 12, 2026Xiaoyuan Li, Yubo Ma, Kexin Yang +5LLM Evaluation Language Model Post-Training
University of Science and Technology of China · Alibaba Group · National University of Singapore
Procedural-skill SFT across capacity tiers: A W-Shaped pre-SFT Trajectory and Regime-Asymmetric Mechanism on 0.8B-4B Qwen3.5 Models
May 12, 2026Igor StrozziSupervised Fine-Tuning LLM Evaluation
Applied Mathematics Department Federal University of Rio de Janeiro
YFPO: Yoked Feature Preference Optimization with Neuron-Guided Rewards
May 12, 2026Yifan LePreference Optimization Language Model Post-Training
Independent Researcher
Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillation
May 12, 2026Yuchen Cai, Ding Cao, Liang Lin +9Efficient Language Model Training On-Policy Distillation
1USTC · 2Tencent · 4HKUST(GZ) +3
From Generic Correlation to Input-Specific Credit in On-Policy Self Distillation
May 12, 2026Guobin Shen, Lei Huang, Xiang Cheng +4Reinforcement Learning On-Policy Self-Distillation
Xiaohongshu Inc. · Institute of Automation, Chinese Academy of Sciences
Annotations Mitigate Post-Training Mode Collapse
May 11, 2026Jacob Mitchell Springer, Madhu Advani, Lukas Aichberger +7Supervised Fine-Tuning Diverse Text Generation
Carnegie Mellon University · Apple · Johannes Kepler University Linz
The Extrapolation Cliff in On-Policy Distillation of Near-Deterministic Structured Outputs
May 9, 2026Xin Li, Hao Jiang, Annan Wang +2On-Policy Distillation Structured Output Generation
Nanyang Technological University, Singapore
Post-training makes large language models less human-like
May 8, 2026Marcel Binz, Elif Akata, Abdullah Almaatouq +76LLM Evaluation LLM Alignment
The Convergence Gap: Instruction-Tuned Language Models Stabilize Later in the Forward Pass
May 8, 2026Yifan ZhouLLM Interpretability Next-Token Prediction
University of California, Los Angeles
Dr. Post-Training: A Data Regularization Perspective on LLM Post-Training
May 8, 2026Pingbang Hu, Xueshen Liu, Z. Morley Mao +1Supervised Fine-Tuning Training Data Selection
University of Illinois Urbana–Champaign · University of Michigan
Post-Reasoning: Improving the Performance of Non-Thinking Models at No Cost
May 7, 2026Richmond Sin Jing Xuan, Rishabh Bhardwaj, Soujanya PoriaLLM Prompting Language Model Post-Training
Nanyang Technological University
RLearner-LLM: Balancing Logical Grounding and Fluency in Large Language Models via Hybrid Direct Preference Optimization
May 6, 2026Qiming Bao, Juho Leinonen, Paul Denny +1LLM Grounding LLM Alignment
University of Auckland · Aalto University
EvoLM: Self-Evolving Language Models through Co-Evolved Discriminative Rubrics
May 5, 2026Shuyue Stella Li, Rui Xin, Teng Xiao +8RL for Language Models LLM Evaluation
University of Washington · 2Allen Institute for AI · University of Pennsylvania
Assert, don't describe: Linguistic features that shift LLM reasoning about animal welfare
Apr 30, 2026Jasmine Brazilek, Harper DunnMoral Reasoning in Language Models Language Model Post-Training
Compassion Aligned Machine Learning (CaML) · Independent researcher
Helpfulness Hurts: Domain-Dependent Degradation of Mid-Trained Compassion Values Under Post-Training
Apr 30, 2026Jasmine Brazilek, Juliana SeawellMoral Reasoning in Language Models RL Post-Training
†Compassion Aligned Machine Learning (CaML)
Instruction-Guided Poetry Generation in Arabic and Its Dialects
Apr 30, 2026Abdelrahman Sadallah, Kareem Elozeiri, Mervat Abassy +5Arabic NLP Controllable Text Generation
Mohamed bin Zayed University of Artificial Intelligence
Backtranslation Augmented Direct Preference Optimization for Neural Machine Translation
Apr 28, 2026Mehrdad Ghassabi, Spehr Rajabi, Hamidreza Baradaran Kashani +3Language Model Post-Training Machine Translation
Faculty of Computer Engineering University of Isfahan Isfahan, Iran · School of Computer Science University of Windsor Windsor, Canada
$\mathcal{S}^2$ IT: Stepwise Syntax Integration Tuning for Large Language Models in Aspect Sentiment Quad Prediction
Apr 25, 2026Bingfeng Chen, Chenjie Qiu, Yifeng Xie +3LLM Fine-Tuning Aspect-Based Sentiment Analysis
School of Computer Science, Guangdong University of Technology · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) · Peng Cheng Laboratory +1
Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-Thought
Apr 24, 2026Keshav Ramji, Tahira Naseem, Ramón Fernandez AstudilloEfficient Language Model Reasoning Implicit Reasoning in Language Models
IBM Research AI
On Reasoning Behind Next Occupation Recommendation
Apr 23, 2026Shan Dong, Palakorn Achananuparp, Hieu Hien Mai +3Large Language Model-Based Recommendation Language Model Post-Training
Singapore Management University · Columbia University
Hierarchical Policy Optimization for Simultaneous Translation of Unbounded Speech
Apr 22, 2026Siqi Ouyang, Shuoyang Ding, Oleksii Hrinchuk +4RL for Language Models Speech Translation
1Carnegie Mellon University · 2NVIDIA
Self-Improving Tabular Language Models via Iterative Reward-Guided Post-Training
Apr 21, 2026Yunbo Long, Tejumade Afonja, Guangya Hao +2Tabular Foundation Models Group Relative Policy Optimization
Department of Engineering, University of Cambridge · CISPA Helmholtz Center for Information Security, Saarbrücken, Germany · The Alan Turing Institute, London
Train Separately, Merge Together: Modular Post-Training with Mixture-of-Experts
Apr 20, 2026Jacob Morrison, Sanjay Adhikesaven, Akshita Bhagia +3Continual Learning for LLMs Mixture-of-Experts Language Models
University of Washington · Allen Institute for AI · University of California, Berkeley
Semantic Step Prediction: Multi-Step Latent Forecasting in LLM Reasoning Trajectories via Step Sampling
Apr 20, 2026Yidi YuanNeural Representation Geometry Language Model Post-Training
Home Team Science and Technology Agency
Post-training is (Massive) Supervised Learning
Apr 20, 2026Michael Hassid, Yossi Adi, Roy SchwartzSupervised Fine-Tuning Language Model Post-Training
1FAIR, Meta AI · 2The Hebrew University of Jerusalem
PoliLegalLM: A Technical Report on a Large Language Model for Political and Legal Affairs
Apr 19, 2026Yuting Huang, Yinghao Hu, Qian Xiao +7LLM Grounding Language Model Post-Training
Zhejiang University, Hangzhou, China · Tongyi Lab, Alibaba Group, Hangzhou, China · Zhejiang Zhengfa Xinxi Guanli Zhongxin, Hangzhou, China
OPSDL: On-Policy Self-Distillation for Long-Context Language Models
Apr 19, 2026Xinsen Zhang, Zhenkai Ding, Tianjun Pan +4Long-Context Language Modeling On-Policy Self-Distillation
Baidu Inc.
Jupiter-N Technical Report
Apr 19, 2026George DraysonLLM Alignment Cultural Alignment
Locai Labs, University College London

May 13, 2026, cs.LGJ/K move · Enter open · S save

Beyond Mode-Seeking RL: Trajectory-Balance Post-Training for Diffusion Language Models

Saba Ahmadi, Prasanna Parthasarathi, Yufei Cui

Noah’s Ark Lab

RL for Language Model Reasoning RL for Diffusion Models Diffusion Language Models Language Model Post-Training

Open paper AskPDF arXiv

Diffusion language models are a promising alternative to autoregressive models, yet post-training methods for them largely adapt reward-maximizing objectives. We identify a central failure mode in this setting we call trajectory locking: sampled reward-driven updates over-concentrate probability mass onto a narrow set of denoising paths, reducing coverage of alternative correct solutions under repeated sampling. To address this, we propose TraFL (Trajectory Flow baLancing), a trajectory-balance objective that trains the policy toward a reward-tilted target distribution anchored to a frozen reference model. We make this practical for diffusion language models with a diffusion-compatible sequence-level surrogate and a learned prompt-dependent normalization. Across mathematical reasoning and code generation benchmarks, TraFL is the only evaluated post-training method that improves over the base model in every benchmark-length setting, with gains that persist as the sampling budget increases. The improvements transfer to held-out evaluations: TraFL stays above the base model on Minerva Math and is the strongest method on every LiveCodeBench difficulty split.

Momentum

Related topics

Latest papers 132