RL Post-Training

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

49 papers in the last four weeks, up 206% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 309

All topics

Compute Aligned Training: Optimizing for Test Time Inference
Apr 27, 2026Adam Ousherovitch, Ambuj TewariTest-Time Optimization Inference-Time Scaling
Department of Statistics University of Michigan
EPM-RL: Reinforcement Learning for On-Premise Product Mapping in E-Commerce
Apr 27, 2026Minhyeong Yu, Wonduk SeoE-Commerce RL Post-Training
AI Research, Enhans Seoul, South Korea
JigsawRL: Assembling RL Pipelines for Efficient LLM Post-Training
Apr 26, 2026Zhengding Hu, Hehua Ouyang, Chang Chen +6RL Post-Training Pipeline Parallelism
University of California, San Diego
Reward Models Are Secretly Value Functions: Temporally Coherent Reward Modeling
Apr 24, 2026Alex NikulkovReward Modeling Value Function Estimation
AI at Meta
Outcome Rewards Do Not Guarantee Verifiable or Causally Important Reasoning
Apr 23, 2026Qinan Yu, Alexa Tartaglini, Peter Hase +2CoT Faithfulness Reinforcement Learning with Verifiable Rewards
1Stanford University · 2Schmidt Sciences
When Policies Cannot Be Retrained: A Unified Closed-Form View of Post-Training Steering in Offline Reinforcement Learning
Apr 23, 2026Elias Hossain, Mohammad Jahid Ibna Basher, Ivan Garibay +2Offline RL RL Post-Training
College of Engineering and Computer Science University of Central Florida, Orlando, FL, USA
ParetoSlider: Diffusion Models Post-Training for Continuous Reward Control
Apr 22, 2026Shelly Golan, Michael Finkelson, Ariel Bereslavsky +2Multi-Objective Reinforcement Learning RL for Image Generation
Tel Aviv University · Lightricks · Adobe Research
EVPO: Explained Variance Policy Optimization for Adaptive Critic Utilization in LLM Post-Training
Apr 21, 2026Chengjun Pan, Shichun Liu, Jiahang Lin +10RL for Language Models Policy Gradient Methods
Peking University · Fudan University · Shanghai AI Lab +1
Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data
Apr 20, 2026Zhenwen Liang, Yujun Zhou, Sidi Lu +3Reinforcement Learning RL for Language Model Reasoning
Tencent AI Lab · University of Notre Dame
Poly-EPO: Training Exploratory Reasoning Models
Apr 19, 2026Ifdita Hasan Orney, Jubayer Ibn Hamid, Shreya S Ramanujam +5Reinforcement Learning RL for Language Model Reasoning
Stanford University
Towards Shutdownable Agents: Generalizing Stochastic Choice in RL Agents and LLMs
Apr 19, 2026Carissa Cullen, Harry Garland, Alexander Roman +3Agent Reliability AI Agent Safety
University of Oxford · University College London · New College of Florida +3
A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions
Apr 19, 2026Zhiyin Yu, Yuchen Mou, Juncheng Yan +17RL for Language Models RL Post-Training
1Peking University · 2Shanghai Artificial Intelligence Laboratory · 3National University of Singapore +6
SPS: Steering Probability Squeezing for Better Exploration in Reinforcement Learning for Large Language Models
Apr 18, 2026Yifu Huo, Chenglong Wang, Ziming Zhu +9Reinforcement Learning RL for Language Model Reasoning
1Northeastern University, Shenyang, China · 2CAS Key Laboratory of Behavioral Science, Beijing, China · 3Independent Researcher, Beijing, China +1
Freshness-Aware Prioritized Experience Replay for LLM/VLM Reinforcement Learning
Apr 18, 2026Weiyu Ma, Yongcheng Zeng, Yan Song +4Reinforcement Learning RL Post-Training
King Abdullah University of Science and Technology (KAUST) · Chinese Academy of Sciences, Institute of Automation (CASIA) · AI Centre, Department of Computer Science, University College London +1
Beyond Distribution Sharpening: The Importance of Task Rewards
Apr 17, 2026Sarthak Mittal, Leo Gagnon, Guillaume LajoieReinforcement Learning RL for Language Model Reasoning
1Mila · 2Universit´e de Montr´eal
S-GRPO: Unified Post-Training for Large Vision-Language Models
Apr 17, 2026Yuming Yan, Kai Tang, Sihong Chen +4VLM Adaptation Group Relative Policy Optimization
Tencent Shenzhen, Guangdong, China
An Imperfect Verifier is Good Enough: Learning with Noisy Rewards
Apr 9, 2026Andreas Plesner, Francisco Guzmán, Anish AthalyeRL for Code Generation Reinforcement Learning
Handshake AI · ETH Zurich
Apriel-Reasoner: RL Post-Training for General-Purpose and Efficient Reasoning
Apr 2, 2026Rafael Pardinas, Ehsan Kamalloo, David Vazquez +1LLM Inference Efficiency RL for Language Model Reasoning
ServiceNow AI Research
Policy Improvement Reinforcement Learning
Apr 1, 2026Huaiyang Wang, Xiaojie Li, Xiaohan Wang +10RL for Language Models RL for Language Model Reasoning
Beihang University · Meituan · Peking University
Finite Difference Flow Optimization for RL Post-Training of Text-to-Image Models
Mar 13, 2026David McAllister, Miika Aittala, Tero Karras +4RL for Image Generation RL for Diffusion Models
UC Berkeley · NVIDIA
GIPO: Gaussian Importance Sampling Policy Optimization
Mar 4, 2026Chengxuan Lu, Zhenquan Zhang, Shukuan Wang +3Reinforcement Learning RL Post-Training
1IROOTECH TECHNOLOGY · 2Wolf 1069 b Lab, Sany Group
Personalized Group Relative Policy Optimization for Heterogenous Preference Alignment
Feb 17, 2026Jialu Wang, Heinrich Peters, Asad A. Butt +6LLM Alignment Group Relative Policy Optimization
1Apple Inc., California, United States.
The Optimal Token Baseline: Variance Reduction for Long-Horizon LLM-RL
Feb 6, 2026Yingru Li, Jiawei Xu, Ziniu Li +10RL for Language Model Reasoning Policy Gradient Methods
The Chinese University of Hong Kong, Shenzhen · Fudan University · Hong Kong University of Science and Technology +4
HeaPA: Difficulty-Aware Heap Sampling and On-Policy Query Augmentation for LLM Reinforcement Learning
Jan 30, 2026Weiqi Wang, Xin Liu, Binxuan Huang +13RL for Language Models RL for Language Model Reasoning
Stores Foundational AI, Amazon Inc., Palo Alto, CA, USA · Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China
Thinking Seeds: Leveraging Historical Diversity for Position-Aware RL in LLMs
Jan 29, 2026Lei Yang, Wei Bi, Chenxi Sun +2RL for Language Model Reasoning RL Exploration
TJUNLP Lab, College of Intelligence and Computing, Tianjin University, Tianjin, China · Kuaishou Technology
R $^2$ PO: Decoupling Rollout and Inference Policies for LLM Reasoning
Jan 17, 2026Jingchu Wang, Bingbing Xu, Yige Yuan +4RL for Language Model Reasoning Policy Optimization
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences · National University of Singapore
Stabilizing Off-Policy Training for Long-Horizon LLM Agent via Turn-Level Importance Sampling and Clipping-Triggered Normalization
Nov 25, 2025Chenliang Li, Adel Elmahdy, Alex Boyd +7RL for Language Model Reasoning Importance Sampling
Texas A&M University · GE HealthCare · Independent Researcher +1
Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting
Oct 21, 2025Howard Chen, Noam Razin, Karthik Narasimhan +1Continual Learning for LLMs RL Post-Training
Princeton Language and Intelligence, Princeton University.
Representation-Based Exploration for Language Models: From Test-Time to Post-Training
Oct 13, 2025Jens Tuyls, Dylan J. Foster, Akshay Krishnamurthy +1RL for Language Model Reasoning Test-Time Scaling
Princeton University · Microsoft Research NYC
BaNEL: Exploration Posteriors for Generative Modeling Using Only Negative Rewards
Oct 10, 2025Sangyun Lee, Brandon Amos, Giulia FantiGenerative Modeling RL for Generative Models
Carnegie Mellon University

Apr 27, 2026, cs.LGJ/K move · Enter open · S save

Compute Aligned Training: Optimizing for Test Time Inference

Adam Ousherovitch, Ambuj Tewari

Department of Statistics University of Michigan

Test-Time Optimization Inference-Time Scaling RL Post-Training LLM Post-Training

Open paper AskPDF arXiv

Scaling test-time compute has emerged as a powerful mechanism for enhancing Large Language Model (LLM) performance. However, standard post-training paradigms, Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), optimize the likelihood of individual samples under a base policy, creating a misalignment with test time procedures that rely on aggregated or filtered outputs. In this work, we propose Compute Aligned Training, which aligns training objectives with test-time strategies. By conceptualizing inference strategies as operators on the base policy, we derive new loss functions that maximize performance when said strategies are applied. We instantiate such loss functions for SFT and RL across common test time strategies. Finally, we provide empirical evidence that this training method substantially improves test time scaling over standard training.

Momentum

Related topics

Latest papers 309