RL for Language Models

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

36 papers in the last four weeks, up 71% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 271

All topics

A Good Self-Teacher Meets the Student Where They Are: Joint On-Policy Learning and Teaching
Oct 7, 2026Randy Ardywibowo, Arnav Dalal, Jiantao JiaoOn-Policy Self-Distillation RL for Language Models
Perplexity · NVIDIA
Marrying Pricing and Advertising with LLMs
Oct 7, 2026Alessandro Barro, Francesco Bacchiocchi, Francesco Emanuele Stradi +1Dynamic Pricing RL for Language Models
Politecnico di Milano
RollVerify: Bridging Efficiency and Accuracy in Long-Tail Rollout Reinforcement Learning
Oct 7, 2026Yongqiang Yao, Jinru Tan, Kaihuan Liang +5RL for Language Models Reinforcement Learning
Shanghai Jiao Tong University · Central South University · SenseTime Research +2
Sherpa: Teaching LLMs to Teach Adaptively
Oct 6, 2026Weixian Xu, Yanzhe Zhang, Zora Zhiruo Wang +2RL for Language Models Intelligent Tutoring Systems
Stanford University · Georgia Tech · Carnegie Mellon University
Revisiting Numerical Forecasting Models for Language-Based Trajectory Prediction
Oct 6, 2026JunGyu Lee, Inhwan Bae, Hae-Gon JeonReward Modeling RL for Language Models
Yonsei University · DGIST
HiPLEX: Hierarchical Policy Factorization for Full Duplex Speech Language Models
Oct 6, 2026Kyudan Jung, Hyunsin Park, Yoonhyung Lee +5RL for Language Models Speech Language Models
Qualcomm AI Research · KAIST AI
The Assistance Dilemma: Learning to Teach via Multi-Turn Reinforcement Learning
Oct 5, 2026Jakub Macina, Manu Kapur, Mrinmaya SachanRL for Language Models Intelligent Tutoring Systems
ETH Zurich
Do Small Language Models Learn to Negotiate? A Controlled Scaling Study of RL-Trained Sellers
Oct 5, 2026Pedro Tabacof, Sagar JoglekarRL for Language Models Multi-Agent Negotiation
Fin AI Research
HuatuoGPT-3: RL-Only Domain Adaptation from Base Models
Oct 5, 2026Junying Chen, Xinyuan Xie, Ziniu Li +7RL for Language Models Reinforcement Learning
The Chinese University of Hong Kong, Shenzhen · Shenzhen Research Institute of Big Data · Shenzhen Loop Area Institute +1
CARM: Cancellation-Aware Response Masking for LLM Reinforcement Learning
Oct 1, 2026Yafei Zhang, Songshuo Lu, Sicong Liao +2RL for Code Generation RL for Language Models
Moore Threads AI
The Asymptotics of Language Model Alignment with Memory
Oct 1, 2026Haricharan Balasundaram, V. Arvind RameshwarRL for Language Models LLM Alignment
School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA · IIT Madras · Department of Electrical Engineering, IIT Madras, Chennai, India
LineupRL: Verifiable Reinforcement Learning for Time Series Captioning via Caption-to-Series Identification
Oct 1, 2026Haochen Zhang, Laura Yao, Zachary Plotkin +2RL for Language Models Reinforcement Learning with Verifiable Rewards
University of North Carolina at Chapel Hill · Foci Labs
Dependency-Aware Reward Shaping for Agentic Reinforcement Learning
Oct 1, 2026Ziyi Chen, Yan Zhang, Jianhui Wei +2RL for Language Models Agentic RL
University of Illinois Urbana-Champaign · National University of Singapore · Zhejiang University +1
Grounding Large Language Models in DSGE Simulators for Policy Generation and Forecasting
Oct 1, 2026Aditya Dubey, Namah Gupta, Vinti AgarwalRL for Language Models Large Language Model-Guided Optimization
Birla Institute of Technology and Science, Pilani, India
Probe with Participation Trophies: Random-Reward RL as a Probe of LLM Capability
Oct 1, 2026Yu Mao, Lei Yu, Zining Zhu +6RL for Language Models Reinforcement Learning
University of Toronto · Stevens Institute of Technology · University of California, Santa Cruz +3
GrammarRL: Effective Grammar-Constrained Decoding via Reinforcement Learning
Sep 30, 2026Gabriele Tuccio, Antonino Furnari, Aldo Gangemi +1RL for Language Models Constrained Decoding
University of Catania · ISTC - National Research Council, Italy · University of Bologna
Multi-LLM Collaborative Alignment via Stackelberg Games
Sep 30, 2026Christina Hahn, Shangbin Feng, Dean Light +3RL for Language Models LLM Alignment
University of Washington · Amazon · University of British Columbia
OpenJev-RLCD: A Working RLCD Implementation
Sep 30, 2026Zhimin Gao, Pichao WangRL for Language Models RL for Language Model Reasoning
Cool the Sampler, Not the Learner: Sampling Temperature Moves the Staleness Cliff of Importance-Corrected GRPO
Sep 29, 2026Taiheng PanRL for Language Models Reinforcement Learning
The University of Melbourne
CorrGRPO: Correlation-Normalized GRPO for Multi-Reward Learning
Sep 29, 2026Wenbin Hu, Huihao Jing, Haochen Shi +3Multi-Objective Reinforcement Learning RL for Language Models
Hong Kong University of Science and Technology
VAA-CSEC: Vote-guided Advantage Allocation for Chinese Semantic Error Correction
Sep 29, 2026Yitong Han, Nankai Lin, Juan Luo +3RL for Language Models Self-Consistency in Language Models
School of Information Science and Technology, Guangdong University of Foreign Studies · Guangdong Engineering Research Center of Data Security Governance and Privacy Computing · College of Computer Science and Technology, National University of Defense Technology
Group-Marginalized Self-Rewarding RL Drives Zero-Label Self-Evolving
Sep 29, 2026Yiming Wang, Yikang Liu, Qingyuan Tian +4RL for Language Models Reinforcement Learning
School of Computer Science, Shanghai Jiao Tong University · Tencent
Sample What You Say: Aligning Language Models to Sample the Distributions They State
Sep 28, 2026Kasra Arabi, Virginia Smith, Chhavi YadavRL for Language Models Group Relative Policy Optimization
New York University · Carnegie Mellon University
Quality Determines Direction, Length Shapes Magnitude: Length Control for Open-Ended Reinforcement Learning
Sep 28, 2026Zijun Weng, Zhongan Bi, Xuanang Gao +5Open-Ended Generation RL for Language Models
Fudan University · Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd. · Zhejiang University +1
ABC-Align: Prediction-Powered Alignment with Adaptive Bias Control
Sep 28, 2026Eric Frankel, Banghua Zhu, Sewoong Oh +1RL for Language Models LLM Alignment
University of Washington, Seattle
CRISP: Cultural Reward Modeling for Implicit Situated Propriety
Sep 28, 2026Zekun Yuan, Yangfan Ye, Baohang Li +6Reward Modeling RL for Language Models
Harbin Institute of Technology · Huawei Technologies Co., Ltd · Peng Cheng Laboratory
Evolving Support Priorities in Empathetic Reinforcement Learning
Sep 28, 2026Pengyu Huang, Zhiyuan Han, Wenwen Tong +6Reward Modeling RL for Language Models
University of Science and Technology of China, Hefei, China · SenseTime Research, Shanghai, China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
GlyphBench: A Playground for Language-Model Reinforcement Learning
Sep 28, 2026Roger Creus Castanyer, Marc-Alexandre Côté, Matthew James Sargent +3RL Benchmarks RL for Language Models
Mila - Quebec Artificial Intelligence Institute · Université de Montréal · Vmax
TGRL: Temperature-Grouped Reinforcement Learning for Efficient Exploration in LLMs
Sep 27, 2026Zihan Lin, Xiaohan Wang, Jie Cao +4RL for Language Models RL for Language Model Reasoning
University of Chinese Academy of Sciences · Meituan · MAIS&NLPR, Institute of Automation, Chinese Academy of Sciences
COEVO: Co-Evolving Context and Parameters for Recursive Self-Improvement
Sep 27, 2026Siwei Chen, Xinping Bao, Xinyu Cai +3RL for Language Models Prompt Optimization
Peking University · Emotional Machine

Oct 7, 2026, cs.LGJ/K move · Enter open · S save

A Good Self-Teacher Meets the Student Where They Are: Joint On-Policy Learning and Teaching

Randy Ardywibowo, Arnav Dalal, Jiantao Jiao

Perplexity · NVIDIA

On-Policy Self-Distillation RL for Language Models On-Policy Distillation Knowledge Distillation

Open paper AskPDF arXiv

Reinforcement Learning (RL) from outcome rewards suffers from sparse supervision, particularly on difficult, long-horizon tasks where successful trajectories are rare and costly to generate. On-Policy Distillation (OPD) offers an attractive alternative by providing dense token-level supervision from a stronger teacher along the student's own generations. Self-distillation methods further remove the need for a separate teacher model by conditioning the same policy on privileged information to serve as its own teacher. However, privileged conditioning alone does not guarantee that the resulting distillation update improves the student. Indeed, privileged information can lead the teacher to solve tasks through shortcuts unavailable to the student, producing supervision poorly matched to the student's current behavior. Consequently, even a higher-performing teacher can provide guidance that degrades student performance. To address this, we analyze how the choice of privileged teacher affects the student's update. We derive a necessary and sufficient condition for the teacher's local distillation update to be a positive multiple of the student's reward gradient. Our analysis suggests that the teacher should not only perform well on the task, but also provide guidance suited to the student's current capabilities. This characterization motivates a practical teacher-training surrogate that combines outcome rewards with token-level Kullback-Leibler (KL) regularization toward the student. Based on this result, we propose Joint On-Policy Learning and Teaching (JOLT), which jointly trains a single policy in two roles: a privileged teacher using a KL-regularized objective, and an unprivileged student using dense on-policy distillation. Across mathematical reasoning, coding, tool use, and terminal use, JOLT improves training efficiency and performance, with further gains from student rewards.

Momentum

Related topics

Latest papers 271