RL for Language Models

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

36 papers in the last four weeks, up 71% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 271

All topics

Cross-Entropy Games and Frost Training
May 26, 2026Arthur Renard, Franck Gabriel, Valentin Hartmann +1RL for Language Models Policy Gradient
Xent Labs · Xent Labs, Université Lyon 1
FSA-GRPO: Teaching Auditory LLMs to Use Few-shot Demonstrations
May 26, 2026Haolong Zheng, Siyin Wang, Xulin Fan +2RL for Language Models Few-Shot Domain Adaptation
University of Illinois Urbana Champaign · Tsinghua University
KARMA: Karma-Aligned Reward Model Adaptation
May 26, 2026Jared Scott, Jesse RobertsReward Modeling RL for Language Models
Tennessee Tech University Cookeville, Tennessee
Focal Reward: Balanced Reinforcement Learning under Rubric-Based Rewards
May 26, 2026Yu Huang, Zihua Zhao, Zhaoxin Huan +9Multi-Objective Reinforcement Learning RL for Language Models
Shanghai Jiao Tong University · Ant Group
From Static Context to Calibrated Interactive RL: Mitigating Distribution Shift in Multi-turn Dialogue with Aligned Simulator
May 26, 2026Xiaohua Wang, Jiakang Yuan, Zisu Huang +5RL for Language Models Reinforcement Learning
Fudan University
SafeCtrl-RL: Inference-Time Adaptive Behaviour Control for LLM Dialogue via RL-Driven Prompt Optimisation
May 25, 2026Michael Orme, Yanchao Yu, Zhiyuan TanRL for Language Models Language Model-Based Control
School of Computing, Engineering and Building Environment. Edinburgh Napier University Edinburgh, EH10 5DT, UK
TIAR: Trajectory-Informed Advantage Reweighting for LLM Abstention Learning
May 25, 2026Muyu Pan, Shu Zhao, Nan Zhang +4RL for Language Models Hallucination Mitigation
Department of Computer Science, The Pennsylvania State University State College, PA, USA
CRPO: Character-centric Group Relative Policy Optimization for Role-aware Reasoning in Role-playing Agents
May 25, 2026Yihong Tang, Kehai Chen, Liang Yue +2RL for Language Models Persona Consistency
Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China · Shenzhen Loop Area Institute (SLAI), Shenzhen, China · The Chinese University of Hong Kong, Shenzhen, China
Learning to Route Languages for Multilingual Policy Optimization
May 25, 2026Geyang Guo, Hiromi Wakaki, Yuki Mitsufuji +2Multilingual Language Models RL for Language Models
Georgia Institute of Technology · Sony Group Corporation · Sony AI
Directional Alignment Mitigates Reward Hacking in Reinforcement Learning for Language Models
May 24, 2026Wenlong Deng, Jiaji Huang, Kaan Ozkara +4Reward Hacking RL for Language Models
University of British Columbia · Vector Institute · Amazon
ARES: Automated Rubric Synthesis for Scalable LLM Reinforcement Learning
May 22, 2026Xiaoyuan Li, Keqin Bao, Moxin Li +5RL for Language Models Reinforcement Learning
University of Science and Technology of China · Alibaba Group · National University of Singapore
F-TIS: Harnessing Diverse Models in Collaborative GRPO
May 21, 2026Nikolay Blagoev, Oğuzhan Ersoy, Wendelin Boehmer +1RL for Language Models Communication-Efficient Distributed Training
Gensyn · University of Neuchatel · TU Delft
Value-Gradient Hypothesis of RL for LLMs
May 20, 2026Arip Asadulaev, Daniil Ognev, Karim Salta +1RL for Language Models Policy Gradient
MBZUAI · Independent
Towards Context-Invariant Safety Alignment for Large Language Models
May 20, 2026Yixu Wang, Yang Yao, Xin Wang +4RL for Language Models LLM Alignment
Fudan University, Shanghai, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China.
Reinforcement Learning over Predictive Distributions for LLM Regression
May 20, 2026Jungsoo Park, Hyungjoo Chae, Ethan Mendes +4Distributional RL RL for Language Models
Georgia Institute of Technology · Allen Institute for AI
Reinforcing Human Behavior Simulation via Verbal Feedback
May 19, 2026Weiwei Sun, Xuhui Zhou, Jiarui Liu +13RL for Language Models Human Behavior Simulation
1Carnegie Mellon University · 2Microsoft
AMARIS: A Memory-Augmented Rubric Improvement System for Rubric-Based Reinforcement Learning
May 18, 2026Peilin Wu, Xinlu Zhang, Kun Wan +4RL for Language Models Reinforcement Learning
The University of Texas at Dallas · Department of Computer Science, University of California, Santa Barbara · Adobe Inc.
Pairwise Preference Reward and Group-Based Diversity Enhancement for Superior Open-Ended Generation
May 18, 2026Guining Cao, Jiaxin Peng, Chu Zeng +3Pairwise Preference Learning Open-Ended Generation
Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd · School of Software and Microelectronics, Peking University · Tsinghua University
Weak-to-Strong Elicitation via Mismatched Wrong Drafts
May 17, 2026Wei DengRL for Language Models RL for Language Model Reasoning
Independent Researcher
From Feedback Loops to Policy Updates: Reinforcement Fine-Tuning for LLM-Based Alpha Factor Discovery
May 14, 2026Lingzhe Zhang, Tong Jia, Yunpeng Zhai +5RL for Language Models Large Language Model-Guided Optimization
Peking University, Beijing, China · Alibaba Group, China · Nanjing University, Nanjing, China +1
Prompting Policies for Multi-step Reasoning and Tool-Use in Black-box LLMs with Iterative Distillation of Experience
May 14, 2026Krishna Sayana, Ketan Todi, Ambarish JashRL for Language Models LLM Prompting
Google Research, Mountain View, CA
Multi-Objective and Mixed-Reward Reinforcement Learning via Reward-Decorrelated Policy Optimization
May 13, 2026Yang Bai, Kaiyuan Liu, Ziyuan Zhuang +5Multi-Objective Reinforcement Learning RL for Language Models
Meituan, China
ICRL: Learning to Internalize Self-Critique with Reinforcement Learning
May 13, 2026Jianbo Lin, Xiaomin Yu, Yi Xin +7RL for Language Models LLM Self-Refinement
Hong Kong University of Science and Technology (Guangzhou) · Nanjing University · Sun Yat-sen University +3
AIS: Adaptive Importance Sampling for Quantized RL
May 13, 2026Jiajun Zhou, Wei Shao, Lingchao Zheng +2RL for Language Models Reinforcement Learning
Huawei · The University of Hong Kong
ODRPO: Ordinal Decompositions of Discrete Rewards for Robust Policy Optimization
May 12, 2026Nirmal Patel, Fei Wang, Inderjit S. DhillonRL for Language Models Reinforcement Learning
University of Texas at Austin · Google
Training LLMs with Reinforcement Learning for Intent-Aware Personalized Question Answering
May 12, 2026Maryam Amirizaniani, Benjamin Charles Germain Lee, Jevin West +1RL for Language Models LLM Personalization
University of Washington, Seattle, WA, USA
CuSearch: Curriculum Rollout Sampling via Search Depth for Agentic RAG
May 12, 2026Jianghan Shen, Siqi Luo, Xinyu Cheng +6RL for Language Models Reinforcement Learning
1Nanjing University · 2Shanghai Artificial Intelligence Laboratory · University of Hong Kong +1
Epistemic Uncertainty for Test-Time Discovery
May 11, 2026Kainat Riaz, Muhammad Ahmed Mohsin, Ahsan Bilal +5RL for Language Models RL Exploration
Stanford University · University of Oklahoma · National University of Sciences and Technology
Compute Where it Counts: Self Optimizing Language Models
May 11, 2026Yash Akhauri, Mohamed S. AbdelfattahLLM Inference Efficiency RL for Language Models
Cornell University
Exploration-Driven Optimization for Test-Time Large Language Model Reasoning
May 11, 2026Changhao Li, Yuchen Zhuang, Chenxiao Gao +4RL for Language Models Inference-Time Optimization
Georgia Institute of Technology

May 26, 2026, cs.AIJ/K move · Enter open · S save

Cross-Entropy Games and Frost Training

Arthur Renard, Franck Gabriel, Valentin Hartmann, Clément Hongler

Xent Labs · Xent Labs, Université Lyon 1

RL for Language Models Policy Gradient LLM-as-a-Judge

Open paper AskPDF arXiv

We present Frost Training, a method for improving Monte Carlo-based policy optimization for a large family of LLM-as-a-judge tasks called Cross-Entropy Games. The key idea is to exploit the gradient of the reward function in embedding space. This signal is used in the Greedy Coordinate Gradient (GCG) jailbreaking technique; we demonstrate for the first time that it can also be used to boost model training. We validate our method using GRPO training for maximum-likelihood infilling. Frost Training improves the model's ability to generate high-scoring outputs, reaching higher maximum scores in a best-of-k setting, and does so at an increased speed.

Momentum

Related topics

Latest papers 271