RL for Language Models

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

36 papers in the last four weeks, up 71% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 271

All topics

KARMA: Karma-Aligned Reward Model Adaptation
May 26, 2026Jared Scott, Jesse RobertsReward Modeling RL for Language Models
Tennessee Tech University Cookeville, Tennessee
Focal Reward: Balanced Reinforcement Learning under Rubric-Based Rewards
May 26, 2026Yu Huang, Zihua Zhao, Zhaoxin Huan +9Multi-Objective Reinforcement Learning RL for Language Models
Shanghai Jiao Tong University · Ant Group
From Static Context to Calibrated Interactive RL: Mitigating Distribution Shift in Multi-turn Dialogue with Aligned Simulator
May 26, 2026Xiaohua Wang, Jiakang Yuan, Zisu Huang +5RL for Language Models Reinforcement Learning
Fudan University
SafeCtrl-RL: Inference-Time Adaptive Behaviour Control for LLM Dialogue via RL-Driven Prompt Optimisation
May 25, 2026Michael Orme, Yanchao Yu, Zhiyuan TanRL for Language Models Language Model-Based Control
School of Computing, Engineering and Building Environment. Edinburgh Napier University Edinburgh, EH10 5DT, UK
TIAR: Trajectory-Informed Advantage Reweighting for LLM Abstention Learning
May 25, 2026Muyu Pan, Shu Zhao, Nan Zhang +4RL for Language Models Hallucination Mitigation
Department of Computer Science, The Pennsylvania State University State College, PA, USA
CRPO: Character-centric Group Relative Policy Optimization for Role-aware Reasoning in Role-playing Agents
May 25, 2026Yihong Tang, Kehai Chen, Liang Yue +2RL for Language Models Persona Consistency
Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China · Shenzhen Loop Area Institute (SLAI), Shenzhen, China · The Chinese University of Hong Kong, Shenzhen, China
Learning to Route Languages for Multilingual Policy Optimization
May 25, 2026Geyang Guo, Hiromi Wakaki, Yuki Mitsufuji +2Multilingual Language Models RL for Language Models
Georgia Institute of Technology · Sony Group Corporation · Sony AI
Directional Alignment Mitigates Reward Hacking in Reinforcement Learning for Language Models
May 24, 2026Wenlong Deng, Jiaji Huang, Kaan Ozkara +4Reward Hacking RL for Language Models
University of British Columbia · Vector Institute · Amazon
ARES: Automated Rubric Synthesis for Scalable LLM Reinforcement Learning
May 22, 2026Xiaoyuan Li, Keqin Bao, Moxin Li +5RL for Language Models Reinforcement Learning
University of Science and Technology of China · Alibaba Group · National University of Singapore
F-TIS: Harnessing Diverse Models in Collaborative GRPO
May 21, 2026Nikolay Blagoev, Oğuzhan Ersoy, Wendelin Boehmer +1RL for Language Models Communication-Efficient Distributed Training
Gensyn · University of Neuchatel · TU Delft
Value-Gradient Hypothesis of RL for LLMs
May 20, 2026Arip Asadulaev, Daniil Ognev, Karim Salta +1RL for Language Models Policy Gradient
MBZUAI · Independent
Towards Context-Invariant Safety Alignment for Large Language Models
May 20, 2026Yixu Wang, Yang Yao, Xin Wang +4RL for Language Models LLM Alignment
Fudan University, Shanghai, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China.
Reinforcement Learning over Predictive Distributions for LLM Regression
May 20, 2026Jungsoo Park, Hyungjoo Chae, Ethan Mendes +4Distributional RL RL for Language Models
Georgia Institute of Technology · Allen Institute for AI
Reinforcing Human Behavior Simulation via Verbal Feedback
May 19, 2026Weiwei Sun, Xuhui Zhou, Jiarui Liu +13RL for Language Models Human Behavior Simulation
1Carnegie Mellon University · 2Microsoft
AMARIS: A Memory-Augmented Rubric Improvement System for Rubric-Based Reinforcement Learning
May 18, 2026Peilin Wu, Xinlu Zhang, Kun Wan +4RL for Language Models Reinforcement Learning
The University of Texas at Dallas · Department of Computer Science, University of California, Santa Barbara · Adobe Inc.
Pairwise Preference Reward and Group-Based Diversity Enhancement for Superior Open-Ended Generation
May 18, 2026Guining Cao, Jiaxin Peng, Chu Zeng +3Pairwise Preference Learning Open-Ended Generation
Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd · School of Software and Microelectronics, Peking University · Tsinghua University
Weak-to-Strong Elicitation via Mismatched Wrong Drafts
May 17, 2026Wei DengRL for Language Models RL for Language Model Reasoning
Independent Researcher
From Feedback Loops to Policy Updates: Reinforcement Fine-Tuning for LLM-Based Alpha Factor Discovery
May 14, 2026Lingzhe Zhang, Tong Jia, Yunpeng Zhai +5RL for Language Models Large Language Model-Guided Optimization
Peking University, Beijing, China · Alibaba Group, China · Nanjing University, Nanjing, China +1
Prompting Policies for Multi-step Reasoning and Tool-Use in Black-box LLMs with Iterative Distillation of Experience
May 14, 2026Krishna Sayana, Ketan Todi, Ambarish JashRL for Language Models LLM Prompting
Google Research, Mountain View, CA
Multi-Objective and Mixed-Reward Reinforcement Learning via Reward-Decorrelated Policy Optimization
May 13, 2026Yang Bai, Kaiyuan Liu, Ziyuan Zhuang +5Multi-Objective Reinforcement Learning RL for Language Models
Meituan, China
ICRL: Learning to Internalize Self-Critique with Reinforcement Learning
May 13, 2026Jianbo Lin, Xiaomin Yu, Yi Xin +7RL for Language Models LLM Self-Refinement
Hong Kong University of Science and Technology (Guangzhou) · Nanjing University · Sun Yat-sen University +3
AIS: Adaptive Importance Sampling for Quantized RL
May 13, 2026Jiajun Zhou, Wei Shao, Lingchao Zheng +2RL for Language Models Reinforcement Learning
Huawei · The University of Hong Kong
ODRPO: Ordinal Decompositions of Discrete Rewards for Robust Policy Optimization
May 12, 2026Nirmal Patel, Fei Wang, Inderjit S. DhillonRL for Language Models Reinforcement Learning
University of Texas at Austin · Google
Training LLMs with Reinforcement Learning for Intent-Aware Personalized Question Answering
May 12, 2026Maryam Amirizaniani, Benjamin Charles Germain Lee, Jevin West +1RL for Language Models LLM Personalization
University of Washington, Seattle, WA, USA
CuSearch: Curriculum Rollout Sampling via Search Depth for Agentic RAG
May 12, 2026Jianghan Shen, Siqi Luo, Xinyu Cheng +6RL for Language Models Reinforcement Learning
1Nanjing University · 2Shanghai Artificial Intelligence Laboratory · University of Hong Kong +1
Epistemic Uncertainty for Test-Time Discovery
May 11, 2026Kainat Riaz, Muhammad Ahmed Mohsin, Ahsan Bilal +5RL for Language Models RL Exploration
Stanford University · University of Oklahoma · National University of Sciences and Technology
Compute Where it Counts: Self Optimizing Language Models
May 11, 2026Yash Akhauri, Mohamed S. AbdelfattahLLM Inference Efficiency RL for Language Models
Cornell University
Exploration-Driven Optimization for Test-Time Large Language Model Reasoning
May 11, 2026Changhao Li, Yuchen Zhuang, Chenxiao Gao +4RL for Language Models Inference-Time Optimization
Georgia Institute of Technology
Not All Turns Matter: Credit Assignment for Multi-Turn Jailbreaking
May 9, 2026Zhida He, Xiaoyu Wen, Han Qi +7RL for Language Models Credit Assignment in RL
Shanghai AI Laboratory · Fudan University · Shanghai Jiao Tong University
ReLibra: Routing-Replay-Guided Load Balancing for MoE Training in Reinforcement Learning
May 9, 2026Chao Jin, Xinming Wei, Yinmin Zhong +6RL for Language Models Expert Load Balancing
School of Computer Science, Peking University · Independent Researcher

May 26, 2026, cs.CLJ/K move · Enter open · S save

KARMA: Karma-Aligned Reward Model Adaptation

Jared Scott, Jesse Roberts

Tennessee Tech University Cookeville, Tennessee

Reward Modeling RL for Language Models LLM Alignment Pragmatic Reasoning in Language Models Preference Alignment

Open paper AskPDF arXiv

Human communication depends on implicit social signals where effectiveness is shaped by tone, context, and conversational norms rather than semantic content alone. We introduce KARMA (Karma-Aligned Reward Model Adaptation), a framework for LLM learning of context-sensitive conversational behavior from large-scale social interaction data. KARMA trains a reward model on Reddit conversations to predict response valuation conditioned on context, and uses this signal to fine-tune language models via reinforcement learning to improve performance on pragmatics-mediated tasks. Critically, we find that the highest performing reward model does not lead to better downstream model alignment: a reward model relying exclusively on conversational context was a worse predictor of Reddit karma but yielded substantially better downstream performance. We evaluate the effects of KARMA applied to a downstream model with and without direct exposure to the social media data. The resulting models show improved pragmatics-mediated behaviors with largely mitigated undesirable side effects. Factuality is consistently diminished by KARMA across all conditions, including when the downstream model has no direct exposure to Reddit data, suggesting that this tension is embedded in the reward signal itself rather than introduced by noisy training data.

Momentum

Related topics

Latest papers 271