RL for Language Models

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

36 papers in the last four weeks, up 71% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 271

All topics

SalesSim: Benchmarking and Aligning Multimodal Language Models as Retail User Simulators
May 8, 2026Yada Pruksachatkun, Elaine Wan, Lyanna Chen +2RL for Language Models Large Language Model-Based Role-Play Simulation
Salesforce Research · University of California Los Angeles
SEIF: Self-Evolving Reinforcement Learning for Instruction Following
May 8, 2026Qingyu Ren, Qianyu He, Jiajie Zhu +7RL for Language Models Reinforcement Learning
Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence, Fudan University · School of Data Science, Fudan University · Ant Group
HTPO: Towards Exploration-Exploitation Balanced Policy Optimization via Hierarchical Token-level Objective Control
May 8, 2026Xincheng Yao, Ruoqi Li, Cheng Chen +4RL for Language Models Exploration-Exploitation Tradeoff
School of Information Science and Electronic Engineering, Shanghai Jiao Tong University · Xiaohongshu Inc. · MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University.
Experience Sharing in Mutual Reinforcement Learning for Heterogeneous Language Models
May 8, 2026Xiaoze Liu, Dhananjay Ram, Yuting Zhang +3RL for Language Models Reinforcement Learning
1Purdue University · *Work done during an internship at Amazon. · 2AWS Agentic AI
Beyond Reasoning: Reinforcement Learning Unlocks Parametric Knowledge in LLMs
May 8, 2026Wanli Yang, Hongyu Zang, Junwei Zhang +5RL for Language Models Factual Knowledge in Language Models
University of Chinese Academy of Sciences · State Key Laboratory of AI Safety, Institute of Computing Technology, CAS
$f$ -Divergence Regularized RLHF: Two Tales of Sampling and Unified Analyses
May 7, 2026Di Wu, Chengshuai Shi, Jing Yang +1RL for Language Models RL from Human Feedback
Department of Electrical and Computer Engineering, University of Virginia, Virginia, United States · Princeton Language and Intelligence, Princeton University, New Jersey, United States
RVPO: Risk-Sensitive Alignment via Variance Regularization
May 7, 2026Ivan Montero, Tomasz Jurczyk, Bhuwan DhingraRL for Language Models Risk-Sensitive RL
Apple
Efficiently Aligning Language Models with Online Natural Language Feedback
May 5, 2026Christine Ye, Joe BentonReward Modeling RL for Language Models
Stanford University · Anthropic Fellows Program for Safety Research · Anthropic
EvoLM: Self-Evolving Language Models through Co-Evolved Discriminative Rubrics
May 5, 2026Shuyue Stella Li, Rui Xin, Teng Xiao +8RL for Language Models LLM Evaluation
University of Washington · 2Allen Institute for AI · University of Pennsylvania
ResRL: Boosting LLM Reasoning via Negative Sample Projection Residual Reinforcement Learning
May 1, 2026Zihan Lin, Xiaohan Wang, Jie Cao +6RL for Language Models Reinforcement Learning
MAIS&NLPR, Institute of Automation, Chinese Academy of Sciences, Beijing, China · Meituan, Beijing, China · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences, Beijing, China.
Cost-Aware Learning
Apr 30, 2026Clara Mohri, Amir Globerson, Haim Kaplan +2Stochastic Optimization RL for Language Models
Kempner Institute · Harvard University · Google Research +1
Rethinking Agentic Reinforcement Learning In Large Language Models
Apr 30, 2026Fangming Cui, Ruixiao Zhu, Cheng Fang +2RL for Language Models Agentic RL
Beijing, China · Shanghai, China
From Coarse to Fine: Benchmarking and Reward Modeling for Writing-Centric Generation Tasks
Apr 30, 2026Qingyu Ren, Tianjun Pan, Xingzhou Chen +1Reward Modeling RL for Language Models
Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence, Fudan University · 2Shanghai Artificial Intelligence Laboratory
TLPO: Token-Level Policy Optimization for Mitigating Language Confusion in Large Language Models
Apr 29, 2026Jinho Choo, JunSeung Lee, Jimyeong Kim +3Multilingual Language Models RL for Language Models
Samsung SDS
Addressing Performance Saturation for LLM RL via Precise Entropy Curve Control
Apr 29, 2026Bolian Li, Yifan Wang, Yi Ding +3RL for Language Models Reinforcement Learning
Purdue University, West Lafayette, IN, USA
DORA: A Scalable Asynchronous Reinforcement Learning System for Language Model Training
Apr 29, 2026Tianhao Hu, Xiangcheng Liu, Youshao Xiao +15RL for Language Models Asynchronous RL
Meituan, China
Frictive Policy Optimization for LLMs: Epistemic Intervention, Risk-Sensitive Control, and Reflective Alignment
Apr 28, 2026James Pustejovsky, Nikhil KrishnaswamyRL for Language Models Language Model Safety Evaluation
Brandeis University · Waltham, MA, USA · Colorado State University +1
BitRL: Reinforcement Learning with 1-bit Quantized Language Models for Resource-Constrained Edge Deployment
Apr 27, 2026Md. Ashiq Ul Islam Sajid, Mohammad Sakib Mahmood, Md. Tareq Hasan +3LLM Quantization RL for Language Models
Department of Computer Science BRAC University Dhaka 1212, Bangladesh · Department of Computer Science Missouri State University Springfield, MO, USA · Department of Computer Science & Engineering Prime University Dhaka, Bangladesh +2
C-MORAL: Controllable Multi-Objective Molecular Optimization with Reinforcement Alignment for LLMs
Apr 24, 2026Rui Gao, Youngseung Jeon, Swastik Roy +2RL for Language Models Molecular Optimization
University of California, Los Angeles · Amazon
Controllable Spoken Dialogue Generation: An LLM-Driven Grading System for K-12 Non-Native English Learners
Apr 24, 2026Haidong Yuan, Haokun Zhao, Wanshi Xu +4RL for Language Models LLM Alignment
Peking University · Fudan University · Tencent +2
Hierarchical Policy Optimization for Simultaneous Translation of Unbounded Speech
Apr 22, 2026Siqi Ouyang, Shuoyang Ding, Oleksii Hrinchuk +4RL for Language Models Speech Translation
1Carnegie Mellon University · 2NVIDIA
WebGen-R1: Incentivizing Large Language Models to Generate Functional and Aesthetic Websites with Reinforcement Learning
Apr 22, 2026Juyong Jiang, Chenglin Cai, Chansung Park +4RL for Code Generation RL for Language Models
The Hong Kong University of Science and Technology (Guangzhou) · Alibaba · Tongyi Lab, Alibaba Group +3
R2IF: Aligning Reasoning with Decisions via Composite Rewards for Interpretable LLM Function Calling
Apr 22, 2026Aijia Cheng, Kailong Wang, Ling Shi +1RL for Language Models LLM Interpretability
Shanghai Key Laboratory of Trustworthy Computing, East China Normal University, Shanghai, China · Huazhong University of Science and Technology · Nanyang Technological University
EVPO: Explained Variance Policy Optimization for Adaptive Critic Utilization in LLM Post-Training
Apr 21, 2026Chengjun Pan, Shichun Liu, Jiahang Lin +10RL for Language Models Policy Gradient Methods
Peking University · Fudan University · Shanghai AI Lab +1
CodePivot: Bootstrapping Multilingual Transpilation in LLMs via Reinforcement Learning without Parallel Corpora
Apr 20, 2026Shangyu Li, Juyong Jiang, Meibo Ren +7Code Translation RL for Language Models
The Hong Kong University of Science and Technology · The Hong Kong University of Science and Technology (Guangzhou) · Monash University +1
SELF-EMO: Emotional Self-Evolution from Recognition to Consistent Expression
Apr 20, 2026Shaowei Zhang, Faqiang Qian, Yan Chen +5RL for Language Models Emotion Recognition in Conversations
1SenseTime · 2Tsinghua University · 3Shanghai Jiao Tong University +1
Beyond Static Snapshots: A Grounded Evaluation Framework for Language Models at the Agentic Frontier
Apr 19, 2026Jazmia HenryReward Hacking RL for Language Models
University of Oxford
CoAct: Co-Active LLM Preference Learning with Human-AI Synergy
Apr 19, 2026Ruiyao Xu, Mihir Parmar, Tiankai Yang +3RL for Language Models LLM Alignment
Northwestern University · Google · University of Southern California +1
Rethinking the Comparison Unit in Sequence-Level Reinforcement Learning: An Equal-Length Paired Training Framework from Loss Correction to Sample Construction
Apr 19, 2026Fei Ding, Yongkang Zhang, Runhao Liu +5RL for Language Models Reinforcement Learning
Alibaba Group · Tsinghua University
A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions
Apr 19, 2026Zhiyin Yu, Yuchen Mou, Juncheng Yan +17RL for Language Models RL Post-Training
1Peking University · 2Shanghai Artificial Intelligence Laboratory · 3National University of Singapore +6

May 8, 2026, cs.CLJ/K move · Enter open · S save

SalesSim: Benchmarking and Aligning Multimodal Language Models as Retail User Simulators

Yada Pruksachatkun, Elaine Wan, Lyanna Chen, Kai-Wei Chang+1

Salesforce Research · University of California Los Angeles

RL for Language Models Large Language Model-Based Role-Play Simulation Personality Modeling in Language Models Multimodal Large Language Models Multi-Objective Language Model Alignment User Simulation

Open paper AskPDF arXiv

We present SalesSim, a framework and testbed for evaluating the ability of Multimodal Large Language Models (MLLMs) to simulate realistic, persona-driven customer behavior in multi-turn, multi-modal, tool-augmented online retail conversations. Unlike prior work that treat user simulation as surface-level dialogue generation, SalesSim models retail interaction and decision-making as a grounded, agentic process, where shoppers with diverse backgrounds, preferences, and dealbreakers interact with a sales agent, seek clarifications, and make informed purchasing decisions. For evaluation, we design a suite of metrics centered on decision alignment, measuring the consistency between the simulator's actions and its persona specifications, as well as conversational quality. We find several behavioral gaps after benchmarking 6 open and closed-source state-of-the-art models. First, while models produce fluent conversations, they display significantly lower lexical diversity and overdisclosure of criteria across personas compared to human conversations. Second, models tend to be persuaded by sales agent suggestions and drift from persona specifications. Even the strongest model achieves less than 79% average alignment with its underlying persona specifications. To make progress on these limitations, we propose UserGRPO, a multi-turn, multi-objective reinforcement learning recipe to optimize both conversational fluency and decision alignment under persona specifications. Our experiments demonstrate that UserGRPO boosts decision alignment of the baseline model by 13.8% while improving conversational quality. By introducing SalesSim, we provide a new testbed for the community to investigate and improve the adherence of user simulators in goal-oriented settings.

Momentum

Related topics

Latest papers 271