Preference-Based RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 77

All topics

MAPL: Multi-Objective Preference Learning for Robot Locomotion
Jun 24, 2026Xiyue Chen, Muhan Lin, Shuyang Shi +1Multi-Objective Reinforcement Learning Reward Modeling
Department of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47906, USA · Department of Computer Science, Purdue University, West Lafayette, IN 47906, USA · independent researcher
Self-Evolution for Multi-Turn Tool-Calling Agents via Divergence-Point Preference Learning
Jun 22, 2026Jiaqiang TangTool-Using Agents Self-Improving Agents
The Hong Kong University of Science and Technology (Guangzhou)
UBP2: Uncertainty-Balanced Preference Planning for Efficient Preference-based Reinforcement Learning
Jun 17, 2026Mohamed Nabail, Leo Kaixuan Cheng, Jingmin Wang +1Pairwise Preference Evaluation RL Exploration
Learning, Embodied Autonomy, and Forecasting (LEAF) Lab, University of Toronto
Video-Based Optimal Transport for Feedback-Efficient Offline Preference-Based Reinforcement Learning
Jun 15, 2026Tung M. Luu, Hwanhee Kim, Younghwan Lee +1Offline RL Semi-Supervised Learning
Korea Advanced Institute of Science and Technology (KAIST).
PAWS: Preference Learning with Advantage-Weighted Segments
Jun 10, 2026Aleksandar Taranovic, Onur Celik, Niklas Freymuth +6Credit Assignment in RL Robotic RL
Autonomous Learning Robots, Karlsruhe Institute of Technology, Karlsruhe, Germany · Bosch Center for Artificial Intelligence, Renningen, Germany · Institute for Material Handling and Logistics (IFL), Karlsruhe Institute of Technology, Karlsruhe, Germany
A Regret Minimization Framework on Preference Learning in Large Language Models
Jun 8, 2026Suhwan Kim, Taehyun Cho, Geon-Hyeong Kim +4Regret Minimization in RL LLM Alignment
Seoul National University, Seoul, South Korea · LG AI Research, Seoul, South Korea · UNIST, Ulsan, South Korea +1
FlowPRO: Reward-Free Reinforced Fine-Tuning of Flow-Matching VLAs via Proximalized Preference Optimization
Jun 3, 2026Yihao Wu, He Zhang, Junbo Tan +2Offline RL Vision-Language-Action Models
Tencent Robotics X · Futian Laboratory · Tsinghua University
Preference-Calibrated Human-in-the-Loop Reinforcement Learning for Robotic Manipulation
Jun 2, 2026Zeyi Liu, Guangyao Liu, Yinuo Qu +6Credit Assignment in RL Robotic RL
Central South University · Nanyang Technological University · Zhejiang University
Pave-GRPO: Beyond Instantaneous Guidance through Principled Average Velocity Decomposition
Jun 1, 2026Pengyang Ling, Jiazi Bu, Yujie Zhou +6Diffusion Model Alignment Group Relative Policy Optimization
University of Science and Technology of China, China · Shanghai Jiao Tong University, China · Fudan University, China +3
S-SPPO: Semantic-Calibrated Self-Play Preference Optimization
Jun 1, 2026Xiwen Chen, Wenhui Zhu, Jingjing Wang +13LLM Alignment Self-Play RL
Morgan Stanley, USA · Arizona State University, USA · Clemson University, USA +5
Efficient Exploration for Iterative Nash Preference Optimization
May 31, 2026Tianlong Nan, Xiaopeng Li, Christian Kroer +1Nash Equilibrium LLM Alignment
Columbia University⋄ · The Chinese University of Hong Kong, Shenzhen†
From Reward-Free Representations to Preferences: Rethinking Offline Preference-Based Reinforcement Learning
May 31, 2026Jun-Jie Yang, Chia-Heng Hsu, Kui-Yuan Chen +1Pairwise Preference Learning Representation Learning
Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
Reinforcement Learning with Pairwise Preferences in Long-Term Decision Problems
May 29, 2026Jonathan Colaço Carr, Prakash Panangaden, Doina Precup +1Pairwise Preference Learning Markov Decision Processes
Stanford University · School of Computer Science, McGill University, Montreal, Quebec, Canada · Mila - Quebec AI Institute, Montreal, Quebec, Canada +1
Learning Kernel-Based MDPs from Episodic Preferential Feedback
May 22, 2026Nikola Pavlovic, Sattar Vakili, Qing ZhaoPairwise Preference Learning Regret Minimization in RL
Cornell University · MediaTek Research
From Correctness to Preference: A Framework for Personalized Agentic Reinforcement Learning
May 22, 2026Ranxu zhang, zeyang li, Jiacheng Huang +5Agentic RL Reinforcement Learning
University of Science and Technology of China · Alibaba Group
Token-weighted Direct Preference Optimization with Attention
May 21, 2026Chengyu Huang, Zhuohang Li, Sheng-Yen Chou +1Pairwise Preference Learning LLM Alignment
Cornell University · Vanderbilt University
Implicit Safety Alignment from Crowd Preferences
May 20, 2026Qian Lin, Daniel S. BrownSafe RL Preference-Based RL
Kahlert School of Computing, University of Utah
How Much Online RL is Enough? Informative Rollouts for Offline Preference Optimization in RLVR
May 20, 2026Richa Verma, Balaraman RavindranDirect Preference Optimization RL for Language Model Reasoning
TCS Research Department of CSE IIT Madras Chennai, India · Department of Data Science & AI, Wadhwani School of Data Science & AI IIT Madras Chennai, India
PREFINE: Preference-Based Implicit Reward and Cost Fine-Tuning for Safety Alignment
May 20, 2026Richa Verma, Bavish Kulur, Sanjay Chawla +1Direct Preference Optimization Constrained RL
TCS Research, Department of CSE, IIT Madras India · Department of Computing Science, University of Alberta Canada · Qatar Computing Research Institute, Hamad Bin Khalifa University Qatar +1
LambdaPO: A Lambda Style Policy Optimization for Reasoning Language Models
May 19, 2026Redacted by arXivRL for Language Model Reasoning Group Relative Policy Optimization
General Preference Reinforcement Learning
May 18, 2026Muhammad Umer, Muhammad Ahmed Mohsin, Ahsan Bilal +5Multi-Objective Language Model Alignment RL Fine-Tuning
Stanford University · The University of Oklahoma
Pairwise Preference Reward and Group-Based Diversity Enhancement for Superior Open-Ended Generation
May 18, 2026Guining Cao, Jiaxin Peng, Chu Zeng +3Pairwise Preference Learning Open-Ended Generation
Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd · School of Software and Microelectronics, Peking University · Tsinghua University
ClaHF: A Human Feedback-inspired Reinforcement Learning Framework for Improving Classification Tasks
May 17, 2026Tianxiang Xu, Xiaoyan Zhu, Xin Lai +1Reward Modeling Reinforcement Learning
School of Computer Science and Technology, Xi’an Jiaotong University
Transitivity Meets Cyclicity: Explicit Preference Decomposition for Dynamic Large Language Model Alignment
May 17, 2026Yucong Huang, Xiucheng Li, Kaiqi Zhao +1Reward Modeling LLM Alignment
Harbin Institute of Technology, Shenzhen, China.
OHP-RL: Online Human Preference as Guidance in Reinforcement Learning for Robot Manipulation
May 15, 2026Yunyang Mo, Jian Li, Qiwei Wu +2Robotic RL Human-in-the-Loop AI
The Hong Kong University of Science and Technology (Guangzhou)
Embedding-perturbed Exploration Preference Optimization for Flow Models
May 15, 2026Sujie Hu, Chubin Chen, Jiashu Zhu +3AI Alignment Group Relative Policy Optimization
1Tsinghua University · ⋆Work done during the internship at AMAP · 2AMAP, Alibaba Group
Distributed Zeroth-Order Policy Gradient for Networked Multi-agent Reinforcement Learning from Human Feedback
May 15, 2026Pengcheng Dai, He Wang, Dongming Wang +2Policy Gradient Decentralized MARL
Singapore University of Technology and Design, Singapore 487372, Singapore · Southeast University, Nanjing 211189, China · University of California, Riverside, CA 92521, USA +2
Driving Intents Amplify Planning-Oriented Reinforcement Learning
May 12, 2026Hengtong Lu, Victor Shea-Jay Huang, Chengmin Yang +4RL for Autonomous Driving Goal-Conditioned RL
Li Auto · Tsinghua University · CUHK MMLab
Semantic Reward Collapse and the Preservation of Epistemic Integrity in Adaptive AI Systems
May 12, 2026William ParrisReward Modeling Reward Hacking
BDB Labs / BagelTech · San Diego, California, USA
RankQ: Offline-to-Online Reinforcement Learning via Self-Supervised Action Ranking
May 11, 2026Andrew Choi, Wei XuQ-Learning Robotic RL
Horizon Robotics

Jun 24, 2026, cs.ROJ/K move · Enter open · S save

MAPL: Multi-Objective Preference Learning for Robot Locomotion

Xiyue Chen, Muhan Lin, Shuyang Shi, Joseph Campbell

Department of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47906, USA · Department of Computer Science, Purdue University, West Lafayette, IN 47906, USA · independent researcher

Multi-Objective Reinforcement Learning Reward Modeling RL for Legged Locomotion Policy Learning Reward Design for RL Preference Learning+1

Open paper AskPDF arXiv

Reward design remains a major bottleneck in reinforcement learning for robot locomotion, where successful policies often depend on carefully tuned, task-specific reward functions. Preference-based reinforcement learning offers an alternative, but existing LLM-based methods typically ask for a single overall judgment between behaviors, making it difficult to capture the multiple competing objectives that underlie high-quality locomotion. We present Multi-Objective AI-Informed Preference Learning (MAPL), a framework that learns locomotion rewards from high-level natural language objectives rather than manually engineered reward equations. MAPL prompts a large language model to compare trajectories independently along semantically meaningful criteria, using generic language descriptions that are terrain-invariant and require little domain expertise. These objective-wise preferences are used to train a multi-head preference scoring model, whose outputs are aggregated to form a scalar reward for policy optimization. Across four quadruped locomotion environments, MAPL trains policies using only LLM-generated preferences and achieves performance comparable to or better than expert-designed rewards, while eliminating task-specific reward engineering.

Momentum

Related topics

Latest papers 77