Multi-Objective Reinforcement Learning

Also known as MORL

Multi-objective reinforcement learning studies sequential decisions where goals such as safety, efficiency, fairness, and reward quality can conflict. Common approaches learn a set of Pareto-optimal policies or condition one policy on user preferences, using methods such as scalarization, vector-valued critics, policy-gradient updates, and multi-agent coordination. Researchers apply these methods in robotics, traffic and energy control, recommendation, and language-model alignment, evaluating trade-offs with Pareto-front coverage, hypervolume, and task-specific benchmarks. A central challenge is balancing reward signals without letting easier objectives dominate or causing reward hacking, while ensuring policies reliably respond to changing preferences.

Latest papers

146 papers25 papers in the last four weeks, up 150% on the four weeks before. 0.1% of all new papers.

Safe Multi-Objective Reinforcement Learning: Zero Constraint Violation and Primal-Averaging Convergence
Oct 29, 2026Chenxi Li, Yao Shu, Siyang Gao +2Multi-Objective Reinforcement Learning Reinforcement Learning
Harbin Institute of Technology, Harbin, Heilongjiang, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, Guangdong, China · City University of Hong Kong, Hong Kong, China
CONCUR: Concordant Auxiliary Rewards for Multi-Task, Multi-Reward Reinforcement Learning
Oct 27, 2026Jixuan Leng, Si Si, Fei Wang +4Reinforcement Learning RL for Language Models
CMU · Google DeepMind · Google +1
Preference-Guided Cooperation in Multi-Objective Multi-Agent Reinforcement Learning
Oct 26, 2026Qiyue Xia, J. Michael HerrmannMulti-Objective Reinforcement Learning Multi-Agent Reinforcement Learning
School of Informatics, University of Edinburgh, Edinburgh, United Kingdom
Multi-Objective Reinforcement Learning Using Routed Ensembles and Trajectory Attribution
Oct 26, 2026Zhenshuo Zhang, Ruoxuan Xiong, Hongyang R. ZhangMulti-Objective Reinforcement Learning Reinforcement Learning
Northeastern University, Boston, MA · Emory University, Atlanta, GA
GPDPO: Learning from Reward Conflicts via Group Pareto-Dominance Policy Optimization
Oct 26, 2026Zixuan Xu, Haoxuan Chen, Jiangxin Sun +1Multi-Objective Language Model Alignment Multi-Objective Reinforcement Learning
ISEE Lab, Sun Yat-sen University · Computer Science and Technology, University of Trento
Locked In: Multi-Objective RL for the AI Data Center Build-Out
Oct 26, 2026Satyapragnya Kar, Rajarshi Das, PR PanigrahiMulti-Objective Reinforcement Learning Data Center Energy Management
New York University · AlphaInfra.ai, MQube Cognition · Envana Software Solutions
Priority-Preserving Advantage Aggregation for Multi-Reward Policy Optimization
Oct 26, 2026Xiangchi Yuan, Xingjian Diao, Zheyuan Liu +7Multi-Objective Reinforcement Learning RL Post-Training
Georgia Institute of Technology · Dartmouth College · University of Notre Dame
PPO vs. SAC: A Comparative Study for Cooperative Motion Control of Dual-Arm Space Robots
Oct 23, 2026Nasim Fallahiarezoodar, Zheng H. ZhuSpace Robotics Bimanual Robotic Manipulation
Department of Earth and Space Science and Engineering, York University, M3J1P3 · Mechanical Engineering, York University, M3J1P3
State-Dependent Objective Coordination for End-to-End Balance and Navigation
Oct 21, 2026Xuhao Huang, Xitao Xu, Yuxin Zheng +2Robotic Control Robot Navigation
One Policy, Many Styles: Where to Aggregate Preferences for Steerable Humanoid Control?
Oct 20, 2026Ziqian Huang, Yuzhuo Chen, Shimin Zhang +3Preference-Based RL Humanoid Robot Control
College of Computing Georgia Institute of Technology
Intra-Rubric Reward Divergence: Mitigating Rubric Hacking through Latent Dimension Recovery and Balancing
Oct 19, 2026Jianqiang Xia, Han Xu, Dianxi ShiReward Hacking Rubric-Based RL
Shanghai Jiao Tong University · Baidu, Inc. · Intelligent Game and Decision Lab
VaDar: Implicit Goal Conditioning via Value Disaggregation
Oct 19, 2026Shashwat Saxena, Mehul Goel, Sreyas Venkataraman +2Long-Horizon RL Reinforcement Learning
Carnegie Mellon University
Pareto-Optimal RTL Code Generation via Multi-Objective Reinforcement Learning with Large Language Models
Oct 16, 2026Koki Takeshita, Kenji Uebayashi, Hikaru Tomonari +6RTL Code Generation Electronic Design Automation
Research & Development Group, Hitachi, Ltd.
Each Reward to Its Relevant Tokens: Reward-Specific Token-Level Credit Assignment for Multi-Reward RL
Oct 15, 2026Skylar Zhai, Haonan Wang, Zhining Liu +4RL for Language Models Token-Level Credit Assignment
University of Minnesota · National University of Singapore · University of Illinois Urbana-Champaign +3
Multi-Reward Hacking: From Aggregation-Induced Drift to Trust-Anchored Defense
Oct 15, 2026Yushu Li, Jiin Woo, Xiaoxiao Li +1Reward Hacking RL for Language Models
University of British Columbia · Amazon Annapurna Labs
Stein Variational Distributional Reinforcement Learning: Score-Guided Particles for Multivariate Return Distributions
Oct 13, 2026Manuel Haußmann, Melih KandemirDistributional RL Risk-Sensitive RL
University of Southern Denmark
MORAD: Reinforcement Learning with Multi-Objective Rewards for RNA Inverse Design
Oct 9, 2026Jixin Tang, Ji Guo, Jun ZhaoInverse Design Reward Design for RL
Fudan University
Strategic Investment Decision Making for Value Creation in Energy Transition: A Reinforcement Learning Approach
Oct 7, 2026Yasaman Cheraghi, Reidar B. Bratvold, Aojie Hong +2Sequential Decision Making Decision-Making under Uncertainty
Department of Energy and Petroleum Engineering, University of Stavanger, Norway · Independent Researcher, Stavanger, Norway · NORCE Norwegian Research Centre, Bergen, Norway
Transporting Unsecured Stacked Payloads with a Quadrupedal Robot via Multi-Objective Reinforcement Learning
Oct 5, 2026Nobuo Namura, Masayuki Hiromoto, Kento Uemura +2Multi-Objective Reinforcement Learning Robot Policy Adaptation
Fujitsu Limited, Kawasaki, Japan
Which Preferences to Train On? End-to-End Multi-Objective Alignment with an Adversarial Preference Distribution
Oct 4, 2026Minjae Lee, Kyunghyun Cho, Sangdon ParkLLM Alignment Multi-Objective Language Model Alignment
GSAI POSTECH · Center for Data Science New York University · GSAI & CSE POSTECH
OuroReward: Sequential Reward Scheduling for Reinforcement Learning in Text-to-3D Generation
Oct 2, 2026Bingyang Cui, Yujie Zhang, Yiling Xu +1Prompt Selection for RL Text-to-3D Generation
Shanghai Jiao Tong University
Lexicographic Multi-Objective On-Policy Distillation
Oct 1, 2026Doseok Jang, Jon Ander Campos, Youran QiMulti-Objective Reinforcement Learning Multi-Objective Optimization
Cohere · Mila, Universit´e de Montr´eal
PROMO: Preference-conditioned Multi-Objective Reinforcement Learning for Quadrupedal Robots
Oct 1, 2026Amr Mousa, Rifny Rachman, Neil Karavis +2Multi-Objective Reinforcement Learning Energy-Efficient Robot Locomotion
The University of Manchester, United Kingdom · BAE Systems, United Kingdom · University of Warwick, United Kingdom
Make Sparse Rewards Count: Density-Aware Reward Aggregation for Multi-Reward RL
Sep 30, 2026Tong Zheng, Skylar Zhai, Zhan Cheng +7Multi-Objective Reinforcement Learning Reinforcement Learning
University of Chinese Academy of Sciences · University of Minnesota Twin Cities · University of Wisconsin–Madison +5
MoFlow: Multi-Objective Agentic Workflow Generation
Sep 29, 2026Yining Lu, Aurelie Lozano, Xi Yang +3Multi-Objective Reinforcement Learning Agentic Workflows
University of Notre Dame · IBM
Adaptive Reward Routing: Dynamic Multi-Reward Optimization for Joint Audio-Video Diffusion via Forward-Process RL
Sep 29, 2026Songlin Yang, Xiaotong Zhao, Jiacheng Zhang +5Multi-Objective Reinforcement Learning Cross-Modal Alignment
Tencent Video · The University of Hong Kong
CorrGRPO: Correlation-Normalized GRPO for Multi-Reward Learning
Sep 29, 2026Wenbin Hu, Huihao Jing, Haochen Shi +3Multi-Objective Reinforcement Learning RL for Language Models
Hong Kong University of Science and Technology
MARCO: Multi-Round Agentic Reinforcement for Conditional Molecular Optimization
Sep 29, 2026Shicheng Fang, Yuxin Wang, Zhuo Yang +6Molecular Optimization RL for Language Models
Fudan University · Shanghai Innovation Institute · Southeast University +2
Optimal Multi-Reward Reinforcement Learning
Sep 29, 2026Zijun Chen, Zihan ZhangMarkov Decision Processes Sample-Efficient RL
Department of Computer Science and Engineering Hong Kong University of Science and Technology
ORPG: Reconciling Multiple Reward Objectives through Objective-wise Policy Gradients
Sep 28, 2026Shicheng Fang, Yiwen Zhao, Wenbo Tian +4Multi-Objective Reinforcement Learning Gradient Interference
Fudan University · Shanghai Innovation Institute

Oct 29, 2026J/K move · Enter open · S save

Safe Multi-Objective Reinforcement Learning: Zero Constraint Violation and Primal-Averaging Convergence

Chenxi Li, Yao Shu, Siyang Gao, Shuang Qiu+1

Harbin Institute of Technology, Harbin, Heilongjiang, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, Guangdong, China · City University of Hong Kong, Hong Kong, China

Multi-Objective Reinforcement Learning Reinforcement Learning Safe RL Sample-Efficient RL

Open paper AskPDF arXiv

Many applications of multi-objective reinforcement learning (MORL) require policies that balance conflicting objectives while satisfying safety constraints. We study safe MORL with zero constraint violation throughout learning, covering both cumulative and stepwise constraints. A key challenge is to explore sufficiently under unknown dynamics and constraint costs without violating the constraints. For a given preference over objectives, we develop an algorithm to learn a safe Pareto optimal policy by combining optimistic reward estimates with pessimistic cost bounds. With high probability, every policy executed over

T

episodes incurs zero constraint violation. The algorithm returns a safe Markov policy induced by the averaged occupancy measure and achieves an

\widetilde{O}(1/\sqrt{T})

suboptimality gap under smooth Tchebycheff scalarization. We further develop a preference-free algorithm that first explores safely without a prespecified preference and then computes a safe policy for any given preference without further interaction. Over

T

exploration episodes and

K

planning iterations, it retains the same safety guarantee and achieves an

\widetilde{O}(1/\sqrt{T}+1/\sqrt{K})

suboptimality gap. Both algorithms require knowledge of a policy that strictly satisfies the constraints and its costs. We then relax these requirements, incurring at most constant cumulative violation. To our knowledge, we provide the first sample-efficient MORL algorithms that guarantee safety for every executed policy.