Multi-Objective Reinforcement Learning

Also known as MORL

Multi-objective reinforcement learning studies sequential decisions where goals such as safety, efficiency, fairness, and reward quality can conflict. Common approaches learn a set of Pareto-optimal policies or condition one policy on user preferences, using methods such as scalarization, vector-valued critics, policy-gradient updates, and multi-agent coordination. Researchers apply these methods in robotics, traffic and energy control, recommendation, and language-model alignment, evaluating trade-offs with Pareto-front coverage, hypervolume, and task-specific benchmarks. A central challenge is balancing reward signals without letting easier objectives dominate or causing reward hacking, while ensuring policies reliably respond to changing preferences.

Latest papers

146 papers25 papers in the last four weeks, up 150% on the four weeks before. 0.1% of all new papers.

PIRS: Physics-Informed Reward Shaping for SAC-Based Building Energy Management
May 27, 2026Shadmehr Zaregarizi, Khashayar YavariMulti-Objective Reinforcement Learning Reward Shaping
Politecnico di Torino Turin, Italy
Focal Reward: Balanced Reinforcement Learning under Rubric-Based Rewards
May 26, 2026Yu Huang, Zihua Zhao, Zhaoxin Huan +9Multi-Objective Reinforcement Learning RL for Language Models
Shanghai Jiao Tong University · Ant Group
Joint Optimization of Training and Inference in Federated Edge Learning via Constrained Multi-Objective Deep Reinforcement Learning
May 25, 2026Zhen Li, Jun Cai, Chao Yang +1Multi-Objective Reinforcement Learning Edge Inference
Department of Electrical and Computer Engineering, Concordia University, Montreal, QC, H3G 1M8, Canada · School of Automation, Guangdong University of Technology, China
DVAO: Dynamic Variance-adaptive Advantage Optimization for Multi-reward Reinforcement Learning
May 25, 2026Guochao Jiang, Jingyi Song, Guofeng Quan +3Multi-Objective Reinforcement Learning RL for Language Model Reasoning
Alibaba Cloud Computing
Micro-Swarm Locomotion Optimization in Dynamic Flow using Multi-Objective Multi-Agent Reinforcement Learning
May 24, 2026Josef Berman, Oren GalMulti-Objective Reinforcement Learning Multi-Agent System Optimization
The Hatter Department of Marine Technologies, Leon H. Charney School of Marine Sciences, University of Haifa, Israel.
Emission-Aware Reinforcement Learning for Sustainable Electric Vehicle Charging and Carbon Dioxide Reduction Under Varying Renewable Penetration
May 23, 2026Ninglin Ou, Mohammad A. Razzaque, Iftekher Islam Shovon +5RL Control Multi-Objective Reinforcement Learning
anasc Research, School of Computer Science & IT, University College Cork, IE · School of Computing, Engineering and Digital Technologies, Teesside University, UK · International Energy Research Centre, Tyndall National Institute, Cork, IE +1
Vector Policy Optimization: Training for Diversity Improves Test-Time Search
May 21, 2026Ryan Bahlous-Boldi, Isha Puri, Idan Shenfeld +6Multi-Objective Reinforcement Learning Reinforcement Learning
1MIT · 2Improbable AI Lab · 4Sakana AI +1
Two is better than one: A Collapse-free Multi-Reward RLIF Training Framework
May 21, 2026Shourov Joarder, Diganta Sikdar, Ahsan Habib Akash +2KL-Regularized RL Intrinsic Reward Methods
Bangladesh University of Engineering and Technology, Bangladesh · West Virginia University, USA · University of Aberdeen, UK +2
Multi-Objective and Mixed-Reward Reinforcement Learning via Reward-Decorrelated Policy Optimization
May 13, 2026Yang Bai, Kaiyuan Liu, Ziyuan Zhuang +5Multi-Objective Reinforcement Learning RL for Language Models
Meituan, China
Adaptive Smooth Tchebycheff Attention for Multi-Objective Policy Optimization
May 12, 2026Alejandro Murillo-Gonzalez, Mahmoud Ali, Lantao LiuMulti-Objective Reinforcement Learning Policy Optimization
Indiana University–Bloomington
Split the Differences, Pool the Rest: Provably Efficient Multi-Objective Imitation
May 12, 2026Ziyad Sheebaelhamd, Luca Viano, Volkan Cevher +1Multi-Objective Reinforcement Learning Imitation Learning
University of Tübingen · EPFL · University of Technology Nuremberg
Hierarchical LLM-Driven Control for HAPS-Assisted UAV Networks: Joint Optimization of Flight and Connectivity
May 12, 2026Zijiang Yan, Hao Zhou, Wael Jaafar +4Multi-Objective Reinforcement Learning Language Model-Based Control
Department of Electrical Engineering and Computer Science, York University, Toronto, ON, Canada · Samsung Research America, Toronto, ON, Canada · Department of Software and IT Engineering, École de technologie supérieure (ÉTS), University of Quebec, Montreal, QC, Canada +1
Controllability in preference-conditioned multi-objective reinforcement learning
May 11, 2026Pau de las Heras Molins, Beyazit Yalcinkaya, Lasse Peters +2Multi-Objective Reinforcement Learning Preference-Based RL
LTCI, Télécom Paris, Institut Polytechnique de Paris · University of California, Berkeley · TU Delft +1
A Single Deep Preference-Conditioned Policy for Learning Pareto Coverage Sets
May 9, 2026Akihiro Kubo, Kosuke Nakanishi, Shin IshiiMulti-Objective Reinforcement Learning Policy Optimization
International Research Center for Neurointelligence, The University of Tokyo, Tokyo, Japan · ATR Neural Information Analysis Laboratories, Kyoto, Japan · Department of Information Science, Kyoto University, Kyoto, Japan
BalCapRL: A Balanced Framework for RL-Based MLLM Image Captioning
May 8, 2026Shaokai Ye, Vasileios Saveris, Yihao Qian +3Multi-Objective Reinforcement Learning Image Captioning
Apple
A Production-Ready RL Framework for Personalized Utility Tuning with Pareto Sweeping in Pinterest Recommender Systems
May 8, 2026Yichu Zhou, Mehdi Ben Ayed, Lin Yang +9Multi-Objective Reinforcement Learning Recommender Systems
Pinterest Inc. Seattle, Washington, USA · Pinterest Inc. New York City, New York, USA · Pinterest Inc. San Francisco, California, USA +1
MARBLE: Multi-Aspect Reward Balance for Diffusion RL
May 7, 2026Canyu Zhao, Hao Chen, Yunze Tong +3Multi-Objective Reinforcement Learning Diffusion Model Alignment
Zhejiang University · Work done during an internship at HiThink. · HiThink +1
Dueling DDQN-Based Adaptive Multi-Objective Handover Optimization for LEO Satellite Networks
May 4, 2026Po-Heng Chou, Chiapin Wang, Chung-Chi Huang +1Multi-Objective Reinforcement Learning Deep Q-Learning
Research Center for Information Technology Innovation (CITI), Academia Sinica (AS), Taipei 11529, Taiwan · Department of Electrical Engineering, National Taiwan Normal University (NTNU), Taipei 106308 Taiwan
ProMORNA: An End-to-End Generative Framework for Full-Length mRNA Design
May 2, 2026Zixi Shao, Tao Wang, Ye Shen +2Multi-Objective Reinforcement Learning Molecular Generation
Nanjing Foreign Language School, Nanjing, China · School of Science, China Pharmaceutical University, Nanjing, China · School of Pharmacy, China Pharmaceutical University, Nanjing, China
A Reward-Free Viewpoint on Multi-Objective Reinforcement Learning
Apr 27, 2026Ying-Tu Chen, Wei Hung, Bing-Shu Wu +2Multi-Objective Reinforcement Learning Reinforcement Learning
National Yang Ming Chiao Tung University, Hsinchu, Taiwan · Massachusetts Institute of Technology
POCA: Pareto-Optimal Curriculum Alignment for Visual Text Generation
Apr 27, 2026Yaohou Fan, Qingzhong Wang, Yongsong Huang +3Multi-Objective Reinforcement Learning RL for Image Generation
Tohoku University · Amazon Web Services
ParetoSlider: Diffusion Models Post-Training for Continuous Reward Control
Apr 22, 2026Shelly Golan, Michael Finkelson, Ariel Bereslavsky +2Multi-Objective Reinforcement Learning RL for Image Generation
Tel Aviv University · Lightricks · Adobe Research
Learning to Credit the Right Steps: Objective-aware Process Optimization for Visual Generation
Apr 21, 2026Rui Li, Ke Hao, Yuanzhi Liang +4Multi-Objective Reinforcement Learning Video Diffusion Models
University of Science and Technology of China HeFei, China · Shanghai Jiao Tong University ShangHai, China · TeleAI ShangHai, China
Multi-objective Reinforcement Learning With Augmented States Requires Rewards After Deployment
Apr 17, 2026Peter Vamplew, Cameron FoaleMulti-Objective Reinforcement Learning Reinforcement Learning
Institute of Innovation, Science and Sustainability, Federation University Australia, University Drive, Mt Helen, 3350, Victoria, Australia
Pareto-Optimal Offline Reinforcement Learning via Smooth Tchebycheff Scalarization
Apr 14, 2026Aadyot Bhatnagar, Peter Mørch Groth, Sebastian Ibarraran +1Multi-Objective Reinforcement Learning Offline RL
Profluent Bio, Inc. · Stanford University
Load constrained wind farm flow control through multi-objective multi-agent reinforcement learning
Apr 13, 2026Teodor Åstrand, Marcus Binder Nilsen, Iasonas Tsaklis +3Multi-Agent Reinforcement Learning Constrained RL
Department of Wind and Energy Systems, Technical University of Denmark, Roskilde, Denmark
One Model for All: Multi-Objective Controllable Language Models
Apr 6, 2026Qiang He, Yucheng Yang, Tianyi Zhou +3Multi-Objective Language Model Alignment Controllable Text Generation
Ruhr University Bochum · Eindhoven University of Technology · Mohamed bin Zayed University of Artificial Intelligence +1
When Rewards Disagree: Reward Weighting for Token-Efficient Video Reasoning
Apr 6, 2026Zinuo Li, Yongxin Guo, Jun Liu +6Video Reasoning VLM Reasoning
University of Western Australia · Tencent Youtu Lab · CUHKSZ +2
HAD: Combining Hierarchical Diffusion with Metric-Decoupled RL for End-to-End Driving
Apr 4, 2026Wenhao Yao, Xinglong Sun, Zhenxin Li +4End-to-End Autonomous Driving Autonomous Driving Planning
Fudan University · NVIDIA
Auction-Based Online Policy Adaptation for Evolving Objectives
Apr 2, 2026Guruprerana Shabadi, Kaushik MallikMulti-Objective Reinforcement Learning Multi-Agent Coordination
University of Pennsylvania, United States · IMDEA Software Institute, Spain

May 27, 2026J/K move · Enter open · S save

PIRS: Physics-Informed Reward Shaping for SAC-Based Building Energy Management

Shadmehr Zaregarizi, Khashayar Yavari

Politecnico di Torino Turin, Italy

Multi-Objective Reinforcement Learning Reward Shaping Building Energy Management Heating, Ventilation, and Air Conditioning Control

Open paper AskPDF arXiv

Occupant comfort and grid-aware energy efficiency are competing objectives whose joint optimization depends critically on how reward functions are specified in deep reinforcement learning (DRL) controllers for buildings. Yet reward design remains largely ad hoc: comfort terms are either hand-tuned heuristics or simple temperature-deviation proxies without explicit grounding in thermal-comfort physics. We present PIRS (Physics-Informed Reward Shaping), which replaces these ad-hoc comfort proxies with the ISO 7730 Predicted Mean Vote (PMV) formulation inside a weighted multi-objective reward for Soft Actor-Critic (SAC). By anchoring the comfort signal in the ISO 7730 PMV formulation, PIRS improves reward interpretability and provides a standards-grounded comfort proxy without changing any other component of the learning pipeline. We evaluate PIRS in CityLearn v2.1.2 (challenge 2022 phase 1) with a central SAC agent trained for 50k steps over five random seeds, and compare against a rule-based controller (RBC), a manually engineered reward (E2), an energy-only reward (E3), and a naive temperature-deviation comfort reward (E4). District-level key performance indicators (KPIs), reported as ratios versus RBC, show that PIRS attains cost, carbon, and electricity metrics on par with the manual baseline while substantially outperforming non-physics-grounded designs -- particularly on load ramping (1.78x vs. ~2.4x RBC) and daily peak demand. All DRL policies remain above RBC at this training budget; we interpret this gap honestly and position PIRS as an interpretable, standards-aligned foundation for reward design rather than a claim of dominance over classical control at limited compute.