Value Decomposition

Value decomposition in reinforcement learning aims to simplify complex multi-agent systems by breaking down the overall value function into individual or group-level components, facilitating more efficient learning and decision-making. Current research focuses on developing novel algorithms, such as those employing factor graphs, attention mechanisms, and ensemble methods, to improve the accuracy and scalability of value decomposition, particularly within the context of cooperative multi-agent reinforcement learning (MARL). These advancements address challenges like high-dimensional action spaces, partial observability, and the need for privacy-preserving solutions, ultimately improving the performance and applicability of MARL in diverse domains such as robotics, autonomous driving, and game playing. The resulting improvements in efficiency and interpretability are significant for both theoretical understanding and practical deployment of MARL systems.

Papers

August 12, 2024

QTypeMix: Enhancing Multi-Agent Cooperative Strategies through Heterogeneous and Homogeneous Value Decomposition
Songchen Fu, Shaojing Zhao, Ta Li, YongHong Yan
Multi Agent Reinforcement Learning Multi Agent Heterogeneous Agent Value Decomposition

May 9, 2024

Dynamic Deep Factor Graph for Multi-Agent Reinforcement Learning
Yuchen Shi, Shihong Duan, Cheng Xu, Ran Wang, Fangwen Ye, Chau Yuen
Multi Agent Reinforcement Learning Graph Generation Factor Graph Value Decomposition Deep Factor Graph

January 16, 2024

REValueD: Regularised Ensemble Value-Decomposition for Factorisable Markov Decision Processes
David Ireland, Giovanni Montana
Reinforcement Learning Multi Agent Reinforcement Learning Markov Decision Process Action Space Optimal Action Value Decomposition Private Valuation

September 13, 2023

Privacy-Engineered Value Decomposition Networks for Cooperative Multi-Agent Reinforcement Learning
Parham Gohari, Matthew Hale, Ufuk Topcu
Differential Privacy Multi Agent Cooperative Multi Agent Reinforcement Learning Multi Agent Coordination Value Decomposition

September 8, 2023

Leveraging World Model Disentanglement in Value-Based Multi-Agent Reinforcement Learning
Zhizun Wang, David Meger
Multi Agent Multi Agent System Disentanglement Capability Value Decomposition Action Value Function Model Based Multi Agent Reinforcement

June 24, 2023

TVDO: Tchebycheff Value-Decomposition Optimization for Multi-Agent Reinforcement Learning
Xiaoliang Hu, Pengcheng Guo, Chuanwei Zhou, Tong Zhang, Zhen Cui
Multi Agent Reinforcement Learning Multi Objective Value Decomposition Action Value Function Value Function Decomposition

May 12, 2023

Boosting Value Decomposition via Unit-Wise Attentive State Representation for Cooperative Multi-Agent Reinforcement Learning
Qingpeng Zhao, Yuanyang Zhu, Zichuan Liu, Zhi Wang, Chunlin Chen
State Space Cooperative Multi Agent Reinforcement Learning State Representation Multi Head Attention Mechanism Value Decomposition Compact Latent

April 26, 2023

N$\text{A}^\text{2}$Q: Neural Attention Additive Model for Interpretable Multi-Agent Q-Learning
Zichuan Liu, Yuanyang Zhu, Chunlin Chen
Reinforcement Learning Credit Assignment Value Decomposition Additive Attention

March 23, 2023

Stochastic Graph Neural Network-based Value Decomposition for MARL in Internet of Vehicles
Baidi Xiao, Rongpeng Li, Fei Wang, Chenghui Peng, Jianjun Wu, Zhifeng Zhao, Honggang Zhang
Multi Agent Reinforcement Learning Vehicle Plan Vehicular Network Multi Agent Reinforcement Value Decomposition Autonomous Vehicle Control

March 16, 2023

SVDE: Scalable Value-Decomposition Exploration for Cooperative Multi-Agent Reinforcement Learning
Shuhan Qi, Shuhao Zhang, Qiang Wang, Jiajia Zhang, Jing Xiao, Xuan Wang
Cooperative Multi Agent Reinforcement Learning Intrinsic Reward State Action Space Value Decomposition

February 14, 2023

Adaptive Value Decomposition with Greedy Marginal Contribution Computation for Cooperative Multi-Agent Reinforcement Learning
Shanqi Liu, Yujing Hu, Runze Wu, Dong Xing, Yu Xiong, Changjie Fan, Kun Kuang, Yong Liu
Cooperative Multi Agent Reinforcement Learning Client Contribution Value Decomposition

February 4, 2023

Dual Self-Awareness Value Decomposition Framework without Individual Global Max for Cooperative Multi-Agent Reinforcement Learning
Zhiwei Xu, Bin Zhang, Dapeng Li, Guangchong Zhou, Zeren Zhang, Guoliang Fan
Cooperative Multi Agent Reinforcement Learning Self Awareness Value Decomposition

November 23, 2022

Contrastive Identity-Aware Learning for Multi-Agent Value Decomposition
Shunyu Liu, Yihe Zhou, Jie Song, Tongya Zheng, Kaixuan Chen, Tongtian Zhu, Zunlei Feng, Mingli Song
Multi Agent Cooperative Multi Agent Reinforcement Learning Multi Agent Network Identity Preserving Value Decomposition Diverse Agent

November 2, 2022

Numerical Optimizations for Weighted Low-rank Estimation on Language Model
Ting Hua, Yen-Chang Hsu, Felicity Wang, Qian Lou, Yilin Shen, Hongxia Jin
Language Model Compression Technique Singular Value Decomposition Numerical Optimization Value Decomposition Low Rank Matrix Estimation

October 26, 2022

Non-Linear Coordination Graphs
Yipeng Kang, Tonghan Wang, Xiaoran Wu, Qianlan Yang, Chongjie Zhang
Multi Agent Reinforcement Learning Multi Agent Coordination Payoff Function Value Decomposition Coordination Graph

August 7, 2022

Maximum Correntropy Value Decomposition for Multi-agent Deep Reinforcemen Learning
Kai Liu, Tianxian Zhang, Lingjiang Kong
Multi Agent Multi Agent Deep Reinforcement Learning Value Decomposition Soft QMIX Maximum Correntropy Criterion

June 24, 2022

Value Function Decomposition for Iterative Design of Reinforcement Learning Agents
James MacGlashan, Evan Archer, Alisa Devlic, Takuma Seno, Craig Sherstan, Peter R. Wurman, Peter Stone
Reinforcement Learning Reinforcement Learning Agent Actor Critic Algorithm Agent Capability Decision Making Agent Value Decomposition Iterative Design Value Function Decomposition

February 10, 2022

Understanding Value Decomposition Algorithms in Deep Cooperative Multi-Agent Reinforcement Learning
Zehao Dou, Jakub Grudzien Kuba, Yaodong Yang
Multi Agent Reinforcement Learning Cooperative Multi Agent Reinforcement Learning Value Decomposition Game Decomposition Value Function Decomposition

December 1, 2021

Conditional Expectation based Value Decomposition for Scalable On-Demand Ride Pooling
Avinandan Bose, Pradeep Varakantham
Value Decomposition Approximate Dynamic Programming Conditional Expectation Ride Pooling

Value Decomposition

Papers

QTypeMix: Enhancing Multi-Agent Cooperative Strategies through Heterogeneous and Homogeneous Value Decomposition

Dynamic Deep Factor Graph for Multi-Agent Reinforcement Learning

REValueD: Regularised Ensemble Value-Decomposition for Factorisable Markov Decision Processes

Privacy-Engineered Value Decomposition Networks for Cooperative Multi-Agent Reinforcement Learning

Leveraging World Model Disentanglement in Value-Based Multi-Agent Reinforcement Learning

TVDO: Tchebycheff Value-Decomposition Optimization for Multi-Agent Reinforcement Learning

Boosting Value Decomposition via Unit-Wise Attentive State Representation for Cooperative Multi-Agent Reinforcement Learning

N$\text{A}^\text{2}$Q: Neural Attention Additive Model for Interpretable Multi-Agent Q-Learning

Stochastic Graph Neural Network-based Value Decomposition for MARL in Internet of Vehicles

SVDE: Scalable Value-Decomposition Exploration for Cooperative Multi-Agent Reinforcement Learning

Adaptive Value Decomposition with Greedy Marginal Contribution Computation for Cooperative Multi-Agent Reinforcement Learning

Dual Self-Awareness Value Decomposition Framework without Individual Global Max for Cooperative Multi-Agent Reinforcement Learning

Contrastive Identity-Aware Learning for Multi-Agent Value Decomposition

Numerical Optimizations for Weighted Low-rank Estimation on Language Model

Non-Linear Coordination Graphs

Maximum Correntropy Value Decomposition for Multi-agent Deep Reinforcemen Learning

Value Function Decomposition for Iterative Design of Reinforcement Learning Agents

Understanding Value Decomposition Algorithms in Deep Cooperative Multi-Agent Reinforcement Learning

Conditional Expectation based Value Decomposition for Scalable On-Demand Ride Pooling