A Game-Theoretic Spatio-Temporal Reinforcement Learning Framework for Collaborative Public Resource Allocation

Songxin Lei; Qiongyan Wang; Yanchen Zhu; Hanyu Yao; Sijie Ruan; Weilin Ruan; Yuyu Luo; Huaming Wu

cs.LGOct 30, 2025

A Game-Theoretic Spatio-Temporal Reinforcement Learning Framework for Collaborative Public Resource Allocation

Authors: Songxin Lei, Qiongyan Wang, Yanchen Zhu, Hanyu Yao, Sijie Ruan, Weilin Ruan, Yuyu Luo, Huaming Wu, +1 more

Organizations: The Hong Kong University of Science and Technology (Guangzhou) · Peking University · Beijing Institute of Technology · Tianjin University

Abstract

Public resource allocation involves distributing resources, including urban infrastructure, energy, and transportation, which are typically limited in capacity, to meet social demands. In real-world scenarios, resources are typically limited in capacity, which makes coordination among multiple resources essential. However, existing methods often optimize resource movements in an isolated manner and do not explicitly account for capacity-aware collaboration under spatio-temporal dynamics. To address this limitation, we introduce the Collaborative Public Resource Allocation (CPRA) problem, and propose a Game-Theoretic Spatio-Temporal Reinforcement Learning (GSTRL) framework to solve it. Our contributions are twofold: 1) We formulate CPRA as a potential game and construct the potential function based on the objective function of CPRA, laying a theoretical foundation for approximating the Nash equilibrium of this NP-hard problem; and 2) Our GSTRL framework effectively captures the spatio-temporal dynamics of the overall system. We evaluate GSTRL on two real-world datasets, where experiments show its superior performance. Our source codes are available at https://github.com/thunderlrr/GSTRL.

Figures & tables

Figure 1 . Dynamic public resource allocation vs. Collaborative public resource allocation (CPRA).

Figure 2 . Overall architecture of the proposed GSTRL framework.

	Method	Algorithms	#Param(K)	E=30		E=40		E=50		E=60
	Method	Algorithms	#Param(K)	ADCC $\uparrow$	$\Delta$ $\uparrow$	ADCC $\uparrow$	$\Delta$ $\uparrow$	ADCC $\uparrow$	$\Delta$ $\uparrow$	ADCC $\uparrow$	$\Delta$ $\uparrow$
Happy Valley	Heuristic	Static	-	1,566	-14%	1,673	-20%	2,076	-17%	2,106	-13%
		BB	-	1,717	-6%	1,686	-20%	2,128	-15%	2,235	-8%
		MYOPIC	-	1,756	-4%	1,789	-15%	2,206	-12%	2,289	-6%
		EADS	-	1,831	-	2,096	-	2,493	-	2,432	-
	RL	REINFORCE	211	1,936 $\pm$ 8	6%	2,181 $\pm$ 3	4%	2,239 $\pm$ 5	-10%	2,103 $\pm$ 10	-14%

Table 1 . Model comparison on the Happy Valley and TaxiBJ datasets. The

\Delta

represents the improvements in ADCC compared to the EADS approach. Bold and underlined digits are the best and EADS approach values.

Variant	Happy Valley			TaxiBJ
Variant	E=30	E=40	E=50	E=30	E=40	E=50
w/o AE	2,016	2,021	2,121	217	219	257
w/o S	1,272	1,372	1,385	139	141	167
w/o T	2,158	2,216	2,300	227	228	273
w/o PF	1,982	2,046	2,165	225	213	262
w/o EM	136	368	551	89	106	130

Table 2 . Ablation Study Results.

Figure 3 . ADCC on different GSTRL variants.

Figure 4 . ADCC on different hidden sizes and batch sizes.

Figure 5 . Strategies visualization of GSTRL compared with EADS in Happy Valley dataset.

Explore similar work

Learning Altruistic Collaboration in Heterogeneous Multi-Team Systems
May 20, 2026Riwa Karam, Ruoyu Lin, Brooks A. Butler +1Multi-Robot Systems
Samueli School of Engineering, University of California, Irvine, Irvine, CA, 92697, USA · University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA
Efficiency, Feasibility, and Incentive-Awareness in Constrained Online Resource Allocation
Jul 13, 2025Yan Dai, Negin Golrezaei, Patrick JailletIncentives Primal-Dual Methods
Operations Research Center, MIT. · Sloan School of Management, MIT. · Department of EECS, MIT.
Heterogeneous AAV Logistics Task Allocation: A Reinforcement Learning Enhanced Overlapping Coalition Formation Game Approach
May 26, 2026Yuze Zhou, Jingliang Sun, Junzhi Li +3Heterogeneous Robot Teams Coalition Formation
Beijing Institute of Technology, Beijing 100081, China · Key Laboratory of Dynamics and Control of Flight Vehicle, Ministry of Education, Beijing 100081, China · Beijing Institute of Technology Chongqing Innovation Center, Chongqing, 401121, China +1

May 20, 2026, cs.ROJ/K move · Enter open · S save

Learning Altruistic Collaboration in Heterogeneous Multi-Team Systems

Riwa Karam, Ruoyu Lin, Brooks A. Butler, Magnus Egerstedt

Samueli School of Engineering, University of California, Irvine, Irvine, CA, 92697, USA · University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA

Multi-Robot Systems

Open paper AskPDF arXiv

This paper studies heterogeneous multi-team collaboration through dynamic robot allocation, where robots are treated as transferable resources. Leveraging Hamilton's rule from ecology as an altruistic decision-making mechanism, we propose a multi-team collaborative resource allocation framework with heterogeneous capabilities, transfer costs, and capability-dependent contributions. The resulting allocation problem is combinatorial and is shown to be NP-hard. To address scalability, we develop a graph neural network policy under centralized training and decentralized execution that approximates the altruistic allocations based on Hamilton's rule. The model operates over the team interaction graph and predicts robot-level transfer decisions and next robot-to-team assignments. The proposed approach is validated in a firefighting scenario through simulations and experiments, demonstrating that the learned policy achieves near-optimal performance while scaling to larger systems.

Abstract

Explore similar work

Learning Altruistic Collaboration in Heterogeneous Multi-Team Systems

Efficiency, Feasibility, and Incentive-Awareness in Constrained Online Resource Allocation

Heterogeneous AAV Logistics Task Allocation: A Reinforcement Learning Enhanced Overlapping Coalition Formation Game Approach