Learning Dynamic Games for Cooperative Iterative Tasks

Mark Pustilnik; Francesco Borrelli

cs.ROOct 20, 2026

Learning Dynamic Games for Cooperative Iterative Tasks

Authors: Mark Pustilnik, Francesco Borrelli

Organizations: University of California at Berkeley, USA

Abstract

We consider cooperative and iterative tasks involving multiple agents whose interactions are modeled as a dynamic game with shared constraints. We propose a general learning framework applicable to nonlinear and nonconvex dynamic games that uses previously executed feasible trajectories to construct a sampled joint safe set and player-specific cost-to-go functions. At each time step, the agents jointly select a previously sampled terminal state and a Generalized Nash equilibrium used to reach it. A new solution is adopted only if it satisfies a cooperative acceptance rule agreed upon by the players. The framework accommodates individual non-degradation requirements and selection criteria such as weighted costs and Nash bargaining. We show that the proposed framework guarantees recursive feasibility and characterize conditions under which each player's cost is non-increasing across iterations.

Figures & tables

Player	State	Position [m]	Velocity[m/s]	Heading
1	Initial	$(0,-9)$	0.25	$135^{\circ}$
	Target	$(0,9)$	0	$90^{\circ}$
2	Initial	$(4.5,4.5)$	0.25	$180^{\circ}$
	Target	$(-9,-9)$	0	$-135^{\circ}$
3	Initial	$(-4.5,4.5)$	0.25	$45^{\circ}$
	Target	$(9,-9)$	0	$-90^{\circ}$

TABLE I: Initial and target positions and headings.

Fig. 1: Trajectory evolution under M-NB at iterations $1$ , $4$ , $8$ , and $15$ , with earlier trajectories shown faintly. Blue, orange, and green identify Players 1-3. Legend values give their iteration costs.

Fig. 2: Player costs across iterations for M-NB, M-WS, and N-NB. Initial costs are annotated because they lie outside the displayed ranges. N-NB terminates at iteration $6$ , with costs unchanged from iteration $5$ to numerical tolerance.

$j$	Method	$\mathcal{J}_{1}^{j}$	$\mathcal{J}_{2}^{j}$	$\mathcal{J}_{3}^{j}$	$\sum_{i}\mathcal{J}_{i}^{j}$
1	All	1714.01	1684.17	2435.17	5833.35
5	M-NB	475.29	475.15	680.94	1631.37
5	N-NB	498.16	504.96	697.14	1700.26
5	M-WS	484.27	493.34	685.54	1663.14
15	M-NB	450.78	449.78	648.36	1548.92
15	N-NB	498.16	504.96	697.14	1700.26

TABLE II: Player and aggregate costs at initialization, iteration

5

, and the final iteration of each method. Bold entries indicate the lowest cost within each comparison group.

Multi-Agent Control

Multi-agent control coordinates interacting autonomous agents to achieve shared goals while respecting safety, communication, and resource constraints.

See all 108 papers

A Forward-Inverse Dynamic Game Framework for Enhanced Multi-Agent Trajectory Planning
Aug 3, 2026Tianle Liu, Youcheng Niu, Jing Zeng +2KL-Regularized RL Game Theory
College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China
Evolutionary Stability Does Not Guarantee Learning Accessibility: A Multi-Agent Reinforcement Learning Perspective on Cooperation Emergence
Sep 23, 2026Yijie WangEmergent Behavior in Multi-Agent Systems Cooperative MARL
Liupanshui Normal University Liupanshui, Guizhou, China
Interleaved Information Structures in Dynamic Games: A General Framework with Application to the Linear-Quadratic Case
Mar 19, 2026Janani S K, Kushagra Gupta, Ufuk Topcu +1Game Theory Nash Equilibrium Computation
Department of Engineering Design, Indian Institute of Technology Madras, Chennai, India · Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX 78712, USA · Department of Aerospace Engineering and the Oden Institute for Computational Engineering and Sciences, The University of Texas at Austin, Austin, TX 78712, USA

Aug 3, 2026J/K move · Enter open · S save

A Forward-Inverse Dynamic Game Framework for Enhanced Multi-Agent Trajectory Planning

Tianle Liu, Youcheng Niu, Jing Zeng, Shuo Li+1

College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China

KL-Regularized RL Game Theory Multi-Robot Coordination Multi-Agent Planning

Open paper AskPDF arXiv

This paper studies feedback Nash equilibrium (FBNE) seeking for multi-agent trajectory planning in nonlinear dynamical systems with unknown agents' objectives and state-dependent inter-agent coupling. While dynamic game theory provides a principled framework for such problems, existing approaches typically assume fully rational agents with known objectives or rely on fixed regularization, limiting their ability to capture bounded rationality and spatially varying interaction intensity in safety-critical settings. To this end, we propose a KL-regularized dynamic game with a state-dependent weight that adaptively balances optimality and behavioral priors. To infer unknown cost parameters from demonstrated behaviors, we develop a context-aware inverse game module based on maximum-entropy inverse reinforcement learning with physics-informed regularization, ensuring structural consistency with the forward game. We establish per-iteration well-posedness of the regularized local game and show that the adaptive weighting function remains Lipschitz continuous under bounded nominal-trajectory updates. Numerical simulations and multi-robot experiments on cooperative navigation and merging scenarios validate the effectiveness of the proposed framework.

Abstract

Multi-Agent Control

Explore similar work

A Forward-Inverse Dynamic Game Framework for Enhanced Multi-Agent Trajectory Planning

Evolutionary Stability Does Not Guarantee Learning Accessibility: A Multi-Agent Reinforcement Learning Perspective on Cooperation Emergence

Interleaved Information Structures in Dynamic Games: A General Framework with Application to the Linear-Quadratic Case