Automated design of multi-agent interactions with desirable equilibrium outcomes is inherently difficult due to the computational hardness, non-uniqueness, and instability of the resulting equilibria. In this work, we propose the use of game-agnostic differentiable equilibrium blocks (DEBs) as modules in a novel, differentiable framework to address a wide variety of incentive design problems from economics and computer science. We call this framework deep incentive design (DID). To validate our approach, we examine three diverse, challenging incentive design tasks: contract design, machine scheduling, and inverse equilibrium problems. For each task, we train a single neural network using a unified pipeline and DEB. This architecture solves the full distribution of problem instances, parameterized by a context, handling all games across a wide range of scales (from two to sixteen actions per player).
Figures & tables
Figure 1: Deep Incentive Design framework. Black (green) arrows show the forward (backward) computation.
Figure 2: A simple example of a contract design problem with action space 2×2 and 3 outcomes.
Table 1: Experimental results. We report results for our approach (DID) and with and without local polishing (+Local) to give an approximate ceiling on the optimal local performance missed by DID. We also evaluate at the equilibrium computed by the DEB and ECOS to measure discrepancies from the DEB.
Figure 3: σω is plotted in cyan, the 0.01 -ME-CCE in magenta. The red polytope corresponds to the 0.01 -CCE set. The underlying game is between a row player with actions U p and D own and a column player with actions L eft and R ight. The four vertices represent the four deterministic joint actions. Figure was produced with a smaller network trained on 2×2 games.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Inverse equilibrium training pipeline.
Figure 5: The architecture of the contract design mechanism generator. Blue boxes show the core generator and the green text shows the shapes.
Figure 6: Training pipeline for the contract design task.
Figure 7: Training pipeline for the machine scheduling problem.
Many real-world competitive systems require multiple decision-makers to act simultaneously under shared constraints, limited information, and repeated interaction, as in auctions, resource allocation, and security competition. We study multi-turn simultaneous bidding as a controlled testbed for such problems and propose DNQ, a solver-in-the-loop equilibrium supervision framework for training bidding agents. DNQ alternates between trajectory collection, critic-based payoff estimation, equilibrium computation, and policy imitation. At each visited state, a shared critic predicts either pairwise payoff matrices or an exact N-player payoff tensor, an external solver computes equilibrium strategies, and the agents are trained by minimizing the KL divergence between their masked policies and the solver-derived equilibrium targets. We focus on a scalable pairwise formulation that greatly reduces equilibrium-solving cost and training time compared with the exact formulation, while the shared critic amortizes payoff learning across agents and states. Experiments compare the pairwise and exact variants using critic loss, policy entropy, bidding resource usage, and training cost, showing that the pairwise method scales to larger numbers of agents, whereas the exact method becomes computationally impractical as the joint game grows. These results illustrate the trade-off between strategic fidelity and scalability in repeated competitive environments.
In dynamic noncooperative games, each player makes conjectures about other players' reactions before choosing a strategy. However, resulting equilibria may be multiple and do not always lead to desirable outcomes. These issues are typically addressed separately, for example, through opponent modelling and incentive design. Drawing inspiration from conjectural variations games, we propose an incentive design framework in which a coordinator first computes an equilibrium by optimizing a predefined objective function, then communicates this equilibrium as a target for the players to reach. In a centralized setting, the coordinator also optimizes the conjectures to steer the players towards the target. In decentralized settings, players independently compute conjectures and update their strategies based on individual targets. We provide a guarantee of equilibrium existence in both cases. This framework uses conjectures not only to guide the system towards desirable outcomes but also to decouple the game into independent optimization problems, enabling efficient computation and parallelization in large-scale settings. We illustrate our theoretical results on classical representative noncooperative games, demonstrating its application potential.
Francesco Morri, Hélène Le Cadre, David Salas +1
Inria, Univ. Lille CNRS, Centrale Lille · Instituto de Ciencias de la Ingeniería Universidad de O’Higgins · Laboratoire PROMESUPR 8521 Université de Perpignan, Tecnosud
In incentive design problems, a system planner seeks to steer self-interested agents toward a socially optimal Nash equilibrium by issuing incentives in the presence of information asymmetry, that is, uncertainty about the agents' cost functions. A common approach formulates the problem as a Mathematical Program with Equilibrium Constraints and optimizes incentives using hypergradients-the total derivatives of the planner's objective with respect to incentives. However, computing the hypergradient typically requires full or partial knowledge of equilibrium sensitivities to incentives, which is generally unavailable under information asymmetry. In this paper, we propose a hypergradient-free incentive law, called the social-gradient flow, for incentive design when the social cost depends on the agents' joint action. We prove that the social cost gradient is a descent direction for the planner's objective, irrespective of the agent cost landscape. In the idealized setting where equilibrium responses are observable, the social-gradient flow converges to the unique socially optimal incentive. When equilibria are not directly observable, the social-gradient flow emerges as the slow-timescale limit of a two-timescale interaction, in which agents' strategies evolve on a faster timescale. We establish that the joint strategy-incentive dynamics converge to the social optimum for any agent learning rule that asymptotically tracks the equilibrium. Finally, we validate theoretical results via numerical experiments.
Georgios Vasileiou, Lantian Zhang, Silun Zhang
Department of Mathematics at the KTH Royal Institute of Technology, Stockholm, 10044, Sweden.