Opinion-Guided Layered Strategies for Decentralized Coordination

Shuhao Qi; Zhiyong Sun; Siep Weiland; Sofie Haesaert

cs.MAAug 22, 2026

Opinion-Guided Layered Strategies for Decentralized Coordination

Authors: Shuhao Qi, Zhiyong Sun, Siep Weiland, Sofie Haesaert

Organizations: Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands · College of Engineering, Peking University, Beijing, China

Abstract

Autonomous agents increasingly interact with other independent agents, and such interactions typically admit multiple joint behaviors. When two agents prefer different ones, their independent strategies may be mutually incompatible and fail to reach a coordinated outcome; when they are identical, neither can differentiate its role when needed. Ideally, an agent should coordinate with any agent it encounters, regardless of which admissible joint behavior that agent aims to realize. We therefore propose a new form of strategy, the opinion-guided strategy, which keeps all the admissible joint behaviors available and postpones the selection to execution time, when the other agent's behavior reveals which one to realize. To realize this, nonlinear opinion dynamics are leveraged in a layered realization to guide the agent to a common admissible joint behavior in response to the other agent's evolving behavior, even without communication. We formally establish the conditions under which the strategy remains robust to every preference the other agent may hold. This robustness has an important implication: two agents running identical strategies can break symmetry when needed, a capability that conventional strategies lack. Three case studies across different applications show that the opinion-guided strategy coordinates with every randomly encountered agent, as long as it is willing to realize one of the admissible joint behaviors. One of them corresponds to a general-sum game: unlike conventional approaches devoted to finding a unique Nash equilibrium in advance, the opinion-guided strategy keeps every equilibrium open and guarantees the agents reach one, decided by their runtime interaction.

Figures & tables

Fig. 1: A typical encounter scenario in dense residential regions.

Fig. 2: Sketch of the running example, a two-robot charging-bay selection task that instantiates the anti-coordination game. The robots $R_{1}$ (blue) and $R_{2}$ (red) start together in the middle zone $p_{2}$ , while a charging bay is available in each of the side zones $p_{1}$ and $p_{3}$ . The dashed arrows mark the two candidate moves of each robot (left toward $p_{1}$ or right toward $p_{3}$ ), and the task succeeds only when the two robots commit to distinct bays, so that both can recharge simultaneously.

Panel of Figure 3 — Fig. 3: Two resolution behaviors randomly generated in a two-airplane encounter. The upper panels show the airplanes’ trajectories, while the bottom panels depict the evolution of the opinion states.

Fig. 5: Representative geometries of a permissive joint strategy,

\pi_{1}(h)

through

\pi_{5}(h)

, illustrating cases in which both, one, or neither agent has a non-empty robust set. In every panel the horizontal axis is

A_{1}

(Agent 1, blue) and the vertical axis

A_{2}

(Agent 2, red). On the axes, thin light lines mark the rational sets

A_{i}^{\mathsf{rat}}(h)

and thick dark lines the robust sets

A_{i}^{\mathsf{rob}}(h;\Sigma_{j}^{\mathsf{rat}})

, and the dotted gray box is their product

A_{1}^{\mathsf{rat}}(h)\!\times\!A_{2}^{\mathsf{rat}}(h)

. The green region is the permissive set

\pi(h)

; blue and red hatching mark the joint actions in which Agent 1 and Agent 2, respectively, commit to their robust set.

Panel of Figure 5 — Fig. 6: Four decision structures for a two-agent game. (a) agents act concurrently with independent strategies $\sigma_{1},\sigma_{2}$ . (b) a reactive strategy realized sequentially, where agent 2 observes $a_{1}$ before selecting $a_{2}$ . (c) mutually reactive strategies, where each agent’s action conditions on the other’s; this is an abstract specification without a realization mechanism. (d) an opinion-guided implementation, where each agent maintains an opinion state $\tilde{z}_{i}$ paired to the other’s through $\Gamma_{i}$ , and selects its action via $\sigma^{\text{op}}_{i}(s,\tilde{z}_{i})$ .

Panel of Figure 6 — Fig. 8: Layered realization of opinion-guided strategy. Strategy layer: at each decision time $t_{k}$ , both agents query the game $\mathcal{G}$ at the current state $s$ and select opinion-guided strategies $\sigma^{\text{op}}_{1},\sigma^{\text{op}}_{2}$ whose action intents are compatible, as required by ( 12 ). Continuous opinion layer: between consecutive decisions, the coupled opinions $z_{1}$ and $z_{2}$ converge to opposite signs within the reserved window $\Delta t_{\mathrm{op}}$ . At the end of each window, the converged opinion is abstracted to its discrete sign $\tilde{z}_{i}$ and passed up to the strategy layer, which commits one of the compatible action pairs.

Fig. 9: Illustration of the opinion-guided strategy from the perspective of agent

i

i\!\in\!\{1,2\}

. The dashed line with a scissor icon denotes the partition of

A_{j}

induced by

\mathcal{O}_{j}

. In (a), only

A_{j}

is partitioned, indexed by

\tilde{z}_{j}\!\in\!\{-1,1\}

. In (b), the action sets of both agents are mutually partitioned according to their respective discrete opinion states

\tilde{z}_{1},\tilde{z}_{2}\!\in\!\mathbb{B}^{n}

Panel of Figure 8 — Fig. 10: Bifurcation diagrams of the two-agent scalar opinion dynamics ( 22 ) as the attention gain $\rho$ increases, shown for the two coupling signs $\gamma$ . (a) $\gamma\!=\!+1$ : the two stable branches emerge along $\pm[1,1]^{\top}$ , pairing same-sign opinions. (b) $\gamma\!=\!-1$ : the stable branches emerge along $\pm[1,-1]^{\top}$ , pairing opposite-sign opinions, as in the running example ( 9 ). In both panels the neutral equilibrium $\boldsymbol{z}\!=\!\mathbf{0}$ is stable for $\rho\!<\!\rho^{\star}$ and undergoes a supercritical pitchfork at the threshold $\rho^{\star}\!=\!d/(2\kappa)\!=\!0.5$ (green star); solid blue curves are the stable equilibrium branches and the dashed red curve is the unstable branch. The vertical slice $\rho\!=\!2$ carries the phase portrait, whose trajectories diverge from the unstable equilibrium (open circle) and converge to the two stable equilibria (filled circles), with the red curves marking the separatrix.

Fig. 11: Block-diagram view of the communication-free opinion-guided coordination framework. Each agent (agent 1 in blue, agent 2 in red) runs an opinion-guided strategy ( $\sigma^{\text{op}}_{i}$ ) in the strategy layer, opinion dynamics (NOD i ) in the opinion layer, and an opinion estimator ( $\Psi_{i}$ ) together with a tracking controller ( $\mu_{i}$ ) acting on the physical plant ( $f_{i}$ ) in the physical layer. The two control stacks are coupled only through the jointly observable physical state $x\!=\!(x_{1},x_{2})$ : each opinion estimator $\Psi_{i}$ infers the opponent’s opinion $\hat{z}_{j}$ from $x$ and feeds it into NOD i , so the opinion loop closes through shared observation rather than direct inter-agent communication.

Fig. 12: One-lane ramp merging against a speed-switching opponent. Top: road scene at successive time instants, where the ego (blue) merges into the lane of the fluctuating opponent (red). Middle: ego and opponent speeds, with the ego reference speed marked by the dotted line. Bottom: the ego opinion state $z$ , whose sign commits the ego to a front ( $z>0$ ) or rear ( $z<0$ ) merge.

Panel of Figure 11 — Fig. 13: Permissive and robust action sets in the joint speed space $A_{e}\!\times\!A_{o}$ , at two instants of the run in Fig. 12 . The green region is the permissive set $\pi(s)$ ; the grey diagonal band is the unsafe region where the vehicles reach the merge point closer than $d_{\text{safe}}$ . Dashed lines mark the partition thresholds $v_{o}^{\mathrm{th}}$ and $v_{e}^{\mathrm{th}}$ ; the light segments on the two axes are the agents’ rational sets $A_{e}^{\mathsf{rat}}(s)$ and $A_{o}^{\mathsf{rat}}(s)$ , while the dark segments on the ego axis are its robust action sets $A_{e}^{\mathsf{rob}}(s;\tilde{z}_{o})$ . The green dot is the executed joint speed at this state.

Fig. 14: Decentralized dual-arm manipulation guided by the opinion state. Top: translucent snapshots of the two arms (arm 1 blue, arm 2 red) at successive instants along their trajectories. (a) Reaching: the arms commit to opposite left/right halves of the table to grasp their cubes. (b)–(c) Transporting: the arms carry the cubes through the shared central space toward the baskets, additionally separating along the upper/lower axis. (d) Returning: the arms retract to their initial poses. Bottom: evolution of each arm’s two opinions, with the solid curves the left/right opinion $z_{i}[0]$ and the dashed curves the upper/lower opinion $z_{i}[1]$ .

Panel of Figure 13 — Fig. 15: Two symmetric coordination outcomes produced by the proposed framework. Despite sharing an identical strategy, the vehicles spontaneously assume distinct roles via opinion-dynamics-driven symmetry breaking. The green lines mark the boundaries of the grid abstraction.

	Vehicle 2
Vehicle 1	NE-A	NE-B	Rule-based	Opinion-guided
NE-A	100	0	52	100
NE-B	0	100	65	100
Rule-based	48	51	76	100
Opinion-guided	100	100	100	100

TABLE I: Success rate (%) between different strategy types, over

100

independent runs per pairing.

Panel of Figure 15 — Fig. 16: Large-scale validation in a constrained environment: (a) overall trajectory of the fleet; (b)–(c) zoomed-in views of two representative interaction segments.

Multi-Agent Coordination

Multi-agent coordination studies how autonomous agents align actions, share information, and divide work to achieve individual or collective goals.

See all 516 papers

Apr 20, 2026

ConventionPlay: Capability-Limited Training for Robust Ad-Hoc Collaboration

Ad-hoc collaboration often relies on identifying and adhering to shared conventions. However, when partners can follow multiple conventions, agents must do more than simply adapt; they must actively steer the team toward the most effective joint strategy. We present ConventionPlay, a reinforcement learning-based approach that extends cognitive hierarchies to include a diverse population of adaptive followers. By training against partners with varied capability limits, our agent learns to probe its partner's repertoire, leading the team when possible and following when necessary. Our results in canonical coordination tasks show that ConventionPlay achieves superior coordination efficiency, particularly in settings where conventions have differentiated payoffs.

Abhishek Sriraman, Eleni Vasilaki, Robert Loftin

University of Sheffield · Sheffield, United Kingdom

Multi-Agent Coordination Multi-Agent Collaboration Ad Hoc Teamwork Cooperative MARL

Ask PDF

Apr 10, 2026

Multi-Agent Decision-Focused Learning via Value-Aware Sequential Communication

Multi-agent coordination under partial observability requires agents to share complementary private information. While recent methods optimize messages for intermediate objectives (e.g., reconstruction accuracy or mutual information), rather than decision quality, we introduce SeqComm-DFL, unifying the sequential communication with decision-focused learning for task performance. Our approach features value-aware message generation with sequential Stackelberg conditioning: messages maximize receiver decision quality and are generated in priority order, with agents conditioning on their predecessors. The guidance potential determined by their prosocial ordering. We extend Optimal Model Design to communication-augmented world models with QMIX factorization, enabling efficient end-to-end training via implicit differentiation. We prove information-theoretic bounds showing that communication value scales with coordination gaps and establish

\mathcal{O}(1/\sqrt{T})

convergence for the bilevel optimization, where

T

denotes the number of training iterations. On collaborative healthcare and StarCraft Multi-Agent Challenge (SMAC) benchmarks, SeqComm-DFL achieves four to six times higher cumulative rewards and over 13% win rate improvements, enabling coordination strategies inaccessible under information asymmetry.

Benjamin Amoh, Geoffrey Parker, Wesley Marrero

Thayer School of Engineering, Dartmouth College

Multi-Agent Coordination Multi-Agent Communication Cooperative MARL Decision-Focused Learning Partially Observable RL

Ask PDF

Feb 26, 2026

Robust Information Design for Multi-Agent Systems with Complementarities: Smallest-Equilibrium Threshold Policies

We study information design in multi-agent systems (MAS) with binary actions and strategic complementarities, where an external designer influences behavior only through signals. Agents play the smallest-equilibrium of the induced Bayesian game, reflecting conservative, coordination-averse behavior typical in distributed systems. We show that when utilities admit a convex potential and welfare is convex, the robustly implementable optimum has a remarkably simple form: perfect coordination at each state: either everyone acts or no one does. We provide a constructive threshold rule: compute a one-dimensional score for each state, sort states, and pick a single threshold (with a knife-edge lottery for at most one state). This rule is an explicit optimal vertex of a linear program (LP) characterized by feasibility and sequential obedience constraints. Empirically, in both vaccination and technology-adoption domains, our constructive policy matches LP optima, scales as

O(|Θ|\log|Θ|)

, and avoids the inflated welfare predicted by obedience-only designs that assume the designer can dictate the (best) equilibrium. The result is a general, scalable recipe for robust coordination in MAS with complementarities.

Farzaneh Farhadi, Maria Chli

Aston University, Birmingham, United Kingdom

Information Design Multi-Agent Coordination Game Theory Imperfect-Information Games

Ask PDF

Abstract

Multi-Agent Coordination

Explore similar work

ConventionPlay: Capability-Limited Training for Robust Ad-Hoc Collaboration

Multi-Agent Decision-Focused Learning via Value-Aware Sequential Communication

Robust Information Design for Multi-Agent Systems with Complementarities: Smallest-Equilibrium Threshold Policies