PlayGen-MoG: Framework for Diverse Multi-Agent Play Generation via Mixture-of-Gaussians Trajectory Prediction

Kevin Song

cs.CVApr 2, 2026

PlayGen-MoG: Framework for Diverse Multi-Agent Play Generation via Mixture-of-Gaussians Trajectory Prediction

Authors: Kevin Song

Organizations: Amazon Web Services, Seattle, Washington

Abstract

Multi-agent trajectory generation in team sports requires models that capture both the diversity of possible plays and realistic spatial coordination between players on plays. Standard generative approaches such as Conditional Variational Autoencoders (CVAE) and diffusion models struggle with this task, exhibiting posterior collapse or convergence to the dataset mean. Moreover, most trajectory prediction methods operate in a forecasting regime that requires multiple frames of observed history, limiting their use for play design where only the initial formation is available. We present PlayGen-MoG, an extensible framework for formation-conditioned play generation that addresses these challenges through three design choices: 1/ a Mixture-of-Gaussians (MoG) output head with shared mixture weights across all agents, where a single set of weights selects a play scenario that couples all players' trajectories, 2/ relative spatial attention that encodes pairwise player positions and distances as learned attention biases, and 3/ non-autoregressive prediction of absolute displacements from the initial formation, eliminating cumulative error drift and removing the dependence on observed trajectory history, enabling realistic play generation from a single static formation alone. On American football tracking data, PlayGen-MoG achieves 1.68 yard ADE and 3.98 yard FDE while maintaining full utilization of all 8 mixture components with entropy of 2.06 out of 2.08, and qualitatively confirming diverse generation without mode collapse.

Figures & tables

Figure 1: PlayGen-MoG training and generation overview (A) Model architecture. Initial formation and role IDs are encoded by a full-attention formation encoder. The input projection maps formation (replicated across all $T{-}1$ frames) and sinusoidal step embeddings to hidden representations. A stack of $L$ SRTE blocks applies relative spatial attention with pairwise distance biases, followed by cross-attention to the formation embeddings (Q = agents, K/V = formation). Bidirectional temporal attention pools agent features per frame, applies full self-attention across frames, and broadcasts the temporal context back to each agent. The MoG output head produces shared mixture weights $\boldsymbol{\pi}$ , per-agent displacement means $\boldsymbol{\mu}$ , and Cholesky-parameterized covariances $\mathbf{L}$ , which are used to generate the final trajectory. (B) Play Generation . The model performs a single forward pass with formation replicated across $T{-}1$ frames. A global mixture component $k$ is selected by sampling from the time-averaged weights $\bar{\boldsymbol{\pi}}$ . Positions are reconstructed as absolute displacements from formation: $\mathbf{x}_{i}^{(t)}=\mathbf{f}_{i}+\boldsymbol{\mu}_{ik}^{(t)}+\mathbf{L}_{ik}^{(t)}\boldsymbol{\epsilon}$ , eliminating cumulative drift.

Figure 2: Formation-conditioned play generation at temperature 1.0 across three personnel groupings. Each row shows a different formation type: 3WR/1TE/1RB tight (top), 3WR/1TE/1RB spread (middle), and 2WR/2TE/1RB 12-personnel (bottom). The leftmost column shows ground truth; columns 2–4 show three independent samples from PlayGen-MoG. Players are colored by position group (see legend). All panels in each row share the same axis scale. The model generates diverse play outcomes that match the scale and structure of real American football plays.

Variant	ADE	FDE	Ent.
AR, frame $\Delta$	40.83	43.79	1.70
AR, sched. samp.	41.53	44.61	1.70
Non-AR, frame $\Delta$	47.51	55.00	1.70
Non-AR, abs. disp.	1.68	3.98	2.06
Non-AR, abs. disp. + $\lambda_{\text{ADE}}$	1.67	4.01	2.03

Table 1: Ablation results on American football tracking data (9,934 plays). ADE/FDE in yards via generation at temperature 0.8. Entropy measures mixture utilization (max

\ln 8\approx 2.08

$M$	ADE	FDE	Ent. / $\ln M$	APD
1	1.60	3.82	—	0.83
2	1.59	3.78	0.68 / 0.69	0.86
4	1.65	3.84	1.37 / 1.39	1.03
8	1.66	3.91	2.06 / 2.08	1.13
16	1.66	3.96	2.74 / 2.77	1.31

Table 2: Effect of varying the number of mixture components

M

. All variants use the non-AR absolute-displacement architecture with

\lambda_{\text{ADE}}{=}1.0

. Max entropy is

\ln M

. APD (Average Pairwise Distance) measures generative diversity across

K{=}10

samples per formation.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Method	ADE $\downarrow$	FDE $\downarrow$	APD $\uparrow$	Params
CVAE (MLP decoder)	2.88	6.56	0.10	11M
LED $\dagger$	1.34	3.14	—	3.4M
PlayGen-MoG (Ours)	1.68	3.98	1.13	1.3M

Appendix

Table 1: Comparison with generative baselines on American football tracking data (9,934 plays). ADE/FDE in yards. APD measures diversity across

K{=}10

samples per formation.

\dagger

LED reports best-of-6 proposals (standard LED evaluation); single-proposal ADE is substantially higher.

Figure 1: Qualitative comparison of generative baselines. Each row shows three independent samples from the same formation. Top (CVAE): Posterior collapse—all samples are nearly identical despite different latent draws. Middle (LED): Diffusion produces high-variance, spatially incoherent trajectories spanning the full field. Bottom (PlayGen-MoG): Each sample represents a distinct, realistic play concept with coordinated player motion.

$T$	Duration	ADE $\downarrow$	FDE $\downarrow$	APD $\uparrow$
10	1.0 s	0.22	0.51	0.08
20	2.0 s	0.63	1.50	0.31
30	3.0 s	0.96	2.28	0.66
40	4.0 s	1.13	2.66	1.17
49	4.9 s	1.19	2.80	1.69

Appendix

Table 2: Generation quality at different prediction horizons.

T

is the number of predicted frames at 10 fps. ADE/FDE in yards. APD measures diversity across

K{=}10

samples.

Figure 2: A single generated play shown at increasing prediction horizons. Circles mark starting positions; diamonds mark endpoints at each horizon. Trajectory width tapers to indicate direction of movement. Routes become progressively distinguishable as the horizon extends.

Trajectory Generation

Trajectory generation creates plausible, goal-directed paths and motion sequences for people, vehicles, and robots.

See all 176 papers

GenTac: Generative Modeling and Forecasting of Soccer Tactics
Apr 13, 2026Jiayuan Rao, Tianlin Gui, Haoning Wu +2Soccer Analytics Multi-Agent Trajectory Prediction
Shanghai Jiao Tong University, Shanghai, China · Nanjing University, Nanjing, China
TacticGen: Grounding Adaptable and Scalable Generation of Football Tactics
Apr 20, 2026Sheng Xu, Guiliang Liu, Tarak Kharrat +12Conditional Generative Modeling Soccer Analytics
School of Data Science, The Chinese University of Hong Kong, Shenzhen, China · Real Analytics, London, UK · Birmingham City Football Club, Birmingham, UK +3
Diverse Yet Consistent: Context-Guided Diffusion with Energy-Based Joint Refinement for Multi-Agent Motion Prediction
May 21, 2026Lei Chu, Yuhuan ZhaoMulti-Agent Trajectory Prediction Human Motion Generation
University of Southern California, Los Angeles, CA, USA

Apr 13, 2026J/K move · Enter open · S save

GenTac: Generative Modeling and Forecasting of Soccer Tactics

Jiayuan Rao, Tianlin Gui, Haoning Wu, Yanfeng Wang+1

Shanghai Jiao Tong University, Shanghai, China · Nanjing University, Nanjing, China

Soccer Analytics Multi-Agent Trajectory Prediction Diffusion Models for Motion Generation

Open paper AskPDF arXiv

Modeling open-play soccer tactics is a formidable challenge due to the stochastic, multi-agent nature of the game. Existing computational approaches typically produce single, deterministic trajectory forecasts or focus on highly structured set-pieces, fundamentally failing to capture the inherent variance and branching possibilities of real-world match evolution. Here, we introduce GenTac, a diffusion-based generative framework that conceptualizes soccer tactics as a stochastic process over continuous multi-player trajectories and discrete semantic events. By learning the underlying distribution of player movements from historical tracking data, GenTac samples diverse, plausible, long-horizon future trajectories. The framework supports rich contextual conditioning, including opponent behavior, specific team or league playing styles, and strategic objectives, while grounding continuous spatial dynamics into a 15-class tactical event space. Extensive evaluations on our proposed benchmark, TacBench, demonstrate four key capabilities: (1) GenTac achieves high geometric accuracy while strictly preserving the collective structural consistency of the team; (2) it accurately simulates stylistic nuances, distinguishing between specific teams (e.g., Auckland FC) and leagues (e.g., A-League versus German leagues); (3) it enables controllable counterfactual simulations, demonstrably altering spatial control and expected threat metrics based on offensive or defensive guidance; and (4) it reliably anticipates future tactical outcomes directly from generated rollouts. Finally, we demonstrate that GenTac can be successfully trained to generalize to other dynamic team sports, including basketball, American football, and ice hockey.

Abstract

Trajectory Generation

Explore similar work

GenTac: Generative Modeling and Forecasting of Soccer Tactics

TacticGen: Grounding Adaptable and Scalable Generation of Football Tactics

Diverse Yet Consistent: Context-Guided Diffusion with Energy-Based Joint Refinement for Multi-Agent Motion Prediction