Organizations: Graduate School of Informatics, Nagoya University, Furo-cho, Chikusa, Nagoya, Aichi, Japan · Institute of Computer Science, Leuphana University of Lüneburg, Lüneburg, Germany · Information Technology Center, Nagoya University, Furo-cho, Chikusa, Nagoya, Aichi, Japan
Football action valuation estimates players' contributions to future scoring. Existing multi-agent deep reinforcement learning (MADRL) models use 2D positions and velocities, omitting posture information relevant to imminent actions. We investigate whether adding 3D pose to a SARSA-based model improves observed-action values relative to attack returns: goals, conceded goals or final expected possession value. Using UEFA Champions League 2022/23 event, tracking and 3D pose data, we compare six representations (no pose, raw joints, torso and head orientation, and spatial-temporal graph convolutional network (ST-GCN) embeddings) across five learning settings and three seeds. The settings vary temporal-difference targets and action-prediction loss, with Monte Carlo return regression as a reference. Across 50 on-ball and off-ball comparisons, pose robustly improved return prediction in only three, all using torso and head orientation, while worsening it in twelve, mostly with ST-GCN. Training choices had greater effects: removing the action-prediction loss raised on-ball skill of position-only values from 0.003 to 0.040 and their correlation with expected goals from 0.23 to 0.60; the improvement in return prediction persisted at matched training length. Separately, pose improved prediction of the on-ball action 0.2 s ahead (macro AUC 0.914 to 0.934 with raw joints), while the passer's pose during execution improved pass-success prediction (AUC 0.844 to 0.856). Within this training protocol, pose informed imminent actions but rarely improved observed-action values.
Figures & tables
Fig. 1: Overview. The 92-dimensional position–velocity state of the ball and all players is concatenated with one of the pose representations of the target player (dimensions on the right): none, raw joints, torso (and head) orientation features, or an ST-GCN embedding with or without torso features. In the action-value model (top), the fused state is projected to 64 dimensions, encoded over the attacking sequence by a GRU, and mapped to action values; unavailable actions are masked according to whether the observed action of the step is an on-ball action. The same state and pose features, except the ST-GCN embeddings, are given to gradient-boosted trees for imminent events (bottom): the on-ball action 0.2 s ahead and, from the pose of the pass step, the success of a pass.
On-ball steps
Off-ball steps
Shots
Condition
Skill
Δ [95% CI]
Skill
Δ [95% CI]
ρ(Q,xG) [95% CI]
Terminal TD, λAS=0.01 (prior model)
none (baseline)
0.003
–
−0.057
–
0.23 [0.15, 0.32]
raw
0.000
+0.40 [ −1.21 , +1.94 ]
−0.017
−3.99 [ −7.01 , −0.91 ]
0.26 [0.16, 0.36]
torso
0.005
−0.89 [ −1.98 , +0.19 ]
−0.085
+5.93 [ +2.90 , +8.99 ]
0.25 [0.16, 0.35]
torso+head
0.010
−2.71† [ −4.04 , −1.49 ]
−0.046
−0.50 [ −3.00 , +2.40 ]
0.22 [0.14, 0.30]
TABLE I: Validity of the action values on the test matches under five learning settings. Skill: 1−MSE/Var(G) , mean over three seeds; Δ : MSE of the seed-averaged values minus that of the baseline of the same setting on the same steps ( ×10−4 , 95% CI from 5,000 bootstrap resamples of test matches); ρ(Q,xG) : Spearman correlation of the seed-averaged shot values with xG on the 250 test shots [95% CI]. Var(G)=3.507×10−2 (on-ball) and 3.319×10−2 (off-ball). Negative Δ means closer to the realised return; bold: best skill and ρ within each setting (ties at the shown precision all in bold); † : robust difference (CI excludes zero and the same sign for all three seeds), in either direction. Package TD with λAS=0.01 did not learn and is not shown.
Fig. 2: Difference in MSE between each pose condition and the baseline of the same setting on the test matches ( ×10−4 , values averaged over three seeds), with 95% CIs from 5,000 bootstrap resamples of test matches. Filled markers: robust differences (CI excludes zero and the same sign for all three seeds). The horizontal scales differ between panels. Package TD with λAS=0.01 did not learn and is not shown.
Features
Log loss
Macro AUC
MCC
state only
0.304
0.914
0.424
+torso
−0.013
+0.009 (5/5)
+0.013 (5/5)
[ −0.017 , −0.009 ]
[ +0.006 , +0.013 ]
[ −0.001 , +0.027 ]
+torso+head
−0.013
+0.010 (5/5)
+0.014 (5/5)
[ −0.016 , −0.009 ]
[ +0.007 , +0.012 ]
[ +0.003 , +0.024 ]
+raw
−0.038
+0.020 (5/5)
+0.089 (5/5)
TABLE II: Prediction of the on-ball action 0.2 s ahead on the test matches (6,388 steps; pass 87%, cross 5%, dribble 4%, shot 4%, through pass 0.3%) by gradient-boosted trees on the state and pose features of the previous step. Differences from the state-only model with 95% CIs over test matches; in parentheses, the number of the five model seeds with the same sign; bold: best value of each column.
Fig. 3: Qualitative cases selected by the rule in Section V-C (terminal TD, λAS=0.01 , torso+head against none, values averaged over three seeds): (a)–(c) success cases with the smallest Δt and (d), (e) failure cases with the largest Δt . Left: players and ball, with the attacking team playing from left to right; middle: front view of the target’s pose averaged over the frames of the step; right: values of the available actions, with the observed action in red, and the realised return Gt . The share of test steps with Δt<0 was 0.61 on-ball and 0.47 off-ball.
Fig. 4: Validation losses of the baseline under terminal TD per epoch (logarithmic scale). (a) TD loss with λAS=0.01 and 0 ; (b) action-prediction loss with λAS=0.01 . With λAS=0.01 , early stopping on the total validation loss ended the runs after 83 to 134 epochs, while the TD loss stayed above its first-epoch value; with λAS=0 , all runs stopped after six epochs, because the minimum improvement of early stopping exceeded the whole decrease of the loss after the first epoch (Section IV-F ). The levels of the two TD losses are not comparable, because each is computed from the model’s own values.
λAS=0.01
λAS=0.01 , 6 ep.
λAS=0
Condition
AUC
MCC
AUC
MCC
AUC
MCC
none
0.735
0.131
0.853
0.270
0.871
0.266
raw
0.792†
0.187
0.818†
0.234†
0.880
0.296†
torso
0.726
0.108
0.840
0.239†
0.873
0.265
torso+head
0.746
0.141
0.830
0.226
0.866
0.283†
ST-GCN
0.800†
0.181†
0.800†
0.208†
0.846†
0.258
TABLE III: Action information in the values (probe on the five on-ball action values, fitted on the training matches, tested on the 6,801 on-ball test steps): macro AUC and MCC for terminal TD with λAS=0.01 trained until early stopping (47–134 epochs) or for six epochs, and with λAS=0 (six epochs). Bold: best value of each column; † : robust difference from the baseline of the same column, in either direction.
Condition
Input dim.
Parameters
none
92
31,952
raw
179
37,520
torso
98
32,336
torso+head
106
32,848
ST-GCN
220
119,504
ST-GCN+torso
226
119,888
TABLE IV: Input dimension and number of trainable parameters of the action-value model for each pose condition (input layer, GRU and output layer with 16 units for the 15 actions and the padding class, plus the ST-GCN encoder where used).
Valuing player actions in football, a complex multi-agent system, requires accounting for strategic interactions among 22 players, including off-ball movements and defensive positioning. Existing reinforcement-learning-based valuation methods commonly incorporate other players through the observed state while retaining independent-agent value formulations. To our knowledge, we introduce the first Markov perfect equilibrium (MPE)-inspired framework for valuing the actions of all 22 players using real football tracking and event data, formulating each possession as a finite-horizon multi-agent dynamic game. To improve interpretability, the framework uses Expandable Decision-Making States (EDMS) and decomposes each Q-value into a successor-feature basis and a linear reward-weight vector. The value basis is estimated through a two-stage procedure combining linear temporal-difference initialization with nonlinear iterative refinement from observed state-action transitions. Using tracking and event data from 95 J1 League matches, we compare the proposed formulation with an independent reinforcement learning baseline under both Position and Velocity States (PVS) and EDMS representations. With EDMS held fixed, the independent baseline assigns the highest value to forward movement in 99.21% of evaluated off-ball states, whereas the most frequent direction under the proposed formulation accounts for 17.63%. Qualitative analyses illustrate context-dependent off-ball valuations and highlight limitations of the current state representation for defensive positioning. Overall, the framework provides an MPE-inspired approach to interpretable action valuation in football, enabling analysis under alternative tactical reward weights.
Kenjiro Ide, Taiga Someya, Kohei Kawaguchi +1
Graduate School of Informatics, Nagoya University, Nagoya, Japan. · Graduate School of Arts and Sciences, The University of Tokyo, Tokyo, Japan. · Department of Economics, The Hong Kong University of Science and Technology, Hong Kong, China. +1
Forecasting future human motion requires jointly reasoning about an agent's global trajectory and 3D body pose, which are tightly coupled, particularly in team sports where multiple interacting players introduce complex social dynamics. We exploit a socio-temporal encoder that fuses trajectory and pose at the input level and models them jointly through shared factorized attention. The model is further enhanced by a per-joint loss reweighting scheme that emphasizes dynamic joints and prevents degenerate solutions, and a frequency-domain trajectory smoothing term that improves trajectory prediction while indirectly benefiting pose forecasting. To ensure fair comparison with prior joint forecasting methods, we identify and correct evaluation biases arising from overlapping sequence construction, adopting instead a non-overlapping protocol and a deterministic, oracle-free strategy for multi-future prediction. Our approach improves over prior joint forecasting methods on both WorldPose and 3DPW, with the largest gains on pose accuracy, demonstrating that the proposed joint formulation generalizes beyond the densely interactive scenarios that motivate it.
Marc Peral, Guillem Capellera, Luis Ferraz +4
Institut de Rob`otica i Inform`atica Industrial, CSIC-UPC · Kognia Sports Intelligence · IT University of Copenhagen
We recast pass evaluation in football (soccer) as a Monte Carlo Tree Search (MCTS)-like evaluation problem whose components mostly exist in the literature under different names: a value model (possession value), a world model (multi-agent trajectories with ball interactions), and a policy over counterfactual actions (sampling pass variants with noise). Building on the first public high-fidelity tracking dataset with 3D ball trajectories from the Bundesliga, we introduce Monte Carlo Pass Search (MCPS), which infers kick parameters for each observed pass, samples execution variants and option variants, rolls each candidate forward with a ball-conditioned world model until the next ball interaction, and scores outcomes with a learned value model to obtain a distribution over gained value. This distribution enables distribution-aware attribution with two complementary execution-surplus scores used for analysis and ranking: mean-based and percentile-based scores. To make the world model sample-efficient under limited public data, we adapt a discrete-token, autoregressive trajectory generator from autonomous driving (SMART) and show it yields strong best-of-20 forecasting accuracy compared to baselines, while supporting fully hypothetical rollouts for downstream evaluation. We have released model checkpoints and code.