Organizations: Graduate School of Informatics, Nagoya University, Furo-cho, Chikusa, Nagoya, Aichi, Japan · Institute of Computer Science, Leuphana University of Lüneburg, Lüneburg, Germany · Information Technology Center, Nagoya University, Furo-cho, Chikusa, Nagoya, Aichi, Japan
Football action valuation estimates players' contributions to future scoring. Existing multi-agent deep reinforcement learning (MADRL) models use 2D positions and velocities, omitting posture information relevant to imminent actions. We investigate whether adding 3D pose to a SARSA-based model improves observed-action values relative to attack returns: goals, conceded goals or final expected possession value. Using UEFA Champions League 2022/23 event, tracking and 3D pose data, we compare six representations (no pose, raw joints, torso and head orientation, and spatial-temporal graph convolutional network (ST-GCN) embeddings) across five learning settings and three seeds. The settings vary temporal-difference targets and action-prediction loss, with Monte Carlo return regression as a reference. Across 50 on-ball and off-ball comparisons, pose robustly improved return prediction in only three, all using torso and head orientation, while worsening it in twelve, mostly with ST-GCN. Training choices had greater effects: removing the action-prediction loss raised on-ball skill of position-only values from 0.003 to 0.040 and their correlation with expected goals from 0.23 to 0.60; the improvement in return prediction persisted at matched training length. Separately, pose improved prediction of the on-ball action 0.2 s ahead (macro AUC 0.914 to 0.934 with raw joints), while the passer's pose during execution improved pass-success prediction (AUC 0.844 to 0.856). Within this training protocol, pose informed imminent actions but rarely improved observed-action values.
Figures & tables
Fig. 1: Overview. The 92-dimensional position–velocity state of the ball and all players is concatenated with one of the pose representations of the target player (dimensions on the right): none, raw joints, torso (and head) orientation features, or an ST-GCN embedding with or without torso features. In the action-value model (top), the fused state is projected to 64 dimensions, encoded over the attacking sequence by a GRU, and mapped to action values; unavailable actions are masked according to whether the observed action of the step is an on-ball action. The same state and pose features, except the ST-GCN embeddings, are given to gradient-boosted trees for imminent events (bottom): the on-ball action 0.2 s ahead and, from the pose of the pass step, the success of a pass.
On-ball steps
Off-ball steps
Shots
Condition
Skill
Δ [95% CI]
Skill
Δ [95% CI]
ρ(Q,xG) [95% CI]
Terminal TD, λAS=0.01 (prior model)
none (baseline)
0.003
–
−0.057
–
0.23 [0.15, 0.32]
raw
0.000
+0.40 [ −1.21 , +1.94 ]
−0.017
−3.99 [ −7.01 , −0.91 ]
0.26 [0.16, 0.36]
torso
0.005
−0.89 [ −1.98 , +0.19 ]
−0.085
+5.93 [ +2.90 , +8.99 ]
0.25 [0.16, 0.35]
torso+head
0.010
−2.71† [ −4.04 , −1.49 ]
−0.046
−0.50 [ −3.00 , +2.40 ]
0.22 [0.14, 0.30]
TABLE I: Validity of the action values on the test matches under five learning settings. Skill: 1−MSE/Var(G) , mean over three seeds; Δ : MSE of the seed-averaged values minus that of the baseline of the same setting on the same steps ( ×10−4 , 95% CI from 5,000 bootstrap resamples of test matches); ρ(Q,xG) : Spearman correlation of the seed-averaged shot values with xG on the 250 test shots [95% CI]. Var(G)=3.507×10−2 (on-ball) and 3.319×10−2 (off-ball). Negative Δ means closer to the realised return; bold: best skill and ρ within each setting (ties at the shown precision all in bold); † : robust difference (CI excludes zero and the same sign for all three seeds), in either direction. Package TD with λAS=0.01 did not learn and is not shown.
Fig. 2: Difference in MSE between each pose condition and the baseline of the same setting on the test matches ( ×10−4 , values averaged over three seeds), with 95% CIs from 5,000 bootstrap resamples of test matches. Filled markers: robust differences (CI excludes zero and the same sign for all three seeds). The horizontal scales differ between panels. Package TD with λAS=0.01 did not learn and is not shown.
Features
Log loss
Macro AUC
MCC
state only
0.304
0.914
0.424
+torso
−0.013
+0.009 (5/5)
+0.013 (5/5)
[ −0.017 , −0.009 ]
[ +0.006 , +0.013 ]
[ −0.001 , +0.027 ]
+torso+head
−0.013
+0.010 (5/5)
+0.014 (5/5)
[ −0.016 , −0.009 ]
[ +0.007 , +0.012 ]
[ +0.003 , +0.024 ]
+raw
−0.038
+0.020 (5/5)
+0.089 (5/5)
TABLE II: Prediction of the on-ball action 0.2 s ahead on the test matches (6,388 steps; pass 87%, cross 5%, dribble 4%, shot 4%, through pass 0.3%) by gradient-boosted trees on the state and pose features of the previous step. Differences from the state-only model with 95% CIs over test matches; in parentheses, the number of the five model seeds with the same sign; bold: best value of each column.
Fig. 3: Qualitative cases selected by the rule in Section V-C (terminal TD, λAS=0.01 , torso+head against none, values averaged over three seeds): (a)–(c) success cases with the smallest Δt and (d), (e) failure cases with the largest Δt . Left: players and ball, with the attacking team playing from left to right; middle: front view of the target’s pose averaged over the frames of the step; right: values of the available actions, with the observed action in red, and the realised return Gt . The share of test steps with Δt<0 was 0.61 on-ball and 0.47 off-ball.
Fig. 4: Validation losses of the baseline under terminal TD per epoch (logarithmic scale). (a) TD loss with λAS=0.01 and 0 ; (b) action-prediction loss with λAS=0.01 . With λAS=0.01 , early stopping on the total validation loss ended the runs after 83 to 134 epochs, while the TD loss stayed above its first-epoch value; with λAS=0 , all runs stopped after six epochs, because the minimum improvement of early stopping exceeded the whole decrease of the loss after the first epoch (Section IV-F ). The levels of the two TD losses are not comparable, because each is computed from the model’s own values.
λAS=0.01
λAS=0.01 , 6 ep.
λAS=0
Condition
AUC
MCC
AUC
MCC
AUC
MCC
none
0.735
0.131
0.853
0.270
0.871
0.266
raw
0.792†
0.187
0.818†
0.234†
0.880
0.296†
torso
0.726
0.108
0.840
0.239†
0.873
0.265
torso+head
0.746
0.141
0.830
0.226
0.866
0.283†
ST-GCN
0.800†
0.181†
0.800†
0.208†
0.846†
0.258
TABLE III: Action information in the values (probe on the five on-ball action values, fitted on the training matches, tested on the 6,801 on-ball test steps): macro AUC and MCC for terminal TD with λAS=0.01 trained until early stopping (47–134 epochs) or for six epochs, and with λAS=0 (six epochs). Bold: best value of each column; † : robust difference from the baseline of the same column, in either direction.
Condition
Input dim.
Parameters
none
92
31,952
raw
179
37,520
torso
98
32,336
torso+head
106
32,848
ST-GCN
220
119,504
ST-GCN+torso
226
119,888
TABLE IV: Input dimension and number of trainable parameters of the action-value model for each pose condition (input layer, GRU and output layer with 16 units for the 15 actions and the padding class, plus the ST-GCN encoder where used).
Graduate School of Informatics, Nagoya University, Nagoya, Japan. · Graduate School of Arts and Sciences, The University of Tokyo, Tokyo, Japan. · Department of Economics, The Hong Kong University of Science and Technology, Hong Kong, China. +1