Modeling Matches as Language: A Generative Transformer Approach for Counterfactual Player Valuation in Football
Organizations: University of Seoul, Seoul, Republic of Korea · Saarland University, Saarbrücken, Germany · KAIST, Daejeon, Republic of Korea · Fitogether Inc., Seoul, Republic of Korea ·
Abstract
Evaluating football player transfers is challenging because player actions depend strongly on tactical systems, teammates, and match context. Despite this complexity, recruitment decisions often rely on static statistics and subjective expert judgment, which do not fully account for these contextual factors. This limitation stems largely from the absence of counterfactual simulation mechanisms capable of predicting outcomes in hypothetical scenarios. To address these challenges, we propose ScoutGPT, a generative model that treats football match events as sequential tokens within a language modeling framework. Utilizing a NanoGPT-based Transformer architecture trained on next-token prediction, ScoutGPT learns the dynamics of match event sequences to simulate event sequences under hypothetical lineups, demonstrating superior predictive performance compared to existing baseline models. Leveraging this capability, the model employs Monte Carlo sampling to enable counterfactual simulation, allowing for the assessment of unobserved scenarios. Experiments on K League data show that simulated player transfers lead to measurable changes in offensive progression and goal probabilities, indicating that ScoutGPT captures player-specific impact beyond traditional static metrics.
Figures & tables
| Group | Symbol | Description | # Tokens |
| Context | Team IDs of the home and away teams | ||
| Role † of each player | |||
| Player ID of each player | |||
| Match state: period, minutes, home/away goals, home/away yellow and red cards | |||
| Event | Team ID of the acting team | ||
| Role † of the acting player |
| Split | Seasons | Matches | Episodes | Events | Events/ep | Players |
| Train | 2021–2023 | 1,320 | 132,315 | 3,528,635 | 26.67 | 1,090 |
| Valid | 2024 | 462 | 43,579 | 1,277,169 | 29.31 | 848 |
| Test | 2025 | 501 | 47,046 | 1,324,363 | 28.15 | 859 |
| Group | Attr. | LSTM | CatBoost | LEM- MLP | LEM- Transformer | ScoutGPT |
| Cat. | Team | 0.94 / 0.94 | 0.94 / 0.94 | 0.66 / 0.37 | 0.96 / 0.49 | 0.92 / 0.92 |
| Role | 0.77 / 0.67 | 0.76 / 0.66 | 0.71 / 0.32 | 0.45 / 0.22 | 0.63 / 0.47 | |
| Type | 0.76 / 0.34 | 0.76 / 0.33 | 0.71 / 0.20 | 0.77 / 0.43 | 0.78 / 0.53 | |
| Out. | 0.89 / 0.59 | 0.89 / 0.59 | 0.92 / 0.14 | 0.89 / 0.61 | 0.95 / 0.86 | |
| Cont. | Start | 0.74 / 2.78 | 0.80 / 3.88 | 0.91 / 2.14 | 0.78 / 4.59 | 0.96 / 0.97 |
| Start | 0.76 / 2.34 | 0.80 / 3.32 | 0.94 / 1.63 | 0.78 / 4.09 | 0.93 / 1.00 |
| Goal-scoring (GS) | Goal-conceding (GC) | |||||
| Method | AUC | Brier | ECE | AUC | Brier | ECE |
| CatBoost [ 10 ] | 0.8424 | 0.0075 | 0.0003 | 0.8051 | 0.0021 | 0.00082 |
| LSTM [ 16 ] | 0.8294 | 0.0230 | 0.0660 | 0.8080 | 0.0029 | 0.01383 |
| GRU [ 8 ] | 0.8250 | 0.0205 | 0.0576 | 0.8091 | 0.0029 | 0.01376 |
| Transformer [ 33 ] | 0.8277 | 0.0210 | 0.0589 | 0.7948 | 0.0031 | 0.01519 |
| ScoutGPT | 0.8344 | 0.0069 | 0.0024 | 0.8153 | 0.0016 | 0.00081 |
| Group | Attr. | Ours | w/o Context | w/o Lineup | w/o Both |
| Cat. | Team | 0.921 / 0.921 | / | / | / |
| Role | 0.634 / 0.469 | / | / | / | |
| Type | 0.782 / 0.529 | / | / | / | |
| Out. | 0.947 / 0.862 | / | / | / | |
| Cont. | Start | 0.961 / 0.973 | / | / | / |
| Start | 0.925 / 1.001 | / | / | / |
| Minute | Score state | Original | Simulated | |
| 0 | Drawing | 0.02678 | 0.023477 | 0.003303 |
| Trailing | 0.02678 | 0.025916 | 0.000864 | |
| Leading | 0.02678 | 0.026392 | 0.000388 | |
| 40 | Drawing | 0.02678 | 0.029647 | 0.002867 |
| Trailing | 0.02678 | 0.029864 | 0.003084 | |
| Leading | 0.02678 | 0.029644 | 0.002864 |
| Model | Top-1 (%) | Top-5 (%) | Top-10 (%) |
| ScoutGPT (Ours) | 9.20 | 21.97 | 30.90 |
| w/o position masking | 8.48 | 20.02 | 27.05 |
| Statistics-based embedding | 8.98 | 21.36 | 30.34 |
| Role | Player | Episode VAEP Sum | Absolute Error w.r.t. GT | |||
| Naive | GT | Sim | |GT Sim| | |GT Naive| | ||
| Average | 4.85 | 4.71 | 4.59 | 1.25 | 1.84 | |
| Left Back | Jinsu Kim | 7.00 | 11.07 | 11.63 | 0.56 | 4.07 |
| Left Wing | Reis | 16.15 | 12.19 | 12.06 | 0.13 | 3.96 |
| Center Back | Hoik Jang | 3.79 | 4.78 | 5.83 | 1.05 | 0.99 |
| Central Midfielder | Jihoon Cho | 8.83 | 5.09 | 5.31 | 0.22 | 3.74 |
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Category | Action Type | Count | % |
| On-ball | Pass | 1,901,605 | 31.02 |
| Pass Received | 1,742,075 | 28.42 | |
| Carry | 841,879 | 13.73 | |
| Recovery | 263,881 | 4.30 | |
| Cross | 70,332 | 1.15 | |
| Shot | 50,165 | 0.82 |
| Abbr. | Full name | Abbr. | Full name |
| GK | Goalkeeper | CAM | Central Attacking Midfielder |
| CB | Center Back | LM | Left Midfielder |
| LB | Left Back | RM | Right Midfielder |
| RB | Right Back | LW | Left Winger |
| LWB | Left Wing Back | RW | Right Winger |
| RWB | Right Wing Back | CF | Center Forward |
| Player | Roles Played (minutes) |
| Jinsub Park | CB (4,383), CDM (2,081), CM (1,849) |
| Masatoshi Ishida | CM (2,282), CAM (1,693), RW (287), CF (237), RF (141), LM (90), LW (77), LF (67) |
| Sangho Na | LW (3,670), RW (1,508), LM (1,073), RM (692), CF (451), LF (405), RF (90) |
| Seungwon Jeong | RWB (2,355), CM (2,021), RW (450), RM (270), RB (199), CAM (90) |