eess.SPNov 28, 2025
SaveWhat If They Took the Shot? A Hierarchical Bayesian Framework for Counterfactual Expected Goals
Organizations: Cardiff University, School of Mathematics, Abacws, Senghennydd Road, Cardiff, CF24 4AG, UK. · Cardiff University, School of Computer Science and Informatics, Abacws, Senghennydd Road, Cardiff, CF24 4AG, UK. · Dead Ball Analytics Limited, Barry, CF63 2QQ, UK.
Abstract
How would the probability of scoring change if a different player took the same shot? Although expected goals models describe the difficulty of scoring opportunities, comparing players on identical chances requires accounting for uncertainty in their estimated conversion probabilities. We address this question through a hierarchical Bayesian framework that evaluates alternative shooters under the same observed circumstances. Using 35,700 eligible shots from four leagues during the 2015/16 season, we compare a contextual benchmark with a primary player-intercept model and a provisional extension that allows player differences to vary with shooting conditions. Paired posterior draws preserve shared uncertainty when individual probability differences are aggregated across opportunities. Chronological validation demonstrates improved predictive performance relative to a constant-rate baseline, but introducing player effects provides negligible additional gains. Controlled single-seed experiments show that greater information per player improves estimation accuracy and narrows uncertainty intervals, although distinguishing small differences remains difficult and confident conclusions about near-null effects can become more frequent. Applications involving Giroud--Vardy and Berardi--Sansone similarly yield estimated differences that are small relative to their posterior uncertainty. These findings highlight an important distinction between predicting the outcome of a shot and establishing whether one player would be more likely to convert it. The proposed framework makes this distinction explicit by evaluating alternative shooters on fixed recorded opportunities while accounting for their contextual exposure. The resulting comparisons concern conditional conversion rather than the causal consequences of replacing a player in a match.
Figures & tables
Figure 1: Comparing two shooters on the same recorded opportunity. The figure illustrates conditional conversion comparisons using two training-period attempts: (a) Vardy’s shot on 21 November 2015 and (b) Giroud’s shot on 24 October 2015. Both players are evaluated under the recorded circumstances using the provisional context-dependent model M2 (Section 4 ). The half-pitches are rotated counter-clockwise, with the goal positioned at the top, while preserving the proportions of the original coordinate system. Orange identifies the shooter, purple the opposing goalkeeper, filled grey markers other opponents and hollow markers teammates. The triangle connects the shot location to the goalposts. The examples were selected retrospectively because they produced the largest opposing posterior-mean contrasts among opportunities within both players’ training distance and angle ranges. Their pressure annotations and body-part/technique combinations also occur in both players’ records, although this marginal overlap does not establish joint contextual support. Posterior-mean probabilities, paired 95% credible intervals and sign probabilities describe the estimated differences between shooters. Both intervals include zero, leaving the preferred shooter unresolved in either example. The primary player-intercept model produces no opposing posterior-mean preferences on these opportunity sets. Source event identifiers document the selected attempts.
| Population or exclusion combination | Shots |
| Raw four-league shot universe | 37,888 |
| Non-open-play free kick | 1,749 |
| Penalty + missing freeze frame + missing goalkeeper geometry | 354 |
| Penalty only | 46 |
| Missing goalkeeper geometry only | 32 |
| Non-open-play corner only | 6 |
Table 1: Shot eligibility and chronological partition. The initial four-league dataset contains 37,888 recorded shots, of which 35,700 satisfy the eligibility criteria. Exclusion categories are mutually exclusive and account for 2,188 attempts in total, including cases where multiple exclusion reasons coincide. The eligible population is divided into training and validation partitions according to match date.
Figure 2: Geometric characteristics of the recorded shooting opportunities. (a) Spatial distribution of all 35,700 eligible shots. Darker regions indicate overlapping observations rather than a smoothed probability surface. (b) The first attempt in Vardy’s opportunity set, recorded on 8 August 2015, illustrating the triangle formed by the shot location and goalposts, together with the positions of the goalkeeper and other recorded players. Goalkeeper distance is measured from the goal centre, while the defensive count includes opposing players strictly inside the triangle, including the goalkeeper when applicable. (c,d) Distributions of shot distance and goal-opening angle, each displayed using 30 equally spaced bins. Pitch coordinates follow the original system, with attacks directed towards . Distances are expressed in source coordinate units and angles in radians. These visualisations describe the geometry of observed opportunities rather than differences in player ability.
| Predictor | Log loss | Brier score |
|---|---|---|
| Training-rate constant | 0.32946 | 0.0915 |
| M0 | 0.26687 | 0.0770 |
| M1 | 0.26677 | 0.0770 |
| M2 (provisional) | 0.26676 | 0.0770 |
Table 2: Predictive performance and posterior sampling diagnostics. The upper panel reports chronological validation scores for the constant training-rate benchmark and the three fitted models. Lower log loss and Brier score indicate better probabilistic predictions. The lower panel summarises posterior sampling diagnostics for M0, the primary hierarchical model M1, and the provisional context-dependent extension M2. BFMI is reported as the minimum across chains, while effective sample sizes refer to the reported diagnostic summaries. No model exhibits divergent transitions or tree-depth saturation. M2 does not satisfy the specified criterion for every player-slope parameter and is therefore treated as provisional.
Figure 3: Chronological predictive evaluation and uncertainty in model comparisons. (a) Training and validation periods, separated chronologically without shared matches, events or calendar dates. (b) Calibration of M1 across ten equal-frequency validation bins. Points represent mean predicted probabilities and observed goal frequencies within each bin, while the diagonal indicates perfect calibration. The plotted points do not include independently replicated uncertainty intervals. (c) Log loss and paired match-bootstrap 95% intervals for differences between posterior-mean predictors. All fitted models improve upon the constant training-rate benchmark, but differences among M0, M1 and provisional M2 remain small. Bootstrap intervals account for match-level resampling without incorporating posterior parameter uncertainty. These evaluations concern predictions for observed outcomes, rather than the recovery of unobserved alternative-shooter conversion differences.
| Scenario/model | Shots/player | RMSE | Width 95 | Cov 95 | Correct 95 | False 95 |
|---|---|---|---|---|---|---|
| S1/M1 | 15 | 0.0231 | 0.1564 | 99.79 | 0 | 0.000 |
| S1/M1 | 60 | 0.0201 | 0.1039 | 99.49 | 0 | 0.000 |
| S1/M1 | 300 | 0.0139 | 0.0499 | 98.27 | 0 | 0.000 |
| S2/M2 | 15 | 0.0528 | 0.3029 | 99.90 | 13 | 0.033 |
| S2/M2 ∗ | 60 | 0.0337 | 0.1589 | 99.74 | 33 | 0.016 |
| S2/M2 | 300 | 0.0224 | 0.0792 | 98.16 | 169 | 0.855 |
Table 3: Recovery of known player differences as training information increases. Results concern canonical seen–seen comparisons at the primary tolerance , with 15, 60 or 300 training shots per player. RMSE measures posterior-mean error, while Width 95 is the mean width of equal-tail 95% credible intervals; both are expressed as probability differences. Cov 95 and False 95 are percentages calculated across 6,080 dependent pair/context cells. Correct 95 counts meaningful contrasts resolved in the true direction, among 1,763 meaningful cells in S1 and 2,299 in S2. False confidence includes wrong-direction claims and confident claims about effects within . The asterisk identifies the provisional S2/M2 fit with 60 shots per player. Results describe one simulated realisation rather than repeated sampling coverage or validated football sample-size requirements.
Figure 4: More observations improve estimation without necessarily resolving player differences. The nested information experiment holds 20 simulated players, their generating conversion effects and 32 evaluation opportunities fixed while increasing the number of training shots from 15 to 60 and 300 per player. Results cover 190 canonical seen–seen pairs and 6,080 dependent pair/context cells at . (a) Posterior-mean RMSE decreases with additional observations. (b) Mean equal-tail 95% credible intervals become narrower. (c) Correct resolution of meaningful contrasts increases under S2/M2 but remains zero under S1/M1. (d) False directional confidence includes wrong-direction claims and confident claims about effects within . The hollow S2/60 marker denotes a provisional fit. Although increasing information improves accuracy and precision in this realisation, it does not eliminate uncertainty about meaningful differences or prevent confident conclusions about near-null effects. The results do not establish population calibration, a general sample-size relationship or validated thresholds for football applications.
| Set | Model | Goals | Factual xG | Alternative xG | [95% CrI] | |
|---|---|---|---|---|---|---|
| M1 | 19 | 19.652 | 19.603 | [ ] | 0.484 | |
| M2 | 19 | 19.405 | 19.439 | [ ] | 0.506 | |
| M1 | 15 | 13.413 | 13.452 | [ ] | 0.516 | |
| M2 | 15 | 13.255 | 13.163 | [ ] | 0.481 | |
| M1 | 3 | 4.156 | 4.190 | [ ] | 0.515 | |
| M2 | 3 | 4.137 | 4.206 | [ ] | 0.549 |
Table 4: Conditional conversion comparisons on fixed historical opportunities. Each direction evaluates two shooters on the same recorded attempts. assigns Giroud to Vardy’s 110 opportunities, assigns Vardy to Giroud’s 102 opportunities, and assigns Sansone to Berardi’s 75 opportunities. Goals are the factual shooter’s observed outcomes on eligible attempts, while xG values are posterior means. denotes alternative minus factual expected goals, with equal-tail 95% credible intervals calculated from paired posterior draws. Positive contrasts favour the alternative shooter. M1 is the primary hierarchical model; M2 remains provisional. All intervals include zero, so none of the comparisons establishes a preferred shooter.
Figure 5: Giroud and Vardy evaluated on each other’s recorded opportunities. (a,b) Spatial distributions of Vardy’s 110 and Giroud’s 102 eligible attempts. Circles identify training-period contexts, triangles validation-period contexts, and rings observed goals. Both players are evaluated on every opportunity within each set, although the two sets contain different attempts. Half-pitches are rotated counter-clockwise, placing the goal at the top while preserving the source coordinates and isotropic proportions. (c) M1 paired probability differences across Vardy’s opportunities, plotted against shot distance. Points show posterior means and lines indicate pointwise equal-tail 95% credible intervals. (d) Aggregate expected-goal contrasts obtained by summing differences within aligned posterior draws. Solid blue intervals represent primary M1 and dashed teal intervals provisional M2. Positive values favour the alternative shooter on the specified opportunity set. All aggregate intervals include zero. The spatial panels describe observed shooting locations, not a probability surface or a model of tactical movement.
Figure 6: Berardi–Sansone comparison across shooting geometry, pressure and observed exposure. (a) Berardi’s 75 eligible opportunities displayed on a counter-clockwise-rotated half-pitch, with the goal at the top. Colour indicates recorded pressure annotation, rings identify factual goals, and triangles denote validation-period contexts. (b) The same attempts in distance–angle space. The rectangle marks Sansone’s marginal training ranges, while hollow orange edges identify opportunities outside at least one range. Ten attempts fall outside his angle range and one outside his distance range. The rectangle does not establish joint contextual support. (c) Paired aggregate contrasts for the 54 attempts without recorded pressure and the 21 annotated-pressure attempts. Solid blue intervals denote primary M1 and dashed teal intervals provisional M2. Every equal-tail 95% credible interval includes zero. (d) Training shot counts and recorded-pressure counts. These panels describe observed exposure and conditional conversion estimates; they do not establish player specialisation, differences in composure or causal effects of pressure.
| Player | Shots | Pressure | Distance range | Angle range |
|---|---|---|---|---|
| Vardy | 85 | 19 | 4.245–30.490 | 0.069–1.490 |
| Giroud | 79 | 29 | 4.494–35.263 | 0.152–1.448 |
| Berardi | 50 | 14 | 5.235–48.894 | 0.066–1.270 |
| Sansone | 33 | 7 | 4.920–35.574 | 0.209–1.241 |
Table 5: Observed training exposure for the four application players. Shot counts describe the eligible training attempts available for estimating player effects, while pressure counts indicate attempts with recorded positive pressure annotations. Distance ranges are expressed in StatsBomb coordinate units and goal-opening angles in radians. The reported minima and maxima describe marginal exposure only; they do not establish whether a player has sufficient observations near a particular combination of contextual features or provide validated measures of joint support.
More in this topic
Counterfactual Prediction
Counterfactual prediction estimates outcomes under hypothetical interventions using observed data and assumptions about how systems generate outcomes.
See all 38 papers