Large Language Models for Forecasting of Electricity Price Spikes
Organizations: School of Electrical and Computer Engineering, The University of Sydney, Australia · School of Electrical and Data Engineering, University of Technology Sydney, Australia · Wood Mackenzie · CSIRO, Australia
Abstract
Can market notices read by a large language model (LLM) improve short-horizon electricity price spike forecasts? We study five regions of Australia's National Electricity Market. A local LLM extracted features once per notice, only notices visible at the forecast origin were used, and a gradient-boosting model produced onset probabilities. In an out-of-sample development back-test (July 2022 to July 2024), notice features did not lower the 60-minute Brier score of a numeric model including the market operator's next-hour forecasts. At 24 hours they did, but rule-based features did so almost equally and attribution to notice content was not established. A locally frozen protocol, not publicly deposited, governed a retrospective single-use holdout from 15 August to 17 October 2026. At 60 minutes, the hypothesis that LLM features lower the numeric model's Brier score was not established (H1; relative Brier skill -0.09%), so the planned comparison with rule-based features was not tested (H2). At 24 hours, the hypothesis that LLM features lower the score against a text-free control was not established (H3; +0.37%, 95% CI [-9.74, 1.39], inconclusive). By the frozen wording rule the non-significant 24-hour improvement comes mainly from recalibration and cannot be attributed to notice content, although post hoc the decomposition could not separate recalibration from resolution. The 24-hour base rate was 1.56% on the holdout against 14.0% in development. With 5 onsets on 3 days and 24-hour positives in 2 of 22 three-day blocks, the holdout neither established nor excluded a 24-hour effect.
Figures & tables
| Study | Target and threshold | NEM | Text source | Test period | Text timing | Non-LLM text baseline | Increment tested | Fixed before test data |
| Christensen et al. [17] | Spike probability; 100 AUD/MWh | Yes | None | July to September 2007 | – | – | – | – |
| Eichler et al. [22] a | Spike probability; 100 and 300 AUD/MWh | Yes | None | July to September 2007; 2011 | – | – | – | – |
| Liu et al. [35] | Extreme-price probability; 95th percentile (also 90th, 99th) | Yes | None | 25% of the 2019 data (dates n.r.) | – | – | – | – |
| Lu and Suthaharan [38] | Daily spike count and probability; threshold estimated for each year | Yes | None | July 2019 to September 2021, NSW | – | – | – | – |
| Hornek et al. [27] | Day-ahead price | No | None | 2024 | – | – | – | – |
| Huang et al. [28] | Five-minute price | Yes | News articles | 22 days of May 2024 | Day | No | Yes | n.r. |
| Item | Source and coverage | Use |
| Numeric data, development | MMSDM dispatch, regional, P5MIN and PD tables, October 2021 to July 2026 | Inputs and labels of the back-test and the final models |
| Numeric data, holdout | NEMweb archive and current directories, downloaded by the frozen opening script | Holdout inputs and labels |
| Training notices | MMSDM MARKETNOTICEDATA, October 2021 to July 2024; 21,954 notices, 14,019 after removing four routine types | Notice features and text increments |
| Holdout notices | NEMweb current directory from 14 August 2026; backfill, then polling every 120 seconds, from shortly after midnight AEST on 13 October 2026; 15 retained types | Holdout notice features |
| No notice text used | August 2024 to mid-August 2026 | None |
| Development back-test | Five regions; rolling origins July 2022 to July 2024 in nine folds; 365,622 scored origins at 60 minutes, 365,313 at 24 hours | Development out-of-sample evidence |
| Comparison | Horizon | Statistic | ||
| H1 | L1 against A | 60 min | 0.025 | Day-cluster bootstrap |
| H2 | L1 against B, tested only after H1 is established | 60 min | 0 | Day-cluster bootstrap |
| H3 | L1 against A0 | 24 h | 0.025 | Bootstrap over non-overlapping three-day blocks |
| Comparison | 95% CI | Label (pooled) | Non-selection | 95% CI | |
| L1 vs A | 0.66 | [ 1.04, 0.24] | equivalent but slightly worse | 0.63 | [ 1.10, 0.11] |
| L1 vs B | 0.14 | [ 0.45, 0.18] | equivalent | 0.14 | [ 0.52, 0.25] |
| B vs A | 0.52 | [ 0.87, 0.15] | equivalent but slightly worse | 0.49 | [ 0.94, 0.05] |
| L2 vs L1 | +0.38 | [0.13, 0.63] | superior but below the smallest effect of interest | +0.33 | [0.02, 0.63] |
| L2 vs A | 0.27 | [ 0.50, 0.02] | equivalent but slightly worse | 0.30 | [ 0.59, 0.03] |
| L1 vs A0 | 0.58 | [ 0.93, 0.18] | equivalent but slightly worse | 0.59 | [ 1.02, 0.09] |
| Model | Brier | REL | RES | AUC | PR-AUC |
| A | 8.270 | 3.77 | 156.1 | 0.941 | 0.267 |
| B | 8.313 | 4.62 | 152.2 | 0.942 | 0.262 |
| L1 | 8.324 | 5.01 | 151.7 | 0.941 | 0.260 |
| L2 | 8.292 | 3.89 | 155.2 | 0.942 | 0.264 |
| A0 | 8.276 | 3.68 | 155.1 | 0.940 | 0.266 |
| P5MIN | 8.584 | 5.77 | 128.3 | 0.879 | 0.192 |
| Comparison | Five regions | 95% CI | Label | Without NSW1 | 95% CI |
| L1 vs A0 | +1.65 | [0.54, 2.74] | superior | +0.78 | [ 0.42, 1.97] |
| B vs A0 | +1.47 | [0.32, 2.56] | superior | +0.59 | [ 0.79, 1.94] |
| L1 vs B | +0.18 | [ 0.69, 1.08] | equivalent | +0.19 | [ 0.79, 1.20] |
| L2 vs A0 | +2.10 | [1.02, 3.19] | superior | +1.53 | [0.34, 2.70] |
| L1 vs A | +2.07 | [0.62, 3.39] | superior | +1.27 | [ 0.33, 2.62] |
| B vs A | +1.90 | [0.44, 3.25] | superior | +1.08 | [ 0.52, 2.53] |
| Development back-test | Holdout | ||||||
| Horizon | Origins | Positives | Base rate (%) | Origins | Positives | Base rate (%) | Mean forecast of A / base rate |
| 60 minutes | 365,622 | 3,631 | 0.993 | 30,720 | 20 | 0.0651 | 1.69 |
| 24 hours | 365,313 | 51,198 | 14.0 | 30,720 | 480 | 1.56 | 2.61 |
| Highest 60-minute probability | ||||||||
| Region | Onset (AEST) | Price at onset (AUD/MWh) | Highest price within the hour (AUD/MWh) | Minutes above 600 within the hour | A | L1 | P5MIN | PD |
| NSW1 | 9 Oct 05:00 | 992.32 | 992.32 | 35 | 0.008 | 0.008 | 0.22 | 0.03 |
| NSW1 | 10 Oct 07:20 | 23,200.00 | 23,200.00 | 5 | 0.0085 | 0.0087 | 0.014 | 0.0026 |
| NSW1 | 12 Oct 14:10 | 1,341.98 | 23,107.00 | 55 | 0.094 | 0.097 | 0.27 | 0.026 |
| QLD1 | 9 Oct 05:00 | 824.88 | 860.05 | 35 | 0.00074 | 0.00074 | 0.096 | 0.012 |
| SA1 | 9 Oct 07:15 | 1,001.00 | 3,210.98 | 10 | 0.033 | 0.033 | 0.0059 | 0.0018 |
| Comparison | Horizon | Bootstrap unit | Skill (%) | 90% CI | 95% CI | Level | Descriptive label | Decision | ||
| H1 | L1 against A | 60 min | day clusters | 0.09 | [ 0.52, 0.02] | [ 1.18, 0.01] | 0.9994 | 0.025 | equivalent but slightly worse | not established |
| H2 | L1 against B | 60 min | day clusters | 0.18 | [0.01, 0.64] | [0.01, 0.64] | 0 | superior but below the smallest effect of interest | not tested | |
| H3 | L1 against A0 | 24 h | three-day blocks | 0.37 | [ 8.81, 1.30] | [ 9.74, 1.39] | 0.4335 | 0.025 | inconclusive | not established |
| Sample | Origins | Positives | Skill (%) | 90% CI | 95% CI | ||
| Holdout | All origins | 30,720 | 480 | 0.37 | [ 8.81, 1.30] | [ 9.74, 1.39] | 0.4335 |
| With a visible notice | 14,700 | 253 | 1.72 | [ 8.18, 2.98] | [ 9.52, 3.13] | 0.2417 | |
| Without a visible notice | 16,020 | 227 | 1.35 | [ 11.75, 0.33] | [ 12.45, 0.23] | 0.9979 | |
| Development back-test | All origins | 365,313 | 51,198 | 1.65 | [0.68, 2.60] | [0.54, 2.74] | 0.0010 |
| With a visible notice | 231,937 | 38,673 | 1.83 | [0.53, 3.07] | [0.36, 3.29] | 0.0070 | |
| Without a visible notice | 133,376 | 12,525 | 1.13 | [0.13, 2.22] | [ 0.02, 2.43] | 0.0265 |
| Sample | Model | Brier | REL | RES | Residual | AUC | PR-AUC |
| Holdout | L1 | 17.062 | 189.0 | 28.9 | 8.0 | 0.772 | 0.053 |
| A0 | 17.126 | 194.3 | 23.7 | 3.9 | 0.764 | 0.037 | |
| A | 17.083 | 188.8 | 23.9 | 5.4 | 0.765 | 0.037 | |
| Development back-test | L1 | 102.044 | 82.0 | 1766.6 | 161.7 | 0.780 | 0.417 |
| A0 | 103.752 | 35.9 | 1582.3 | 129.0 | 0.766 | 0.390 | |
| A | 104.205 | 19.1 | 1506.3 | 142.9 | 0.762 | 0.380 |
| Comparison | Skill (%) | 90% CI | 95% CI | Label |
| L1 against A0 | 0.37 | [ 8.81, 1.30] | [ 9.74, 1.39] | inconclusive |
| L1 against A | 0.13 | [ 10.79, 1.21] | [ 11.78, 1.32] | inconclusive |
| B against A0 | 1.47 | [ 13.90, 0.47] | [ 15.39, 0.68] | inconclusive |
| L1 against B | 1.82 | [0.39, 5.68] | [0.26, 6.12] | superior |
| L2 against A0 | 0.65 | [ 12.60, 0.56] | [ 14.13, 0.69] | inconclusive |
| L1 against A0, without NSW1 | 3.28 | [ 13.57, 1.01] | [ 14.31, 0.80] | inferior |
| 60 minutes (day clusters) | 24 hours (three-day blocks) | |||||||
| Comparison | Skill | 90% CI | 95% CI | Label | Skill | 90% CI | 95% CI | Label |
| A against P5MIN | 7.82 | [ 2.42, 49.70] | [ 4.44, 75.94] | inconclusive | 20.47 | [5.35, 68.57] | [3.53, 70.48] | superior |
| A against PD | 2.09 | [ 10.86, 7.14] | [ 18.20, 14.07] | inconclusive | 14.93 | [2.15, 62.40] | [0.69, 64.59] | superior |
| A against climatology | 16.15 | [4.89, 60.42] | [3.43, 86.22] | superior | – | – | – | – |
| A against P5MIN, without NSW1 | 21.52 | [8.85, 92.15] | [7.56, 93.80] | superior | 30.46 | [11.05, 66.18] | [8.85, 67.79] | superior |
| Analysis | Comparison | Skill (%) | 90% CI | 95% CI | Label | |
| 60 minutes (day-cluster bootstrap) | ||||||
| Main analysis | L1 against A | 0.09 | [ 0.52, 0.02] | [ 1.18, 0.01] | 0.9994 | equivalent but slightly worse |
| L1 against B | 0.18 | [0.01, 0.64] | [0.01, 0.64] | superior but below the smallest effect of interest | ||
| L1 against A0 | 0.14 | [0.03, 0.56] | [0.03, 1.11] | superior but below the smallest effect of interest | ||
| Single seed | L1 against A | 0.15 | [ 0.74, 0.03] | [ 1.53, 0.02] | 1.0000 | equivalent but slightly worse |
| L1 against B | 0.13 | [ 0.003, 0.523] | [ 0.01, 0.56] | 0.1118 | equivalent | |
| Model | Onsets | Detected | Missed | False alarms | Lead time of detections (min) |
| A | 5 | 1 | 4 | 6 | 30 |
| B | 5 | 1 | 4 | 6 | 30 |
| L1 | 5 | 1 | 4 | 6 | 30 |
| L2 | 5 | 1 | 4 | 6 | 30 |
| P5MIN | 5 | 2 | 3 | 23 | 30 |
| PD | 5 | 0 | 5 | 12 | – |
| Numeric signal | Onsets with both | Notice first (%) | 95% CI | Expected (%) | Difference (pp) | 95% CI | Signal first or same (%) | 95% CI |
| Any numeric alarm (pooled) | 398 | 15.3 | [10.9, 20.7] | 13.7 | 1.7 | [ 1.0, 4.8] | 84.7 | [79.3, 89.1] |
| Numeric model’s alarm | 366 | 70.2 | [63.8, 77.0] | 62.9 | 7.3 | [2.4, 12.0] | 29.8 | [23.0, 36.2] |
| P5MIN price above 600 AUD/MWh | 339 | 69.3 | [61.5, 77.5] | 62.0 | 7.3 | [2.4, 12.4] | 30.7 | [22.5, 38.5] |
| PD price above 300 AUD/MWh for the onset half-hour | 334 | 6.9 | [4.1, 10.5] | 6.0 | 0.9 | [ 0.8, 2.9] | 93.1 | [89.5, 95.9] |
| P5MIN reference alarm at its frozen threshold | 332 | 67.5 | [58.6, 76.7] | 61.2 | 6.3 | [0.7, 11.9] | 32.5 | [23.3, 41.4] |
| PD reference alarm at its frozen threshold | 303 | 63.4 | [53.4, 74.3] | 57.2 | 6.2 | [0.7, 12.1] | 36.6 | [25.7, 46.6] |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Item | Identifier |
| Manifest of file hashes (SHA-256) | 20aff96243bd6aae1913683ffd7042f0eff3c2a3db15671e2eb6953df8d29ff9 |
| Prompt and token limits (SHA-256) | 86d01917a40ab956be9bdaeb1caceafbccfa62e8be337bc0225b5853c1c14f2f |
| Confirmatory output file (SHA-256) | 15b81749fa3208bf8eadcfabae77fea8bb35f1a641527fd1273e9e033afb231e |
| Text-only build of Qwen3.5-9B | principled-intelligence/Qwen3.5-9B-text-only |
| Model revision | 2ba8a2e101fb96c67b3ce5a83beddc64c23fff43 |
| No. | Item | Type | Affects a decision? |
|---|---|---|---|
| 1 | The frozen text contains two out-of-date details. It quotes the development 24-hour skill of L1 against B as 0.06%, a value computed with the earlier 30-minute visibility delay; with the frozen 25-minute delay it is 0.18%, and 0.25% in the post-freeze refit. A section heading names an earlier version of the opening script than the one frozen and run. Neither was changed, because the text is frozen. | Out-of-date text | No |
| 2 | The descriptive split by the presence of notices was computed by a separate script after scoring. Its first run stopped while writing its output file; after a one-line fix it was rerun, and all tables were identical. The step lines of the opening script, which contain no scores, were visible on the console during the run, although the procedure allowed only the last line to be read, and only after completion. | Procedural | No |
| 3 | A post-freeze, pre-opening 24-hour refit, with training sets ending 24 hours before each fold, is listed beside H3 and in the development reference of the notice split, for description only. This was decided on 19 October 2026, before opening. H3 rests on the original back-test. | Added descriptive comparison | No |
| 4 | The holdout had far fewer events than the design assumed: 5 onsets, against 20–34 in a design-stage projection from monthly onset counts of 2023–2025, counted on trading prices with a definition that differs from the frozen label (Section 8 ). | Disclosure | No |
| 5 | The 25-minute visibility delay rests on 43 notices observed live. The excluded notices, the percentile for retained types and the unmeasured period are given in Section 3 . | Limitation | No |
| 6 | Notice extraction ran in a separate software environment from the numeric pipeline. The environment of the training-period extraction was not recorded at the time. The environment of the holdout extraction reproduced the stored training-period outputs byte for byte on four training notices before the freeze and on 40 before opening (raw output, parse and embedding), which indicates, but does not prove, that the two are equivalent. | Disclosure | No |
Electricity Price Forecasting
Electricity price forecasting predicts wholesale and retail prices to support trading, grid operations, and energy planning.
See all 24 papers