TRACE: A Reproducible Benchmark for Electricity Price Forecasting with Official Operational Text

Xinyi Yi; Zhangdie Yuan; Ioannis Lestas

cs.LGSep 30, 2026

TRACE: A Reproducible Benchmark for Electricity Price Forecasting with Official Operational Text

Authors: Xinyi Yi, Zhangdie Yuan, Ioannis Lestas

Organizations: Department of Engineering, University of Cambridge · Department of Computer Science, University of Cambridge · University of Cambridge

Abstract

Electricity price forecasting (EPF) supports scheduling, bidding, and risk management in electricity markets, yet existing benchmarks focus mainly on numerical inputs, leaving the forecasting value of forecast-time textual context insufficiently evaluated. We introduce TRACE, a reproducible benchmark of 7,300 zone--day instances pairing prices from five zones in a major U.S. market with official operational text available at the forecast cutoff. TRACE reconstructs official operational text at each cutoff, preventing post-cutoff information leakage. We evaluate TRACE for semantic alignment and forecasting value. Semantic assessments align with central movement and both tail risks in ground-truth prices, most consistently for upper-tail price risk. Forecasting value is reflected in a median 7.4% reduction in upper-tail pinball loss across time-series foundation models. A controlled cross-day text-mismatch ablation reverses the gains, falling below the no-text baseline.

Figures & tables

Figure 1: TRACE instance from COMED (target: 7 January 2022). A Cold Weather Alert was issued four hours before the forecast cutoff (b); 9 prices exceeded the prior 14-day maximum (a,c).

Figure 2: TRACE construction.

Zone	Central $A_{C}$	Upper $A_{U}$	Lower $A_{L}$
COMED	2/4	4/4	2/4
AEP	3/4	4/4	2/4
DOM	4/4	4/4	4/4
PSEG	4/4	4/4	4/4
BGE	4/4	4/4	4/4
All	17/20	20/20	16/20

Table 1: Cross-zone semantic alignment. Full results are in Appendix B.2 .

Figure 3: Direct text fusion (A) and structured semantic fusion (B).

Zone	Direct	$\downarrow\%$ vs. NumOnly	$\downarrow\%$ vs. Shuffle
COMED	4.8973	6.4%	19.6%
AEP	4.4380	7.2%	20.6%
DOM	8.4872	7.9%	20.7%
PSEG	5.2073	3.7%	23.8%
BGE	6.9271	9.1%	27.4%

Table 2: Forecasting results of direct text fusion and structured semantic fusion.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Source	Median age	Within 1 day	Within 3 days	Within 7 days
Emergency Procedures	1.15 days	44.5%	77.4%	94.9%
Inside Lines	3.00 days	21.3%	50.1%	79.9%

Appendix

Table 3: Recency of candidate PJM text sources under unlimited lookback.

Context window	Context-bearing days	Coverage
1 day	326	42.2%
7 days	621	80.3%
14 days	677	87.6%
28 days	719	93.0%
Full archive	773	100.0%

Appendix

Table 4: Sensitivity of forecast-time context coverage to the lookback window. Coverage is relative to the 773 context-bearing days obtained from the full archive.

		Central $A_{C}$		Upper tail $A_{U}$		Lower tail $A_{L}$
Zone	LLM	prev. day	same wkday	prev. day	same wkday	prev. day	same wkday
COMED	Claude	0.369 (0.140)	0.402 (0.012)	0.164 (0.006)	0.429 (0.006)	$-0.008$ (0.661)	0.339 (0.006)
	DeepSeek	0.361 (0.250)	0.394 (0.012)	0.177 (0.006)	0.415 (0.006)	0.014 (0.457)	0.341 (0.006)
AEP	Claude	0.413 (0.006)	0.384 (0.060)	0.247 (0.006)	0.402 (0.006)	0.033 (0.272)	0.336 (0.006)
	DeepSeek	0.413 (0.006)	0.399 (0.006)	0.263 (0.006)	0.390 (0.006)	0.069 (0.069)	0.310 (0.006)
DOM	Claude	0.411 (0.006)	0.419 (0.006)	0.282 (0.006)	0.363 (0.006)	0.107 (0.012)	0.330 (0.006)

Appendix

Table 5: Five-zone semantic alignment on context-bearing training and validation days. Entries are observed statistics with Holm-corrected permutation

p

-values in parentheses; bold denotes positive alignment with

p_{\mathrm{Holm}}<0.05

A_{C}

is balanced accuracy (chance

1/3

), while

A_{U}

and

A_{L}

are Spearman correlations (null

0

	$\tau=0.1$		$\tau=0.5$		$\tau=0.9$
Zone	NumOnly	Direct	NumOnly	Direct	NumOnly	Direct	Shuffle mean	$\downarrow$ NumOnly	$\downarrow$ Shuffle
COMED	3.1988	3.4357	6.8241	7.1517	5.2348	4.8973	6.0941	6.4	19.6
AEP	2.9475	3.2939	6.2434	6.6321	4.7835	4.4380	5.5876	7.2	20.6
DOM	4.1392	4.5165	11.0238	10.7575	9.2139	8.4872	10.7006	7.9	20.7
PSEG	3.3089	3.5489	7.0988	7.4453	5.4075	5.2073	6.8318	3.7	23.8
BGE	3.5840	4.1484	9.4686	9.5586	7.6196	6.9271	9.5423	9.1	27.4

Appendix

Table 6: Direct text fusion against the NumOnly control and shuffled-context control, five zones, 131 context-bearing test days each (pinball loss, USD/MWh, at the released four-decimal precision).

\downarrow

columns are relative reductions (%) of Direct from NumOnly and from Shuffle mean at

\tau=0.9

		Numerical		Claude Opus 5		DeepSeek V4-Flash		Claude, other $\tau$
Zone	Baseline	Frozen	Cnum	Fusion	Shuffle	Fusion	Shuffle	$\Delta_{0.1}$	$\Delta_{0.5}$
COMED	TimesFM 2.5	4.1233	4.0277	3.6358	4.2001	3.6732	4.2970	$+$ 0.1222	$-$ 0.0354
	TimesFM 3.0	3.7222	3.6047	3.4027	3.8609	3.4160	3.8716	$+$ 0.1203	$-$ 0.0326
	Chronos-Bolt-Tiny	3.8887	3.7881	3.5255	3.9029	3.5229	3.9579	$-$ 0.0052	$-$ 0.0433
	Chronos-Bolt-Base	3.6671	3.5301	3.3962	3.7125	3.4100	3.7610	$+$ 0.0560	$-$ 0.0312
	Chronos-2	3.7438	3.6154	3.4640	3.8474	3.4765	3.8603	$-$ 0.1128	$+$ 0.0238

Appendix

Table 7: Structured semantic fusion for all 70 zone–model–LLM settings on 131 context-bearing test days (pinball loss, USD/MWh; lower is better). Frozen is the frozen numerical baseline and Cnum the separately trained control with the semantic vector zeroed during training. Fusion uses the correct forecast-time day–context pairing; Shuffle is the mean over

1{,}000

test-time context re-pairings. The final two columns report Claude Fusion minus Cnum at the other quantiles; negative favors structured semantic information.

Electricity Price Forecasting

Electricity price forecasting predicts wholesale and retail prices to support trading, grid operations, and energy planning.

See all 20 papers

Foundation models for electricity price forecasting and battery arbitrage: Can they replace market-specific forecasting models?
Aug 31, 2026Arkadiusz Lipiecki, Rafał WeronTabular Foundation Models Electricity Price Forecasting
Department of Computational Social Science, Wrocław University of Science and Technology, Poland · Department of Operations Research and Business Intelligence, Wrocław University of Science and Technology, Poland · Center for Research in Energy (CoRE), Aarhus University, 8000 Aarhus C, Denmark
Forecast Workflow Bench: Evaluating Language-Model Decisions with Budgeted Forecast Tools
Sep 23, 2026Shunya NagashimaForecasting Benchmarks LLM Tool Use
Neurogica Inc.
Evaluating Time Series Foundation Models for Electricity Price Forecasting: Contamination Risk, Distributional Shifts, and Covariate Dependence
Jul 2, 2026Zhenghua Pan, Ahmed Aziz EzzatZero-Shot Time Series Forecasting Electricity Price Forecasting
Department of Industrial & Systems Engineering, Rutgers, The State University of New Jersey, USA.

Aug 31, 2026J/K move · Enter open · S save

Foundation models for electricity price forecasting and battery arbitrage: Can they replace market-specific forecasting models?

Arkadiusz Lipiecki, Rafał Weron

Department of Computational Social Science, Wrocław University of Science and Technology, Poland · Department of Operations Research and Business Intelligence, Wrocław University of Science and Technology, Poland · Center for Research in Energy (CoRE), Aarhus University, 8000 Aarhus C, Denmark

Tabular Foundation Models Electricity Price Forecasting Time Series Forecasting Probabilistic Forecasting Time-Series Foundation Models

Open paper AskPDF arXiv

Foundation models promise accurate forecasts with little or no task-specific training, but whether they can replace models designed specifically for electricity price forecasting remains unclear. We compare nine variants from five foundation model families, evaluated in zero-shot mode, with two state-of-the-art electricity price forecasting benchmarks in Germany, Poland, and Spain over 2021-2025. Their performance is assessed in terms of point and probabilistic forecasting accuracy, as well as economic value in battery energy storage arbitrage. Only the TabPFN models consistently and significantly outperform the benchmarks across all three markets and all statistical measures. However, this statistical dominance does not translate directly into economic dominance: TabPFN performs best under unlimited bids and riskier quantile-based strategies, whereas the Distributional Deep Neural Network benchmark is more profitable when risk tolerance is lower. Thus, foundation models cannot universally replace market-specific models, and their value depends on both model architecture and the decision problem.

Abstract

Electricity Price Forecasting

Explore similar work

Foundation models for electricity price forecasting and battery arbitrage: Can they replace market-specific forecasting models?

Forecast Workflow Bench: Evaluating Language-Model Decisions with Budgeted Forecast Tools

Evaluating Time Series Foundation Models for Electricity Price Forecasting: Contamination Risk, Distributional Shifts, and Covariate Dependence