JudgeCast: Time Series Forecasting with Experience-Informed Covariate Judgements

Donguk Kwon; Wooseok Jeong; Dongha Lee

cs.LGSep 29, 2026

JudgeCast: Time Series Forecasting with Experience-Informed Covariate Judgements

Authors: Donguk Kwon, Wooseok Jeong, Dongha Lee

Organizations: Yonsei University · Konkuk University

Abstract

Covariate effects vary across contexts and shift over time, requiring forecasters to assess how to use them for each forecasting context. As forecasting proceeds, observations for earlier forecasts become available, providing feedback on past covariate use for subsequent forecasts. However, when multiple covariates act together, the forecast error reveals the numerical discrepancy from the observation but not how the covariates should have been used. We introduce JudgeCast, an experience-based framework for time series forecasting with covariates. Following the judgmental adjustment practice, a frozen TSFM provides the base forecast, while a frozen LLM uses the current context and relevant experience to adjust it. Within the adjustment, assessing covariate effects and determining the numerical adjustment serve distinct roles, so JudgeCast first forms explicit covariate-wise judgments and then determines the adjustment. After observation, JudgeCast uses the observed residual of the base forecast to reconstruct alternative judgments and evaluates the original and alternatives through their resulting adjustments. The best-performing decision is selected and retained as validated experience for subsequent forecasts. Across diverse real-world datasets, JudgeCast outperforms strong baselines. Ablations show that explicit covariate-wise judgment can improve forecast-time adjustment, while residual-guided experience construction yields more reliable forecasting gains than retaining raw decisions as experience.

Figures & tables

Figure 1: Illustration of post-observation judgment reconstruction. A single reflection provides one plausible explanation, whereas JudgeCast revisits forecast-time judgments by exploring alternatives and comparing their resulting adjustments to construct reusable experience.

Figure 2: Overview of a single forecasting window in JudgeCast. A frozen TSFM provides the base forecast, while a frozen LLM uses relevant experience to form explicit covariate-wise judgments and determine the numerical adjustment. After observation, the residual guides alternative judgment generation and adjustment evaluation to select the best-performing decision, which is added to memory as validated experience only when its adjusted forecast improves on the base forecast.

Method	NP		PJM		BE		FR		DE		ENTSO-e		Rossmann
Method	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE a	MAE b	MSE a	MAE b
ARIMA	45.681	4.108	53.988	5.217	1,319.458	16.622	1,391.465	12.591	334.941	11.841	46.943	3.161	8.760	2.065
Prophet	50.383	5.086	70.194	6.379	950.769	17.790	1,003.981	14.898	464.086	15.197	7.544	1.986	10.793	2.228
DLinear	34.315	3.817	41.687	4.640	758.796	12.426	792.836	9.619	236.175	10.537	3.573	1.301	10.210	2.248
PatchTST	29.360	3.519	34.711	4.349	769.533	11.695	835.216	9.039	211.958	9.870	3.514	1.359	11.169	2.261
iTransformer	30.941	3.537	35.467	4.240	767.894	12.195	943.188	11.195	247.900	10.857	3.548	1.380	11.220	2.354

Table 1: Overall forecasting performance across diverse real-world datasets. Chronos-2 is the target-only frozen TSFM used as the base forecaster in JudgeCast. Lower values indicate better performance. The best and second-best results are shown in bold and underlined , respectively. a and b denote MSE and MAE reported in

\times 10^{6}

and

\times 10^{3}

, respectively.

Backbone	NP		PJM
Backbone	MSE	MAE	MSE	MAE
TimesFM 2.5	23.655	3.079	33.135	4.106
+ Qwen	19.314	2.979	26.860	3.827
+ Gemini	17.637	2.829	28.503	3.884
Toto 2.0-313M	26.921	3.165	32.307	4.115
+ Qwen	21.831	3.092	24.905	3.673

Table 2: Backbone robustness on NP and PJM. Qwen and Gemini abbreviate Qwen3.5-27B and Gemini 3.5 Flash-Lite. TSFM-only rows report base forecasts.

Explicit Judgment	Experience Construction	NP		PJM		BE		FR		DE		ENTSO-e		Rossmann
Explicit Judgment	Experience Construction	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE a	MAE b	MSE a	MAE b
$\times$	None	22.551	3.206	30.493	4.012	624.713	10.003	689.195	7.461	178.935	8.699	3.128	1.209	4.469	1.249
$\checkmark$	None	21.514	3.096	30.290	4.112	619.435	10.842	680.784	8.250	161.724	8.564	3.359	1.201	4.556	1.386
$\times$	Raw	22.422	2.908	34.429	4.219	641.466	10.310	718.869	7.564	210.147	9.421	2.773	1.068	4.363	1.247
$\checkmark$	Raw	22.767	2.990	29.975	3.958	623.891	10.158	712.784	7.710	192.831	9.012	3.162	1.195	7.194	1.649
$\checkmark$	Raw + Valid.	21.961	2.884	29.855	3.952	615.480	10.028	701.613	7.812	176.561	8.720	3.216	1.180	7.255	1.632

Table 3: Ablation study of explicit covariate-wise judgment and experience construction across datasets. Raw experience retains the original forecast-time judgment or adjustment without post-observation reassessment. Raw + Valid. retains the original judgment and adjustment only when the resulting adjusted forecast improves on the base forecast. Validated experience applies residual-guided experience construction and corresponds to full JudgeCast. Lower values indicate better performance. a and b denote MSE and MAE reported in

\times 10^{6}

and

\times 10^{3}

, respectively.

Table 6

Figure 3: Experience accumulation and retrieval analysis. (a) Normalized forecasting error as validated experience accumulates, relative to no experience. (b) Retrieval distance for judgment and adjustment retrieval across memory sizes. (c) Relative MSE change under relevant and random retrieval at the judgment and adjustment stages using the full memory.

Figure 4: PJM case study of JudgeCast and MemCast. For the same window, retrieved experience informs each method’s forecast-time decision and resulting forecast. Only judgment retrieval is shown for JudgeCast. Steps 9 to 16 highlight the interval where their forecasts differ most.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Target	# Cov.	Frequency	Train	Test	Date Range
NP	Nord Pool electricity price	2	1 hour	11,616	2,880	2017-04-30 – 2018-12-24
PJM	COMED zonal electricity price	2	1 hour	11,616	2,880	2017-04-30 – 2018-12-24
BE	Belgian electricity price	2	1 hour	11,616	2,880	2015-05-08 – 2016-12-31
FR	French electricity price	2	1 hour	11,616	2,880	2015-05-08 – 2016-12-31
DE	German electricity price	2	1 hour	11,616	2,880	2016-05-07 – 2017-12-31
ENTSO-e	Austrian electricity load	3	1 hour	34,944	8,736	2015-01-01 – 2019-12-26

Appendix

Table 6: Dataset statistics used in our experiments. Rossmann uses a fixed random subset.

Dataset	Result	Method / Covariate	0h		6h		12h
			MSE	MAE	MSE	MAE	MSE	MAE
NP	Forecasting	Target-only	25.129	2.990	25.129	2.990	25.129	2.990
		Direct cov.	15.899	2.336	23.695	2.848	25.043	2.998
		JudgeCast	19.657	2.875	20.700	2.991	21.548	3.069
	0 Judgment	Grid load	23.6%		26.7%		30.0%
		Wind	12.5%		13.2%		11.5%

Appendix

Table 7: Full results for temporal covariate misalignment. Offsets are applied to grid load on NP and PV wind on DE while the other covariate remains unchanged. Forecasting rows report MSE and MAE, and 0 Judgment rows report the covariate-wise rate of 0 judgments.

Dataset	Memory	Forecasting		Experience Construction			Avg. Judgment	Avg. Adjustment
Dataset	Memory	MSE	MAE	# Constructed	# Validated	Valid. (%)	Retrieval Distance	Retrieval Distance
NP	0%	21.514	3.096	0	0	–	–	–
	25%	20.382	2.971	121	101	83.5	0.550	0.874
	50%	19.857	2.871	238	202	84.9	0.484	0.774
	75%	20.583	2.931	354	304	85.9	0.464	0.728
	100%	19.657	2.875	477	405	84.9	0.452	0.702

Appendix

Table 8: Dataset-level results for experience accumulation and retrieval behavior. Memory indicates the fraction of training windows used for experience construction. The 0% and 100% settings denote no experience and full validated experience, respectively.

Dataset	Retrieval	Forecasting		Judgment	Adjustment
Dataset	Retrieval	MSE	MAE	Retr. Dist.	Retr. Dist.
NP	No Experience	21.514	3.096	–	–
	Random / Random	20.974	3.003	1.154	1.035
	Relevant / Random	20.576	3.001	0.452	1.037
	Random / Relevant	21.469	2.996	1.154	0.692
	Relevant / Relevant	19.657	2.875	0.452	0.702

Appendix

Table 9: Dataset-level results for selecting relevant experience. Retrieval denotes the strategies used for judgment and adjustment retrieval, respectively. Relevant uses the similarity-based retrieval criterion of JudgeCast, while Random replaces it with random selection. All retrieval settings use the full validated memory, and Relevant / Relevant corresponds to JudgeCast.

Figure 5: Case visualization on NP. Most competing forecasts underestimate the sustained high price level, whereas JudgeCast adjusts upward and remains closer to the target observation.

Figure 6: Case visualization on PJM. Most competing forecasts overestimate the price peak, whereas JudgeCast reduces this overestimation and remains closer to the target observation.

Figure 7: Representative failure cases of JudgeCast on PJM. (a) Incorrect Adjustment Direction, where the numerical adjustment moves the base forecast in the direction opposite to the observed residual. (b) Unnecessary Adjustment, where the base forecast is already close to the target observation but the numerical adjustment moves the final forecast away from it. (c) Missed Within-Horizon Reversal, where the correction needed relative to the base forecast changes direction within the forecast horizon while the applied adjustment remains largely one-sided.

Explore similar work

Expert-Guided Forecast Editing for Time-Series Foundation Models
Jul 22, 2026Hung Le, Minh Hoang Nguyen, Manh Nguyen +2Time Series Foundation Models Data-Driven Forecasting
Deakin Applied Artifical Intelligence Initiative Deakin University, Australia
Bridging the Last Mile of Time Series Forecasting with LLM Agents
Jun 1, 2026Yuhua Liao, Zetian Wang, Qiangqiang Nie +1Retrieval-Augmented Forecasting Time Series Forecasting
Trip.com Group, Shanghai, China
$t_0$ : A Time-Series Foundation Model for Forecasting with Context
Sep 21, 2026Lucas Meyer, Claudio Sole, Huikan Xiang +6Time Series Foundation Models Probabilistic Forecasting
The Forecasting Company · Macrocosm

Jul 22, 2026, cs.LGJ/K move · Enter open · S save

Expert-Guided Forecast Editing for Time-Series Foundation Models

Hung Le, Minh Hoang Nguyen, Manh Nguyen, Huu Hiep Nguyen+1

Deakin Applied Artifical Intelligence Initiative Deakin University, Australia

Time Series Foundation Models Data-Driven Forecasting Bayesian Optimization

Open paper AskPDF arXiv

Time-series foundation models can forecast across heterogeneous domains without task-specific training, but their forecasts are fixed once produced and cannot directly incorporate task-specific expert feedback. We study expert-guided forecast editing: a frozen foundation model generates candidate future trajectories, and an expensive expert evaluator scores them to guide forecast revision. Under a tight query budget, two natural strategies sit at opposite ends: best-of-

N

purely exploits the foundation model's predictive distribution, while optimization approaches mostly explore the forecast horizon as an unstructured high-dimensional vector. Each extreme is individually sub-optimal. We introduce \textbf{DEFT}, an expert-guided forecast editing framework that balances the two by first exploiting the foundation model's predictive samples in a decomposed trend--seasonal space, then exploring around them via component-wise refinement. DEFT queries the expert only on complete trajectories, then reuses scores for the trend and seasonal components that appeared in the queried recombinations. This lets each expert query provide structured component-level feedback while keeping the foundation model frozen. We compare DEFT against direct search approaches, including best-of-

N

, cross-entropy methods, and Bayesian optimization, under matched expert-query budgets. Across two forecasting benchmarks consisting of 78 datasets, three time-series foundation models, four feedback types, and seven query budgets, DEFT consistently improves the effectiveness of expert guidance. A molecular-dynamics case study further suggests that the same principle extends to more physically grounded feedback, supporting the hypothesis that sparse test-time guidance should be spent balancing prior exploitation with structured exploration.

Abstract

Explore similar work

Expert-Guided Forecast Editing for Time-Series Foundation Models

Bridging the Last Mile of Time Series Forecasting with LLM Agents

t0t_0t0​: A Time-Series Foundation Model for Forecasting with Context

$t_0$ : A Time-Series Foundation Model for Forecasting with Context