Beyond Similarity: Coverage-Aware Prompt Selection for Time Series Forecasting with LLMs

Daeun Ji; Minkyoung Kim; Dongkuk Kim; Yohan Lee; Beomsoo Kim; Beakcheol Jang

cs.LGSep 19, 2026

Beyond Similarity: Coverage-Aware Prompt Selection for Time Series Forecasting with LLMs

Authors: Daeun Ji, Minkyoung Kim, Dongkuk Kim, Yohan Lee, Beomsoo Kim, Beakcheol Jang

Organizations: Graduate School of Information, Yonsei University, Seoul, Republic of Korea

Abstract

Similarity-based retrieval is the dominant rule for conditioning large language models (LLMs) in in-context learning, retrieval-augmented generation, and prompt-based time series forecasting. The rule concentrates on near-duplicate candidates, an issue that has motivated diversity-aware retrieval but remains unexamined in other retrieval-conditioned pipelines. We study this issue using prompt-based time series forecasting as a test bed, where a learned prompt pool is retrieved by similarity. Dominant methods in this setting retrieve top-K entries by cosine similarity without redundancy control, producing a bias toward dominant temporal patterns while overlooking rare but informative events. We propose CASP-LLM, a coverage-aware semantic prompting framework that addresses this prompt selection bias by combining usage-tracking and saturating-gate techniques into a coverage regularizer that adds no learnable parameters. On six long-term benchmarks and the M4 short-term benchmark, CASP-LLM matches or improves on similarity-based LLM forecasters on most dataset-horizon settings, with the exceptions of Electricity, M4-Monthly, and the few-shot long-horizon setting. A controlled study locates the failure mode at the cross-batch usage level rather than per-retrieval redundancy: within-retrieval diversification such as MMR does not help, whereas regularizing anchor usage across training does.

Figures & tables

Figure 1: Selection bias in similarity-based retrieval. (a) Top- $K$ cosine similarity concentrates selection on a narrow subset of semantic anchors across training batches. (b) CASP-LLM redistributes anchor usage during training through a coverage penalty while preserving similarity-based relevance. Cell shade is proportional to selection frequency accumulated over training batches.

Figure 2: Overall architecture of the CASP-LLM framework. The framework consists of three main stages: (1) Semantic Alignment of multivariate time-series patches with the LLM semantic space, (2) Coverage-Aware Retrieval that balances semantic similarity with anchor usage across training, and (3) Prompt-Augmented Forecasting via the GPT decoder.

		Ours		Time-LLM		S 2 IP-LLM		CALF		OFA		iTransformer		PatchTST		DLinear
Dataset	H	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
Weather	96	0.1483 $\pm$ 0.0013	0.2007 $\pm$ 0.0019	0.1663	0.2200	0.1506	0.2011	0.1571	0.2071	0.1498	0.2020	0.1708	0.2212	0.1551	0.2123	0.1704	0.2298
	192	0.1935 $\pm$ 0.0007	0.2445 $\pm$ 0.0022	0.2024	0.2501	0.1941	0.2426	0.2036	0.2494	0.1944	0.2429	0.2118	0.2564	0.2122	0.2636	0.2140	0.2719
	336	0.2445 $\pm$ 0.0025	0.2821 $\pm$ 0.0036	0.2500	0.2855	0.2450	0.2810	0.2523	0.2873	0.2482	0.2848	0.2604	0.2933	0.2486	0.2857	0.2596	0.3112
	720	0.3181 $\pm$ 0.0026	0.3345 $\pm$ 0.0041	0.3197	0.3338	0.3237	0.3383	0.3322	0.3437	0.3216	0.3369	0.3341	0.3391	0.3192	0.3378	0.3225	0.3634
ETTh1	96	0.3714 $\pm$ 0.0036	0.4003 $\pm$ 0.0009	0.4136	0.4292	0.3711	0.4007	0.3863	0.4135	0.3833	0.4049	0.3981	0.4227	0.3834	0.4122	0.3682	0.3981

Table 1: Long-term forecasting results for

\{96,192,336,720\}

horizons across six multivariate benchmarks. CASP-LLM entries are three-seed means

\pm

std. Lower is better; best in bold, second-best underlined.

		$\lambda_{\text{cov}}$	0	0.01	0.1	0.5
ETTh1	96	Entropy	$2.0794$	$2.0794$	$2.0991$ $\pm$ 0.0341	$2.2535$ $\pm$ 0.1692
		Cosine	$0.9971$ $\pm$ 0.0017	$0.9985$ $\pm$ 0.0005	$0.3657$ $\pm$ 0.5483	$0.1512$ $\pm$ 0.0543
		MSE	$0.3752$ $\pm$ 0.0067	$0.3746$ $\pm$ 0.0074	$\mathbf{0.3714}$ $\pm$ 0.0036	$0.3748$ $\pm$ 0.0028
	336	Entropy	$2.0794$	$2.0794$	$2.0794$	$2.0794$
		Cosine	$0.9730$ $\pm$ 0.0291	$0.9834$ $\pm$ 0.0098	$0.9293$ $\pm$ 0.1116	$0.9923$ $\pm$ 0.0018
		MSE	$0.4451$ $\pm$ 0.0025	$0.4331$ $\pm$ 0.0156	$\mathbf{0.4263}$ $\pm$ 0.0183	$0.4438$ $\pm$ 0.0041

Table 2: Coverage weight sweep on the primary cells. Entropy is hard-assignment usage entropy on the test split, with floor

\ln K

2.079

for ETTh1 (

K=8

) and

1.386

for ETTm1 (

K=4

). Cosine is the mean pairwise cosine among selected anchor keys.

Figure 3: Selection similarity score distributions under different prompt selection mechanisms.

	ETTh1-96	ETTh1-192	ETTm1-96	ETTm1-192
w/o P+A, w/o D	0.3760	0.4193	0.2990	0.3424
w/ P+A, w/o D	0.3742	0.4094	0.2983	0.3324
w/ P+A, w/ D	0.3671	0.3973	0.2917	0.3290

Table 3: Ablation on prompt alignment (P+A) and decomposition (D). Single run per cell.

	ETTh1-96	ETTh1-192	ETTh2-96	ETTh2-192	ETTm1-96	ETTm1-192	ETTm2-96	ETTm2-192
Full (Ours)	0.3671	0.3973	0.2776	0.3422	0.2917	0.3290	0.1642	0.2235
Shuffled anchor	0.3739	0.3962	0.2801	0.3489	0.2944	0.3351	0.1659	0.2274
Random anchor	0.3812	0.4051	0.2798	0.3561	0.3024	0.3318	0.1698	0.2289

Table 4: Anchor replacement ablation. We replace the retrieved semantic anchors with similarity-shuffled or randomly sampled anchors, holding all other components fixed.

Input	Head	$\lambda_{\text{cov}}=0$	$\lambda_{\text{cov}}=0.1$	Improved	$p$
Decomposed	3-way	$0.3726$ $\pm$ 0.0050	$\mathbf{0.3702}$ $\pm$ 0.0026	7/8	$0.039$
Raw	3-way	$\mathbf{0.3663}$ $\pm$ 0.0029	$0.3717$ $\pm$ 0.0058	1/8	$0.023$
Raw	1-way	$0.3753$ $\pm$ 0.0026	$0.3753$ $\pm$ 0.0027	3/8	$1.000$

Table 5: Coverage crossed with decomposition on ETTh1-96.

Figure 4: Ablation on prompt length and pool size. Single run per cell.

Figure 5: t-SNE visualization of semantic anchors (blue), time series embeddings (orange), and prompt embeddings (green) under different coverage loss coefficients $\lambda_{\text{cov}}$ . As $\lambda_{\text{cov}}$ increases from 0 to 0.5, anchor embeddings become more dispersed, leading to a more balanced alignment of time series and prompt embeddings across the anchor space.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

	Dataset	Channels	Forecast Horizon	Length / # Series	Freq.
Long-term	Weather	21	{96, 192, 336, 720}	52696	10 min
	ETTh1	7	{96, 192, 336, 720}	17420	1 hour
	ETTh2	7	{96, 192, 336, 720}	17420	1 hour
	ETTm1	7	{96, 192, 336, 720}	69680	15 min
	ETTm2	7	{96, 192, 336, 720}	69680	15 min
	Electricity	321	{96, 192, 336, 720}	26304	1 hour

Appendix

Table 6: Statistics of datasets in long-term and short-term forecasting tasks.

Dataset	$H$	$\lambda_{\text{sim}}$	$\lambda_{\text{cov}}$	$K$	Trend	Seas.	Batch	LR
ETTh1	96	0.05	0.1	8	96	96	64	$10^{-4}$
	192	0.05	0.1	8	96	12	64	$10^{-4}$
	336	0.1	0.1	8	96	12	64	$10^{-4}$
	720	0.01	0.1	4	48	48	64	$10^{-3}$
ETTh2	96	0.1	0.05	16	96	12	128	$10^{-4}$
	192	0.1	0.05	4	96	12	128	$10^{-4}$

Appendix

Table 7: Selected hyperparameters per dataset and horizon.

Design choice	Failure mode prevented	Precedent
EMA usage statistic	Scale drift between usage and similarity	Tarvainen and Valpola (2017)
Saturating $\min$ gate	Non-saturating regularization at high usage	Lin and Bilmes (2011) ; Carbonell and Goldstein (1998)
Stop-gradient on $\tilde{c}_{i}$	Trivial similarity collapse	Tarvainen and Valpola (2017)
Training-only update	Evaluation-time drift of gate	batch-norm convention

Appendix

Table 8: Coverage mechanism design choices and their literature anchors.

Dataset	Ours		Time-LLM		S 2 IP-LLM		CALF		OFA		iTransformer		PatchTST		DLinear
	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
Weather	0.2261	0.2654	0.2346	0.2724	0.2284	0.2658	0.2363	0.2719	0.2285	0.2667	0.2443	0.2775	0.2338	0.2749	0.2416	0.2941
ETTh1	0.4299	0.4429	0.4409	0.4516	0.4154	0.4346	0.4281	0.4434	0.4265	0.4356	0.4485	0.4612	0.4600	0.4643	0.4184	0.4386
ETTh2	0.3532	0.3995	0.3686	0.4056	0.3624	0.4052	0.3666	0.4020	0.3742	0.4065	0.3846	0.4154	0.4036	0.4305	0.4995	0.4803
ETTm1	0.3480	0.3823	0.3790	0.3966	0.3509	0.3835	0.3580	0.3869	0.3596	0.3899	0.3689	0.3989	0.3770	0.4048	0.3586	0.3816
ETTm2	0.2554	0.3202	0.2649	0.3273	0.2599	0.3236	0.2628	0.3203	0.2663	0.3288	0.2717	0.3312	0.2798	0.3353	0.2768	0.3422

Appendix

Table 9: Average long-term forecasting results on benchmark datasets across all horizons. Lower MSE/MAE is better. Best in bold, second-best underlined.

		Ours	Time-LLM	S 2 IP-LLM	OFA	PatchTST	iTransformer	DLinear	Autoformer	FEDformer
Yearly	sMAPE	13.280	13.721	13.899	14.368	13.664	14.282	14.358	18.118	14.214
	MASE	3.003	3.036	3.093	3.397	3.086	3.202	3.128	4.131	3.166
	OWA	0.806	0.802	0.814	0.867	0.806	0.840	0.833	1.074	0.833
Quarterly	sMAPE	10.509	10.861	10.545	10.444	10.916	10.740	10.500	14.516	10.713
	MASE	1.204	1.293	1.239	1.237	1.289	1.280	1.239	1.856	1.282
	OWA	0.912	0.965	0.931	0.925	0.966	0.954	0.929	1.336	0.954

Appendix

Table 10: Short-term forecasting results on the M4 dataset (horizons 6–48). We report the three principal frequency splits (Yearly, Quarterly, Monthly); the Average follows the official M4 protocol and aggregates all six frequencies (Yearly through Hourly). Best in bold, second-best underlined.

		Ours		S 2 IP-LLM		Time-LLM		OFA		iTransformer		DLinear		PatchTST
Dataset	H	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTh1	96	0.437	0.451	0.518	0.491	0.747	0.545	0.570	0.516	0.837	0.609	0.565	0.538	0.598	0.524
	192	0.553	0.511	0.664	0.570	0.793	0.551	0.608	0.535	0.780	0.575	0.721	0.622	0.657	0.550
	336	0.574	0.530	0.711	0.584	0.880	0.584	0.725	0.591	1.234	0.811	0.986	0.743	0.762	0.610
	720	0.652	0.573	0.593	0.529	0.785	0.553	0.590	0.525	0.910	0.860	0.691	0.600	0.633	0.542
ETTh2	96	0.354	0.371	0.401	0.423	0.430	0.438	0.402	0.411	0.470	0.474	0.569	0.519	0.403	0.414

Appendix

Table 11: Few-shot forecasting results (10% of training data). Lower MSE/MAE is better. The best result in each row is in bold. Each cell is the average over three runs.

Figure 6: Anchor selection concentration during training across $\lambda_{\text{cov}}$ on ETTh1-96 (left) and ETTh1-336 (right). Curves plot cumulative selection counts of the top-20 most frequently selected anchors.

Figure 7: PCA projections of anchor, time-series, and prompt embeddings under varying $\lambda_{\text{cov}}$ .

	$\lambda_{\text{cov}}=0$		$\lambda_{\text{cov}}=0.01$		$\lambda_{\text{cov}}=0.1$		$\lambda_{\text{cov}}=0.5$
	PC1	PC2	PC1	PC2	PC1	PC2	PC1	PC2
Anchor embeddings	0.9850	0.0013	0.7185	0.1231	0.5583	0.4096	0.6226	0.3035
Total	0.9862		0.8416		0.9679		0.9261
Time-series embeddings	0.3932	0.3338	0.3102	0.2504	0.3162	0.2557	0.3668	0.3024
Total	0.7270		0.5606		0.5719		0.6692
Prompt embeddings	0.4161	0.3434	0.3160	0.2514	0.3223	0.2560	0.3788	0.3063

Appendix

Table 12: PCA variance ratio of PC1 and PC2 across different

\lambda_{\text{cov}}

values for anchor, time-series, and prompt embeddings.

Dataset	$H$	CASP-LLM	$\lambda_{\text{cov}}=0$	$\Delta$	$\Delta$ %
ETTh1	96	$0.3714\pm 0.0036$	$0.3752\pm 0.0067$	$-0.0038$	$-1.0$
	192	$0.4161\pm 0.0139$	$0.3987\pm 0.0087$	$+0.0174$	$+4.4$
	336	$0.4263\pm 0.0183$	$0.4451\pm 0.0025$	$-0.0189$	$-4.2$
	720	$0.5057\pm 0.0296$	$0.5063\pm 0.0495$	$-0.0007$	$-0.1$
ETTh2	96	$0.2846\pm 0.0040$	$0.2826\pm 0.0023$	$+0.0019$	$+0.7$
	192	$0.3474\pm 0.0011$	$0.3490\pm 0.0033$	$-0.0016$	$-0.4$

Appendix

Table 13: Paired comparison against

\lambda_{\text{cov}}=0

, three seeds per cell. Negative

\Delta

favors CASP-LLM.

Cell	CASP-LLM	$\lambda_{\text{cov}}=0$	Improved	$W$	$p$
ETTh1-96	$0.3702\pm 0.0026$	$0.3726\pm 0.0050$	7/8	3.0	$\mathbf{0.039}$
ETTh1-336	$0.4294\pm 0.0114$	$0.4369\pm 0.0086$	6/8	7.0	0.148
ETTm1-96	$0.2944\pm 0.0033$	$0.2936\pm 0.0030$	5/8	16.0	0.844
ETTm1-336	$0.3566\pm 0.0021$	$0.3577\pm 0.0013$	6/8	5.0	0.078

Appendix

Table 14: Eight-seed paired tests on the primary cells. Bold marks significance at

\alpha=0.05

			$\lambda_{\text{cov}}=0$	Entropy reg.	Load balance	Coverage
ETTh1	96	Entropy	$2.0794$	$2.0794$	$4.9442$ $\pm$ 0.5136	$2.0991$ $\pm$ 0.0341
	96	MSE	$0.3752$ $\pm$ 0.0067	$0.3767$ $\pm$ 0.0079	$0.3876$ $\pm$ 0.0064	$\mathbf{0.3714}$ $\pm$ 0.0036
	336	Entropy	$2.0794$	$2.0794$	$2.4155$ $\pm$ 0.1498	$2.0794$
	336	MSE	$0.4451$ $\pm$ 0.0025	$0.4366$ $\pm$ 0.0192	$0.4499$ $\pm$ 0.0038	$\mathbf{0.4263}$ $\pm$ 0.0183
ETTm1	96	Entropy	$1.3863$	$1.3863$	$5.4030$ $\pm$ 0.2108	$1.3975$ $\pm$ 0.0194
	96	MSE	$0.2937$ $\pm$ 0.0032	$0.2943$ $\pm$ 0.0027	$\mathbf{0.2927}$ $\pm$ 0.0027	$0.2965$ $\pm$ 0.0044

Appendix

Table 15: Usage-balancing objectives substituted for the coverage term on the primary cells, three paired seeds. Entropy is hard-assignment usage entropy on the test split, with floor

\ln K

2.079

for ETTh1 (

K=8

) and

1.386

for ETTm1 (

K=4

Panel of Figure 8 — Figure 8: Embedding visualization and alignment. (a) time series embeddings, (b) prefix-prompted embeddings, and (c) their cosine similarity map highlight structural consistency.

LLM Prompting

Large language model prompting studies how instructions, examples, context, and formats steer model behavior at inference time.

See all 429 papers

Forecasting With LLMs: Improved Generalization Through Feature Steering
Jun 25, 2026Humzah Merchant, Bradford LevyLook-Ahead Bias in Language Models LLM Interpretability
University of Chicago, Illinois, United States.
InA-Probe: Instruction-Aware Active Probing for Time Series Forecasting with LLMs
Jun 7, 2026Peiliang Gong, Emadeldeen Eldele, Chenyu Liu +8Zero-Shot Time Series Forecasting Time Series Forecasting
College of Computing and Data Science, Nanyang Technological University, Singapore · Department of Computer Science, Khalifa University, UAE · Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education, College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China +1
Rethinking the Role of LLMs in Time Series Forecasting
Feb 16, 2026Xin Qiu, Junlong Tong, Yirong Sun +3LLM-Based Time Series Forecasting Time Series Forecasting
Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative, Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Zhejiang University · LMU Munich

Jun 25, 2026J/K move · Enter open · S save

Forecasting With LLMs: Improved Generalization Through Feature Steering

Humzah Merchant, Bradford Levy

University of Chicago, Illinois, United States.

Look-Ahead Bias in Language Models LLM Interpretability Temporal Reasoning in Language Models Language Model Steering LLM-Based Time Series Forecasting Causal Interventions in Language Models+1

Open paper AskPDF arXiv

Successful forecasting involves identifying patterns between historical and future states of the world which generalize to future observations. We apply LLMs to a variety of forecasting tasks and inspect their internal states using sparse autoencoders to understand whether they appear to rely on time-specific pieces of knowledge versus generalizable patterns. Our analyses identify features associated with both time-aware reasoning and look-ahead-biased reasoning. We then apply the LLMs to an entirely different domain and intervene on these features. We find that amplifying time-awareness features substantially reduces look-ahead bias on forecasting prompts while preserving general reasoning performance. In contrast, steering the candidate look-ahead-bias features does not produce an effect. These results suggest that interpretable temporal features can be used to causally shift LLMs toward more historically grounded reasoning.

Abstract

LLM Prompting

Explore similar work

Forecasting With LLMs: Improved Generalization Through Feature Steering

InA-Probe: Instruction-Aware Active Probing for Time Series Forecasting with LLMs

Rethinking the Role of LLMs in Time Series Forecasting