Cryptocurrency markets exhibit extreme volatility and non-stationary dynamics that challenge conventional forecasting methods. Although Large Language Models (LLMs) have shown promise for time series forecasting, the combined effects of adaptation choices remain largely unexplored in financial settings. This study introduces PRICE, a structured approach for adapting LLMs to short-term Bitcoin price forecasting. Built on a 4-bit quantized LLaMA-3 8B model, PRICE investigates how fine-tuning, numerical representation, prompting, inference, and decoding jointly influence forecasting performance. PRICE integrates Parameter-efficient fine-tuning with Low-Rank Adaptation (LoRA), Recursive multi-step inference, Integer-rounded numerical representation, Context-Task-Format (CTF) prompting, and Exact zero-temperature decoding. Ablation studies show that each component contributes to forecasting accuracy and reliability. LoRA enables efficient training on limited hardware, recursive inference improves accuracy, integer-rounded values reduce errors, CTF prompting outperforms Chain-of-Thought, Implicit Chain-of-Thought (iCoT), and few-shot prompting, and zero-temperature decoding improves stability during recursive forecasting. Comparative evaluation against eight transformer-based and time-series foundation models shows that PRICE achieves the lowest forecasting errors on both validation and test sets while maintaining robust performance across evaluation periods. Despite being based on a model primarily pretrained on text rather than time-series data, PRICE achieves competitive or superior performance relative to specialized foundation models. These findings demonstrate that adaptation choices critically determine the accuracy and robustness of LLMs for numerical time-series forecasting.
Figures & tables
Figure 1: Overall workflow of the PRICE framework for adapting a quantized LLaMA-3 8B model to short-term Bitcoin price forecasting.
Figure 2: Stages of instruction preparation for Context-Task-Format (CTF) prompting.
Figure 3: Final Context-Task-Format (CTF) forecasting prompt used in PRICE.
Timestamp
Open
High
Low
Close
Volume
2024-06-01 00:00:00
67540.01
67585.89
67540.00
67573.41
11.67408
2024-06-01 00:01:00
67573.41
67586.35
67561.40
67564.65
8.17416
2024-06-01 00:02:00
67564.65
67582.90
67561.40
67582.90
9.24164
Table 1: Example format of the constructed BTC/USDT dataset.
Subset
Start Time
End Time
Ratio
Training
2023-03-06 01:00
2024-08-01 16:00
80%
Validation
2024-08-01 16:00
2024-10-05 00:00
10%
Test
2024-10-05 00:00
2024-12-08 08:00
10%
Table 2: Chronological data split used for training, validation, and testing.
Hyperparameter
Value
Learning rate
5×10−5
LR scheduler
Cosine
Warmup ratio
0.1
Training epochs
2
Batch size (per device)
1
Gradient accumulation
4
Table 3: Fine-tuning hyperparameters
Figure 4: Fixed zero-shot prompt used to evaluate the effect of input sequence representation.
Format
MAPE
SMAPE
NRMSE
VNP
Example
Raw
4.23
5.42
0.12
1441
54504.0606666667
Standard Normalization
9.43
9.39
0.16
1441
0.8423
Instance Normalization
548.36
132.19
1.41
1441
-1.52752
Rounded
3.01
3.05
0.05
1441
54504
Table 4: Forecasting performance under different input sequence representations using MAPE, SMAPE, NRMSE, and VNP. Example inputs for each representation are shown in the last column.
Figure 5: Single-step forecasting prompt.
Figure 6: Multi-Step forecasting prompt.
Paradigm + Strategy
MAE
MSE
VNP
Zero-shot + Single-step
2063.62
7,405,313.89
1182
Zero-shot + Multi-step
1779.88
9,264,283.10
1441
Fine-tuned + Single-step
534.85
3,263,720.91
1440
Fine-tuned + Multi-step
445.60
423,562.43
1441
Table 5: Comparison of forecasting performance across learning paradigms and forecasting strategies using MAE, MSE, and VNP.
Table 6: Forecasting performance comparison across prompt engineering strategies under fine-tuned and zero-shot inference settings. Lower MAE and MSE indicate better performance, whereas higher VNP indicates greater output validity. Best results within each configuration group are highlighted in bold.
Temperature
MAE ↓
MSE ↓
VNP ↑
0.0
415.89
385,195.52
1441
0.2
417.32
386,571.30
1441
0.6
422.83
391,328.30
1441
0.8
426.77
397,966.41
1441
0.95
432.48
404,533.49
1441
1.0
434.25
405,248.15
1441
Table 7: Effect of temperature on forecasting performance under the fine-tuned multi-step forecasting setting using the CTF prompt. Lower MAE and MSE indicate better performance, whereas higher VNP indicates greater output validity. Best results are highlighted in bold.
Method
Validation Set
Test Set
MAE
MSE
MAPE
SMAPE
NRMSE
MAE
MSE
MAPE
SMAPE
NRMSE
Transformer
0.05
0.0045
3.57
3.51
0.04
0.29
0.17
9.97
10.90
0.18
Autoformer
0.09
0.01
6.08
6.08
0.08
0.10
0.0178
4.78
4.85
0.06
Crossformer
0.12
0.02
8.57
8.07
0.10
0.65
0.78
22.74
27.60
0.38
Pyraformer
0.04
0.0030
2.77
2.74
0.04
0.35
0.23
11.71
12.99
0.21
PatchTST
0.03
0.0022
2.23
2.23
0.03
0.04
0.0035
1.78
1.79
0.03
Table 8: Performance comparison between PRICE and the evaluated transformer-based and time-series foundation models for multi-step forecasting ( pred_len=8 ) on the validation and test sets. Lower values indicate better performance. Best results are highlighted in bold.
Recent advances in Large Language Models (LLMs) have spurred cross-modal solutions for time-series forecasting. However, existing methods rely heavily on textual prompts for modality alignment-introducing nontrivial computational overhead and failing to leverage the rich spectral dynamics inherent in time-series data. To enable prompt-free, frequency-aware adaptation of frozen LLMs, we propose FM-LLM (Frequency-Enhanced Mixture-of-Experts for adapting LLMs to Time Series Forecasting), an autoregressive framework grounded in constrained asymmetric coupling. A Fourier Analysis Network (FAN)-based spectral token aligner injects structured harmonic representations directly into the frozen LLM with numerical compatibility. An asymmetric Mixture-of-Experts (MoE) decoder enforces role separation: shared experts with lightweight FAN layers reconstruct the global periodic backbone, while routed experts-restricted to standard FFNs-specialize in modeling non-periodic residual dynamics. A time-frequency hybrid loss function jointly optimizes temporal accuracy and spectral consistency, mitigating error accumulation during long-horizon autoregressive rollouts. Evaluated across eleven public benchmarks, FM-LLM achieves state-of-the-art performance on 59 out of 78 evaluation metrics. Compared to the strongest autoregressive LLM-based baseline, it delivers average improvements of 5.3% in MSE and 5.6% in MAE, with maximum gains reaching 8.0% for MSE and 8.4% for MAE. FM-LLM also demonstrates robust transferability, maintaining superior performance in 10% few-shot and zero-shot forecasting scenarios.
Rentao Gu, Yihang Ding, Junjie Li +5
Beijing University of Posts and Telecommunications, Beijing, 100876, China · China Telecom Research Institute, Beijing, 100032, China
Forecasting cryptocurrency prices remains a formidable challenge due to inherent non-stationarity, abrupt regime shifts, and multi-scale stochastic dependencies. Conventional deep learning models often struggle to capture complex underlying dynamics, frequently resulting in persistent phase-lagged predictions. To address these limitations, we propose MoFE, a novel deep learning framework that integrates Fourier Neural Operators (FNOs) within a Mixture-of-Experts (MoE) architecture. Rooted in the theoretical framework of stochastic differential equations, MoFE conceptualizes cryptocurrency volatility as a superposition of multi-frequency components, which includes user network based fundamental growth, mining costs and halving mechanism caused seasonal volatility, and market sentiment-induced chaos. Specifically, specialized adaptive FNO (AFNO) and Convolution dual-domain experts learn continuous function-to-function mappings to encapsulate global spectral trends, cyclical adjustments and microstructures, while a dynamic gating based MoE mechanism enables adaptive strategy switching across diverse market regimes. Extensive experiments on Bitcoin datasets spanning January 2020 to December 2025 demonstrate that MoFE achieves state-of-the-art (SOTA) performance in both T+1 and T+5 forecasting horizons. Notably, the model effectively mitigates the phase-lag effect, delivering superior Directional Accuracy (DA) and Information Coefficient (IC). In high-fidelity simulated trading environments, these predictive gains transfer into significant excess returns and robust risk-adjusted performance, characterized by a high Sharpe ratio.
Bowen Liu, Mingming Sun
School of Art and Science University of Rochester Rochester, USA · AGI Lab Beijing Institute of Mathematical Sciences and Applications Beijing, China
Bitcoin price prediction on sub-daily timescales is a hard open problem in computational finance. Bitcoin exhibits fat-tailed returns, non-stationary dynamics, and a price discovery process influenced by social discourse on Reddit and Twitter. Conventional approaches fuse OHLCV technical features with sentiment via static concatenation, applying identical fusion weights regardless of market state. This is inconsistent with the behavioural finance literature, which shows that retail sentiment is most predictive during volatile periods and noisy during calm ones. This paper proposes Regime-Aware Multi-Modal Learning (RAML), which conditions fusion of sentiment and price features on a dynamically detected binary market regime. Rolling 24-hour volatility partitions observations into stable and volatile regimes; a learnable sigmoid gate adjusts the weight of the sentiment embedding relative to the price embedding, trusting sentiment more during volatility and price dynamics more during stable phases. The system is evaluated on 3,491 hourly observations (July 2024-September 2025), combining Bitcoin OHLCV data with Reddit /r/Bitcoin FinBERT sentiment. Four models are compared - price-only BiLSTM, sentiment-only classifier, static-concatenation BiLSTM, and RAML - across 3-hour and 6-hour horizons, with an ablation study isolating the sentiment branch, regime detection, and adaptive fusion. RAML achieves macro-F1 of 0.5474 (3h) and 0.5513 (6h), with the highest AUC at 3 hours (0.5084), indicating better calibration. Ablation confirms every component is necessary, and replacing adaptive weighting with concatenation causes recall collapse at 6 hours (F1: 0.14). These results establish regime-conditioned adaptive fusion as a necessary design principle for multi-modal financial forecasting.
Muhammad Abdullah Haroon
Department of Computer Science, FAST National University of Computer and Emerging Sciences (FAST-NUCES), Lahore, Pakistan