cs.LGMar 27, 2026

A Benchmark of Classical and Deep Learning Models for Agricultural Commodity Price Forecasting on A Novel Bangladeshi Market Price Dataset

Authors: Tashreef Muhammad, Tahsin Ahmed, Meherun Farzana, Md. Mahmudul Hasan, Abrar Eyasir, Md. Emon Khan, Mahafuzul Islam Shawon, Ferdous Mondol, 

Organizations: Department of Computer Science and Engineering, Southeast University, Dhaka, Bangladesh · Department of Computer Science and Engineering, University of Dhaka, Bangladesh

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Accurate short-term forecasting of agricultural commodity prices is critical for food security planning and smallholder income stabilisation in developing economies, yet machine-learning-ready datasets for this purpose remain scarce in South Asia. This paper makes two contributions. First, we introduce AgriPriceBD, a benchmark dataset of 1,779 daily retail mid-prices for five Bangladeshi commodities - garlic, chickpea, green chilli, cucumber, and sweet pumpkin - spanning July 2020 to June 2025, extracted from government reports via an LLM-assisted digitisation pipeline. Second, we evaluate seven forecasting approaches spanning classical models - naïve persistence, SARIMA, and Prophet - and deep learning architectures - BiLSTM, Transformer, Time2Vec-enhanced Transformer, and Informer - with Diebold-Mariano statistical significance tests. Commodity price forecastability is fundamentally heterogeneous: naïve persistence dominates on near-random-walk commodities. Time2Vec temporal encoding provides no statistically significant advantage over fixed sinusoidal encoding and causes catastrophic degradation on green chilli (+146.1% MAE, p<0.001). Prophet fails systematically, attributable to discrete step-function price dynamics incompatible with its smooth decomposition assumptions. Informer produces erratic predictions (variance up to 50x ground-truth), confirming sparse-attention Transformers require substantially larger training sets than small agricultural datasets provide. All code, models, and data are released publicly to support replication and future forecasting research on agricultural commodity markets in Bangladesh and similar developing economies.

Figures & tables

StudyGeo.CommodityDLLearn. Temp.Multi-commod.BD Public Retail
Hassan et al. [ 4 ]BDRice––––
Hasan et al. [ 5 ]BDRice✓–––
Imran et al. [ 6 ]BDRice✓–––
Islam et al. [ 26 ]BDRice yield✓–––
Bahar et al. [ 16 ]MYPalm oil✓–––
Nensi et al. [ 17 ]IDVegetables✓–✓–
Table 1: Gap analysis of related work. BD = Bangladesh; IN = India; MY = Malaysia; ID = Indonesia; TR = Turkey. ✓ = present; – = absent.
Figure 1: LLM-assisted dataset extraction pipeline. Daily government PDF market reports are parsed via the Gemini API using bilingual structured prompts (English and Bangla commodity name synonyms), validated against domain constraints, and aggregated into per-commodity CSV files. The forecast target is the retail mid-price: pt=(mint+maxt)/2p_{t}=(\mathrm{min}_{t}+\mathrm{max}_{t})/2 .
Figure 2: Cross-commodity Pearson correlation matrix of daily retail mid-prices (July 2020–June 2025). Garlic–chickpea show the strongest positive correlation ( r=0.61r=0.61 ), reflecting shared import-parity pricing dynamics. Cucumber shows low correlation with green chilli ( r=0.09r=0.09 ) and sweet pumpkin ( r=0.11r=0.11 ), but a moderate correlation with chickpea ( r=0.44r=0.44 ).
CommodityADF pMinMaxMeanStdStationary?R/S
Garlic0.42839.5325.0119.169.1No0.93
Chickpea0.60772.5142.590.617.8No1.32
Green Chilli0.0030.0260.087.356.2Yes0.74
Cucumber<0.001{<}0.00119.0115.045.815.8Yes1.23
Sweet Pumpkin0.00811.552.525.37.3Yes0.70
Table 2: Dataset summary statistics (July 2020 – June 2025, prices in BDT/kg).
Figure 3: Architecture comparison. (a) Vanilla Transformer uses fixed sinusoidal positional encoding computed from sequence position. (b) T2V-Transformer replaces this with Time2Vec learnable temporal embeddings parameterised by global normalised time τ∈[0,1]\tau\in[0,1] . All other components are held constant across both variants, enabling a clean controlled ablation of the temporal encoding contribution.
ModelHyperparameterValue
SARIMAOrder (p,d,q)(p,d,q)Auto (AIC minimisation, pmdarima)
 Seasonal order (P,D,Q,m)(P,D,Q,m)Auto, m=7m=7 (weekly period)
 EvaluationRolling expanding window
ProphetYearly seasonalityEnabled
 Weekly seasonalityEnabled
 HolidaysBD-specific (Eid ul-Fitr, Eid ul-Adha, Ramadan; 2020–2025)
Table 3: Hyperparameter settings for all models. DL models share the training protocol in the lower block. “Auto” = automatically selected by pmdarima’s Hyndman-Khandakar algorithm per commodity.
CommodityModelMAERMSEMAPE (%)
GarlicNaïve4.668.043.95
 SARIMA15.2824.519.73
 Prophet47.6452.9629.25
 BiLSTM5.347.014.65
 Transformer7.4910.366.39
 T2V-Transformer18.8522.7116.63
Table 4: Forecasting performance: all models, all commodities, all metrics. Bold = best overall per metric per commodity; underline = best deep learning model. All MAE and RMSE values in BDT/kg; MAPE in percent.
CommodityMAERMSEMAPE (%)PredVar (%)
Garlic6.579.347.61116.4
Chickpea2.202.692.584987.4
Green Chilli13.4016.1243.771108.2
Cucumber9.6713.4832.7340.9
Sweet Pumpkin2.182.6325.9476.9
Table 5: Informer performance on all commodities. PredVar = prediction variance as a percentage of ground-truth variance. Values near 100% indicate well-calibrated variance; values ≫\gg 100% indicate erratic oscillating predictions; values ≪\ll 100% indicate under-dispersed predictions.
CommodityTrans.T2VΔ\Delta MAETrans.T2VΔ\Delta RMSETrans.T2V
 MAEMAE RMSERMSE MAPEMAPE
Garlic7.4918.85↑\uparrow 151.6%10.3622.71↑\uparrow 119.2%6.39%16.63%
Chickpea3.5412.50↑\uparrow 253.3%3.9812.96↑\uparrow 225.9%3.35%11.81%
Green Chilli7.3818.16↑\uparrow 146.1%9.1620.58↑\uparrow 124.8%17.09%40.49%
Cucumber9.4410.91↑\uparrow 15.5%13.4213.37↓\downarrow 0.4%15.40%20.09%
Sweet Pumpkin4.176.33↑\uparrow 51.9%4.947.31↑\uparrow 48.0%23.64%39.09%
Table 6: Ablation study: Vanilla Transformer vs T2V-Transformer. All other architectural components are held constant.
CommodityDM stat.p -valueDirectionSig.
Garlic−16.378-16.378<0.001<0.001Trans. better∗∗∗
Chickpea−27.419-27.419<0.001<0.001Trans. better∗∗∗
Green Chilli−1.09×1010-1.09\times 10^{10} †<0.001<0.001Trans. better∗∗∗
Cucumber+0.047+0.0470.962T2V marginaln.s.
Sweet Pumpkin−4.119-4.119<0.001<0.001Trans. better∗∗∗
Table 7: Diebold-Mariano test (HLN corrected): T2V-Transformer vs Vanilla Transformer. 14-step-ahead, squared errors, HLN small-sample correction [ 30 ] . Positive DM stat indicates T2V-Transformer has lower loss (is more accurate). Significance: ∗∗∗p<0.01{}^{***}p<0.01 , n.s. not significant.
Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Informer prediction on the garlic test set. Despite training convergence (early stopping at epoch 50), predictions follow an erratic oscillating pattern. Garlic shows relatively moderate prediction variance (PredVar 116%); other commodities show far worse inflation (chickpea: 4987%, green chilli: 1108%), indicating that ProbSparse attention cannot learn coherent structure from this training-set size.
Figure 5: STL decomposition – Garlic (BDT/kg). Non-stationary (ADF p=0.428p=0.428 ). Trend follows a U-shape from 2020 to late 2024, reflecting a major import-supply disruption that drove retail prices sharply higher. Harvest-cycle periodicity is present (R/S ratio: 0.93), supporting exploitable periodic structure though training-scale constraints limit deep learning gains.
Figure 6: STL decomposition – Chickpea (BDT/kg). Non-stationary (ADF p=0.607p=0.607 ). Smooth, monotonically accelerating upward trend from ≈\approx 73 BDT to ≈\approx 133 BDT. Seasonal amplitude is small relative to trend; residual spike in early 2024 corresponds to the portal-outage anomaly documented in Section 3.1 . Naïve persistence dominates; R/S ratio of 1.32 reflects relatively high residual noise.
Figure 7: STL decomposition – Green Chilli (BDT/kg). Stationary (ADF p=0.003p=0.003 ). Residual amplitudes substantially exceed the trend component, reflecting exogenous shock dominance. Despite an R/S ratio of 0.74 suggesting detectable seasonality, naïve persistence dominates because price dynamics are threshold-driven (border closures, monsoon disruptions) rather than cycle-driven—a limitation of R/S as a forecastability proxy.
Figure 8: STL decomposition – Cucumber (BDT/kg). Stationary (ADF p<0.001p<0.001 ). U-shaped trend ( ≈\approx 35–65 BDT). Seasonal oscillations are present but a moderate residual-to-trend ratio (R/S: 1.23) preserves a marginal advantage for SARIMA’s differencing-based approach over deep learning models.
Figure 9: STL decomposition – Sweet Pumpkin (BDT/kg). Stationary (ADF p=0.008p=0.008 ). Bell-shaped trend peaking 2023–24. Seasonal oscillations reflect post-monsoon harvest cycles (R/S ratio: 0.70). Despite this periodic structure, DM testing confirms no statistically significant T2V advantage at this training scale.
Figure 10: Test-period forecasts – Garlic (May–June 2025). Ground truth (solid black) declines sharply in June 2025. Naïve and BiLSTM track most closely; SARIMA and Transformer diverge upward. T2V-Transformer (MAE 18.85) substantially underperforms the Vanilla Transformer (MAE 7.49) on this commodity. Prophet (not shown at this scale) predicts far below the actual retail price range.
Figure 11: Test-period forecasts – Chickpea (May–June 2025). Ground truth is near-flat ( ≈\approx 88–92 BDT/kg). Naïve persistence matches exactly by construction. Prophet diverges to ≈\approx 130 BDT, confirming inability to handle step-function dynamics. All other models cluster near naïve with small errors.
Figure 12: Test-period forecasts – Green Chilli (May–June 2025). Ground truth fluctuates between ≈\approx 28–90 BDT/kg in sharp discrete steps; all models fail to track these transitions. T2V-Transformer deviates most severely (MAE 18.16), consistent with overfitting to noise via spurious learnable frequencies.
Figure 13: Test-period forecasts – Cucumber (May–June 2025). SARIMA (MAE 8.97) is the best model overall by MAE; Transformer (MAE 9.44) is the best deep learning model by MAE. T2V-Transformer achieves a marginally better deep learning RMSE (13.37) vs Vanilla Transformer (13.42), a difference not statistically significant ( p=0.962p=0.962 ). Prophet predicts below actual prices, consistent with its smooth-trend assumption failing.
Figure 14: Test-period forecasts – Sweet Pumpkin (May–June 2025). BiLSTM (MAE 2.66) is the best deep learning model; Vanilla Transformer (MAE 4.17) is second. Prophet substantially over-predicts actual retail prices, giving MAPE of 74.56%.
Figure 15: Training curves – Garlic. All three deep learning models converge cleanly with train and validation loss tracking closely, indicating good generalisation. T2V-Transformer reaches best validation loss in ≈\approx 25 epochs under early stopping.
Figure 16: Training curves – Chickpea. T2V-Transformer shows a persistent train-val gap: train loss approaches zero while validation loss plateaus, consistent with learnable temporal parameters overfitting the near-random-walk training series.
Figure 17: Training curves – Green Chilli (dropout =0.3=0.3 ). Despite increased regularisation, persistent train-val gap across all models reflects irreducible noise rather than modelling deficiency. Dropout increase prevents further divergence but cannot recover signal absent from the training data.
Figure 18: Training curves – Cucumber. Clean convergence across all models; T2V-Transformer validation loss tracks training loss closely, supporting the RMSE improvement observed on this commodity.
Figure 19: Training curves – Sweet Pumpkin (dropout =0.3=0.3 ). Under default dropout (0.1), all deep learning models showed validation loss divergence from early epochs. Increasing dropout to 0.3 substantially stabilised training; the Vanilla Transformer achieves the best deep learning performance on this commodity (MAE 4.17 BDT/kg).
Figure 20: Ablation study – MAE comparison across BiLSTM, Vanilla Transformer, and T2V-Transformer for all five commodities. T2V-Transformer degrades on all five commodities by MAE. Garlic ( +151.6%+151.6\% ) and chickpea ( +253.3%+253.3\% ) show the most extreme degradation alongside green chilli ( +146.1%+146.1\% , p<0.001p<0.001 ).
Figure 21: Ablation study – RMSE comparison across BiLSTM, Vanilla Transformer, and T2V-Transformer for all five commodities. The sole RMSE improvement is a modest gain on cucumber ( ↓\downarrow 0.4%), which is not statistically significant ( p=0.962p=0.962 , DM test, Table 7 ).
Figure 22: Ablation study – MAPE comparison across BiLSTM, Vanilla Transformer, and T2V-Transformer for all five commodities. Four of five commodities show statistically significant Transformer superiority ( p<0.001p<0.001 ), confirming that learnable temporal encoding provides no benefit at this training scale.
More in this topic

Time Series Forecasting

Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.

See all 858 papers

Explore similar work

CardsList
May 29, 2026

Kalimati Vegetable Price Index Forecasting with a Momentum Corrected Online Stacking Ensemble

Forecasting agricultural commodity prices in emerging economies is difficult due to high volatility, frequent supply disruptions, and strong cultural influences on demand. This study introduces the Kalimati Vegetable Price Index (KVPI), a new inverse-volatility weighted composite index that aggregates 135 daily wholesale commodities from Kathmandu over ten years (2013-2023). By creating a stable macro-level signal, the KVPI reduces the noise inherent in modelling individual crops. A rich set of 64 causally valid features was developed, including festival lead-lag effects, rolling statistics, and calendar variables. Fourteen forecasting models spanning statistical, tree-based, deep learning, hybrid, and transformer architectures were rigorously evaluated across short (7-day), medium (14- and 30-day), and long-term (90-day) horizons. Tree-based ensembles proved notably robust, while classical statistical models and complex transformers struggled with the noisy dataset. The proposed Momentum-Corrected Online Stacking Ensemble achieved the strongest performance, yielding a Root Mean Square Error (RMSE) of 1.771, an exceptionally low Mean Absolute Percentage Error (MAPE) of 0.68%, and explaining 84.5% of the variance (R-squared = 0.845) at the 90-day horizon. This open-source pipeline provides policymakers and supply chain actors in Nepal and similar markets with a practical, reliable tool for anticipating price movements and strengthening food security.

Sahaj Raj Malla

Department of Mathematics, Kathmandu University, Dhulikhel, Nepal

Long-Term Time Series ForecastingTime Series ForecastingDeep Ensembles
AskPDF
Jun 28, 2026

When Prices Double in a Week: Forecasting of Agricultural Volatility in Import-Isolated Markets

Vegetable prices in Sri Lanka are highly volatile because the market is largely import-isolated, so supply disruptions quickly drive prices up. This study develops a machine learning framework to forecast such volatility by incorporating supply-chain-aware features and explicitly modelling the country's two cultivation seasons, Maha (October-April) and Yala (May-September). An integrated dataset was constructed by combining retail and farmer-gate prices with origin-aligned weather variables, diesel costs, and exchange rates across 12 vegetable varieties and 14 market centres from 2013 to 2019. A gradient-boosted ensemble model (XGBoost and LightGBM) was trained and optimised using Optuna, and unified and season-specific configurations were compared. Results show that season-specific models improve within-season fit, with the Yala-specific model achieving the highest R2 of 0.9420 (95% CI [0.690, 1.000]), while the unified model delivers the best overall predictive accuracy of 90.84% (95% CI [88.34%, 91.52%]) and an R2 of 0.9281 (95% CI [0.760, 1.000]). Notably, the unified model maintains 85.96% accuracy on a completely unseen 2024 hyperinflationary period without retraining, successfully tracking major price surges. These findings suggest that agricultural price movements in import-constrained markets are meaningfully predictable when models capture supply-chain dynamics, offering practical value for early warning and decision making by farmers, traders, and policymakers. Existing studies on Sri Lankan vegetable prices are confined to Autoregressive Integrated Moving Average (ARIMA) and Generalized Autoregressive Conditional Heteroskedasticity (GARCH) applied to single markets, with no supply-chain features, seasonal segmentation, or cross-regime validation.

Ranuga Weerasekara, Heshan Nethmina, Manuja Ranathunga +6

Dept. of Computer Science & Engineering, University of Moratuwa, Sri Lanka.

Time Series ForecastingNon-Stationary Time Series Forecasting
AskPDF
Sep 29, 2026

An Input-Frugal Deep Learning Framework for Weather-Driven National Crop-Yield Forecasting: A Case Study of Brazilian Soybean

Reliable, timely crop-yield forecasts are essential for market stability and risk management, yet many approaches rely on costly or hard-to-scale inputs. We present a frugal, transferable, and architecture-agnostic deep learning framework that uses routine weather as the only time-varying input plus two lightweight static context inputs (crop year and an agro-environmental label) to capture long-run change and regional heterogeneity, while supporting multiple sequence encoders under identical data requirements. Using a 20-season Brazilian soybean case study (2001/02-2020/21) with leave-one-year-out cross-validation, we benchmark MLP, CNN, LSTM, CNN-LSTM, a Transformer encoder and the Mamba state-space model against linear ridge regression and a five-year moving-average "farmer" baseline. All deep learning variants outperform ridge, and all sequential encoders surpass the non-sequential MLP. The Transformer achieves the best national accuracy (RMSE 149 kg ha^-1; rRMSE 5.3%; R^2 = 0.784), reducing error by 47.6% relative to the farmer baseline. In-season forecasts improve monotonically from early- to late-season issuance, reaching approximately 50% lower error than the baseline at the latest forecast point. Ablations indicate that the agro-environmental label and spatial instance expansion (multiple grid-node weather sequences per municipality-year) contribute positively without increasing input complexity. SHAP diagnostics suggest crop year explains most of the long-run trajectory, whereas within-season weather and agro-environmental context primarily drive interannual deviations, with moisture/cloud and thermal-demand variables dominating. Overall, the framework is straightforward to deploy across other crops and geographic regions and is naturally compatible with operational weather forecasts for routine monitoring.

Fernando Dupin da Cunha Mello, Prashant Kumar, Erick G. Sperandio Nascimento

Stricto Sensu Department, SENAI CIMATEC University, Salvador, Bahia, Brazil · Global Centre for Clean Air Research · Institute for Sustainability, University of Surrey, Guildford, GU2 7XH, United Kingdom +1

Multivariate Time Series ForecastingSpatiotemporal ForecastingCrop Yield Prediction
AskPDF