Benchmarking Scientific Machine Learning Models for Air Quality Data

Khawja Imran Masud; Venkata Sai Rahul Unnam; Sahara Ali

cs.LGMar 22, 2026

Benchmarking Scientific Machine Learning Models for Air Quality Data

Authors: Khawja Imran Masud, Venkata Sai Rahul Unnam, Sahara Ali

Organizations: University of North Texas, 76205 Texas, USA

Abstract

Accurate air quality index (AQI) forecasting is essential for the protecting public health in rapidly growing urban regions, and the practical model evaluation and selection are often challenged by the lack of rigorous, region-specific benchmarking on standardized datasets. Physics-guided machine learning and deep learning models could be a good and effective solution to resolve such issues with more accurate and efficient AQI forecasting. This research study presents an explainable and comprehensive benchmark that enables a guideline and proposed physics-guided best model by benchmarking classical time-series, machine-learning, and deep-learning approaches for multi-horizon AQI forecasting in North Texas (Dallas County). Using publicly available U.S. Environmental Protection Agency (EPA) daily observations of air quality data from 2022 to 2024, we curate city-level time series for PM2.5 and O3 by aggregating station measurements and constructing lag-wise forecasting datasets for LAG in {1,7,14,30} days. For benchmarking the best model, linear regression (LR), SARIMAX, multilayer perceptrons (MLP), and LSTM networks are evaluated with the proposed physics-guided variants (MLP+Physics and LSTM+Physics) that incorporate the EPA breakpoint-based AQI formulation as a consistency constraint through a weighted loss. Experiments using chronological train-test splits and error metrics MAE, RMSE showed that deep-learning models outperform simpler baselines, while physics guidance improves stability and yields physically consistent pollutant with AQI relationships, with the largest benefits observed for short-horizon prediction and for PM2.5 and O3. Overall, the results provide a practical reference for selecting AQI forecasting models in North Texas and clarify when lightweight physics constraints meaningfully improve predictive performance across pollutants and forecast horizons.

Figures & tables

Related Study	Model Architecture	Datasets	Task	Results	Tentative Research Gap
Li et al. [ 12 ]	Random Forest regression	Huaihai Economic Zone, China (2000–2020)	Daily PM 2.5 estimation	$R^{2}=0.85$ , RMSE = 14.63 $\mu$ g/m 3 , MAE = 10.03 $\mu$ g/m 3	No physics; cannot capture long-range temporal interactions.

Ravindiran et al. [ 13 ]	CatBoost / ensemble ML	Visakhapatnam, India (2017–2022)	AQI forecasting	$R^{2}\approx 0.9998$ , RMSE $\approx 0.76$	Extremely high fit; may overfit; relies entirely on historical correlations.

Yenkikar et al. [ 15 ]	Random Forest + ARIMA residual modeling	India (station-level data)	AQI forecasting	MSE $\approx$ 508.46; $R^{2}\approx 0.94$	No DL; limited temporal modeling; no physical knowledge.

Table 1: Comparison of major ML, DL, hybrid, and physics-informed approaches for air-quality prediction and forecasting.

Statistic	PM 2.5 Dataset (2022–2024)	O 3 Dataset (2022–2024)
Number of daily records	1094	1091
AQI (min)	6.00	2.67
AQI (max)	124.00	168.33
AQI (mean $\pm$ std)	46.18 $\pm$ 14.90	41.82 $\pm$ 20.78
Pollutant value (min)	0.45 $\mu$ g/m 3	0.0027 ppm
Pollutant value (max)	44.85 $\mu$ g/m 3	0.0927 ppm

Table 2: Summary statistics of the base PM 2.5 and O 3 datasets used for constructing multi-horizon AQI targets.

PM 2.5 Data							O 3 Data
DATE	PM 2.5	AQI	L1	L7	L14	L30	O 3	AQI	L1	L7	L14	L30
01-01-22	6.10	34.00	26.00	61.00	27.00	19.00	0.0250	23.00	30.33	15.67	26.67	31.33
02-01-22	4.60	26.00	52.00	23.00	55.00	24.50	0.0323	30.33	27.67	24.33	29.00	33.00
03-01-22	9.60	52.00	29.00	31.00	64.25	21.00	0.0297	27.67	33.33	22.00	28.67	27.67
04-01-22	5.30	29.00	32.67	49.00	37.00	6.00	0.0360	33.33	30.67	26.67	40.67	28.67
05-01-22	5.90	32.67	39.00	35.00	44.00	41.33	0.0330	30.67	23.00	29.67	23.33	34.00

Table 3: First 10 samples of the lag-wise PM 2.5 and O 3 datasets across multiple forecasting horizons.

Figure 1: Proposed conceptual model for Physics Guided Scientific Machine Learning for Air Quality Index Forecasting.

Figure 2: Architecture of the MLP+Physics Model.

Figure 3: Architecture of the LSTM+Physics Model.

Hyperparameter	LR	SARIMAX	MLP	MLP+Physics	LSTM	LSTM+Physics
Architecture / Order	OLS	(1,1,1) $\times$ (1,0,1,7)	64–32	64–32	LSTM(32,32)+MLP	LSTM(32,32)+MLP
Activation	–	–	ReLU	ReLU	ReLU	ReLU
Optimizer	–	–	AdamW	Adam	Adam	Adam
Learning rate	–	–	0.001	0.001	0.001	0.001
Batch size	–	–	Full	Full	32	32
Scaling method	None	None	StandardScaler	StandardScaler	MinMaxScaler [ $-1,1$ ]	MinMaxScaler [ $-1,1$ ]

Table 4: Generalized hyperparameter settings for all evaluated models

Figure 4: Multi-horizon AQI target visualization for PM 2.5 (top row) and O 3 (bottom row). Each subplot compares the current-day AQI (t) with future AQI values at 1, 7, 14, and 30-day LAG’s prediction.

Model	LAG 1			LAG 7			LAG 14			LAG 30
Model	MAE	RMSE	NMSE	MAE	RMSE	NMSE	MAE	RMSE	NMSE	MAE	RMSE	NMSE
LR	8.2803	10.7370	0.6646	10.2012	13.1408	0.9986	10.1016	12.6088	0.9894	10.2011	12.7831	1
SARIMAX	9.8861	12.9285	0.8967	31.7073	33.8115	0.9994	48.4128	50.1106	1	12.4783	15.2045	0.9999
MLP	8.2149	10.6535	0.6583	10.2289	13.1867	1.0305	10.0839	12.6538	1.0185	10.2548	12.8299	1.0464
LSTM	8.1761	10.6289	0.6552	10.0669	12.9638	0.9960	9.9955	12.5401	1.0003	9.9895	12.5433	1.0002

Table 5: Baseline performance of LR, SARIMAX, MLP, and LSTM models for PM 2.5 AQI forecasting across all lag-wise datasets.

Model	LAG 1			LAG 7			LAG 14			LAG 30
Model	MAE	RMSE	NMSE	MAE	RMSE	NMSE	MAE	RMSE	NMSE	MAE	RMSE	NMSE
LR	12.8202	20.2004	0.6357	17.1723	25.9813	1	16.1777	24.0898	0.8936	17.7215	25.5043	0.9892
SARIMAX	15.5971	22.6346	0.8065	16.997	25.4225	0.9910	17.8621	25.4614	1	29.7733	33.3162	1
MLP	12.8785	20.2860	0.6451	17.4830	26.4023	1.0877	16.0078	23.8830	0.8860	17.4687	25.2393	0.9768
LSTM	12.7642	19.4974	0.5959	16.7570	25.2099	0.9917	16.3613	23.9691	0.8924	17.7549	25.3641	0.9865

Table 6: Baseline performance of LR, SARIMAX, MLP, and LSTM models for O 3 AQI forecasting across all lag-wise datasets.

Model	$\lambda_{\text{data}}$	$\lambda_{\text{phys}}$	LAG 1			LAG 7			LAG 14			LAG 30
			MAE	RMSE	NMSE	MAE	RMSE	NMSE	MAE	RMSE	NMSE	MAE	RMSE	NMSE
MLP+Physics	0.0	1.0	8.1695	10.6349	0.6560	10.1783	13.1374	1.0228	10.0925	12.6804	1.0228	10.2176	12.7926	1.0404
MLP+Physics	0.3	0.7	8.1826	10.6395	0.6565	10.1934	13.1508	1.0249	10.1092	12.6937	1.0249	10.2260	12.8006	1.0417
MLP+Physics	0.5	0.5	8.1918	10.6430	0.6570	10.2015	13.1591	1.0262	10.0649	12.6396	1.0162	10.2332	12.8081	1.0429
MLP+Physics	0.7	0.3	8.2017	10.6477	0.6576	10.2152	13.1725	1.0283	10.0713	12.6445	1.0170	10.2399	12.8143	1.0439
MLP	1.0	0.0	8.2149	10.6535	0.6583	10.2289	13.1867	1.0305	10.0839	12.6538	1.0185	10.2548	12.8299	1.0464

Table 7: Physics-guided MLP performance for PM 2.5 across forecasting horizons with different

\lambda_{\text{data}}

and

\lambda_{\text{phys}}

settings.

Model	$\lambda_{\text{data}}$	$\lambda_{\text{phys}}$	LAG 1			LAG 7			LAG 14			LAG 30
			MAE	RMSE	NMSE	MAE	RMSE	NMSE	MAE	RMSE	NMSE	MAE	RMSE	NMSE
MLP+Physics	0.0	1.0	12.9227	20.2878	0.6452	17.5359	26.4362	1.0905	16.0406	23.8743	0.8853	17.5058	25.2256	0.9757
MLP+Physics	0.3	0.7	12.9086	20.2862	0.6451	17.5197	26.4273	1.0898	16.0338	23.8792	0.8857	17.4972	25.2312	0.9762
MLP+Physics	0.5	0.5	12.9000	20.2855	0.6451	17.5082	26.4238	1.0895	16.0278	23.8808	0.8858	17.4954	25.2322	0.9763
MLP+Physics	0.7	0.3	12.8913	20.2854	0.6451	17.4972	26.4139	1.0887	16.0212	23.8823	0.8859	17.4841	25.2338	0.9764
MLP	1.0	0.0	12.8785	20.2860	0.6451	17.4830	26.4023	1.0877	16.0078	23.8830	0.8860	17.4687	25.2393	0.9768

Table 8: Physics-guided MLP performance for O 3 across forecasting horizons with different

\lambda_{\text{data}}

and

\lambda_{\text{phys}}

settings.

Panel of Figure 5 — Figure 5: Time-series comparison of true and predicted AQI values on the test set using the physics-guided MLP model for (a) PM 2.5 and (b) O 3 . The plots show the last portion of the test period to highlight short-term forecasting behavior.

Model	$\lambda_{\text{data}}$	$\lambda_{\text{phys}}$	LAG 1			LAG 7			LAG 14			LAG 30
			MAE	RMSE	NMSE	MAE	RMSE	NMSE	MAE	RMSE	NMSE	MAE	RMSE	NMSE
LSTM+Physics	0.0	1.0	8.1558	10.6348	0.6560	10.0467	12.9498	0.9938	9.9795	12.5423	1.0006	9.9896	12.5433	1.0002
LSTM+Physics	0.3	0.7	8.1943	10.6364	0.6562	10.0305	12.9414	0.9925	9.9807	12.5406	1.0004	9.9894	12.5433	1.0002
LSTM+Physics	0.5	0.5	8.1704	10.6322	0.6556	10.0403	12.9432	0.9928	9.9815	12.5398	1.0002	9.9902	12.5435	1.0002
LSTM+Physics	0.7	0.3	8.1639	10.6237	0.6546	10.0622	12.9576	0.9950	10.0042	12.5447	1.0010	9.9909	12.5438	1.0003
LSTM	1.0	0.0	8.1761	10.6289	0.6552	10.0669	12.9638	0.9960	9.9955	12.5401	1.0003	9.9895	12.5433	1.0002

Table 9: Physics-guided LSTM performance for PM 2.5 across forecasting horizons with different

\lambda_{\text{data}}

and

\lambda_{\text{phys}}

settings.

Model	$\lambda_{\text{data}}$	$\lambda_{\text{phys}}$	LAG 1			LAG 7			LAG 14			LAG 30
			MAE	RMSE	NMSE	MAE	RMSE	NMSE	MAE	RMSE	NMSE	MAE	RMSE	NMSE
LSTM+Physics	0.0	1.0	12.7986	19.4895	0.5954	16.6970	25.2380	0.9939	16.4360	23.9733	0.8927	17.7256	25.3531	0.9856
LSTM+Physics	0.3	0.7	12.7856	19.5148	0.5970	16.6655	25.2851	0.9976	16.4991	23.9886	0.8938	17.8042	25.3755	0.9874
LSTM+Physics	0.5	0.5	12.8550	19.4646	0.5939	16.6616	25.2858	0.9977	16.4206	23.9859	0.8936	17.7970	25.3737	0.9872
LSTM+Physics	0.7	0.3	12.8602	19.4922	0.5956	16.6410	25.3056	0.9992	16.4376	23.9728	0.8926	17.7830	25.3708	0.9870
LSTM	1.0	0.0	12.7642	19.4974	0.5959	16.7570	25.2099	0.9917	16.3613	23.9691	0.8924	17.7549	25.3641	0.9865

Table 10: Physics-guided LSTM performance for O 3 across forecasting horizons with different

\lambda_{\text{data}}

and

\lambda_{\text{phys}}

settings.

Panel of Figure 6 — Figure 6: Time-series comparison of true and predicted AQI values on the test set using the physics-guided LSTM model for (a) PM 2.5 and (b) O 3 .

Air Quality Prediction

Air quality prediction estimates future pollutant levels and maps local exposure to support health warnings, planning, and pollution control.

See all 42 papers

AirQualityBench: A Realistic Evaluation Benchmark for Global Air Quality Forecasting
May 7, 2026Xing Xu, Xu Wang, Yudong Zhang +3Benchmark Design Time Series Forecasting
University of Science and Technology of China (USTC), Hefei, Anhui, China · 2Suzhou Institute for Advanced Research, USTC, Suzhou, Jiangsu, China · 3The Hong Kong Polytechnic University, Hong Kong, China
Interpretable PM2.5 Forecasting for Urban Air Quality: A Comparative Study of Adaptive Time-Series Models
Mar 26, 2026Moazzam Umer Gondal, Hamad ul Qudous, Asma Ahmad Farhan +1Air Quality Prediction Time Series Forecasting
School of Computing, National University of Computer & Emerging Sciences (FAST), Lahore 54000, Pakistan · College of Computing and Informatics, Saudi Electronic University, Riyadh, 11673, Saudi Arabia
AirDDE: Multifactor Neural Delay Differential Equations for Air Quality Forecasting
Mar 18, 2026Binqing Wu, Zongjiang Shang, Shiyu Liu +3Air Quality Prediction Multivariate Time Series Forecasting
State Key Laboratory of Blockchain and Data Security, Zhejiang University · College of Computer Science and Technology, Zhejiang University · State Key Laboratory of Clean Energy Utilization, Zhejiang University

May 7, 2026J/K move · Enter open · S save

AirQualityBench: A Realistic Evaluation Benchmark for Global Air Quality Forecasting

Xing Xu, Xu Wang, Yudong Zhang, Huilin Zhao+2

University of Science and Technology of China (USTC), Hefei, Anhui, China · 2Suzhou Institute for Advanced Research, USTC, Suzhou, Jiangsu, China · 3The Hong Kong Polytechnic University, Hong Kong, China

Benchmark Design Time Series Forecasting Forecasting Benchmarks Air Quality Prediction Spatiotemporal Forecasting

Open paper AskPDF arXiv

Air-quality forecasting models are commonly evaluated on regional, preprocessed, and normalized datasets, where missing observations are removed or artificially completed. Such protocols simplify comparison but hide the conditions that dominate real monitoring networks: uneven global coverage, structured missingness, heterogeneous pollutant scales, and deployment cost. We introduce AirQualityBench, a global multi-pollutant benchmark designed to evaluate forecasting models under these realistic conditions. The benchmark contains hourly observations from 3,720 monitoring stations over 2021--2025, covers six major pollutants, and preserves provider-native observation masks. Rather than imputing a dense data tensor, AirQualityBench exposes missingness as part of the forecasting problem and reports errors on valid future observations after inverse transformation to physical concentration scales. Evaluating representative spatio-temporal models under this unified protocol shows that strong performance on sanitized datasets does not reliably transfer to global, fragmented monitoring streams. AirQualityBench therefore serves as a realistic testbed for scalable, mask-aware, and physically interpretable air-quality forecasting. All benchmark data, code, evaluation scripts, and baseline implementations are available at GitHub.

Abstract

Air Quality Prediction

Explore similar work

AirQualityBench: A Realistic Evaluation Benchmark for Global Air Quality Forecasting

Interpretable PM2.5 Forecasting for Urban Air Quality: A Comparative Study of Adaptive Time-Series Models

AirDDE: Multifactor Neural Delay Differential Equations for Air Quality Forecasting