Meteorology-Driven GPT4AP: A Multi-Task Forecasting LLM for Atmospheric Air Pollution in Data-Scarce Settings
Organizations: ADAPT SFI Research Centre, School of Computer Science, Technological University Dublin, Ireland · ADAPT SFI Research Centre, School of Computer Science, University College Dublin · School of Computer Science, Technological University Dublin, Ireland
Abstract
Accurate forecasting of air pollution is important for environmental monitoring and policy support, yet data-driven models often suffer from limited generalization in regions with sparse observations. This paper presents Meteorology-Driven GPT for Air Pollution (GPT4AP), a parameter-efficient multi-task forecasting framework based on a pre-trained GPT-2 backbone and Gaussian rank-stabilized low-rank adaptation (rsLoRA). The model freezes the self-attention and feed-forward layers and adapts lightweight positional and output modules, substantially reducing the number of trainable parameters. GPT4AP is evaluated on six real-world air quality monitoring datasets under few-shot, zero-shot, and long-term forecasting settings. In the few-shot regime using 10% of the training data, GPT4AP achieves an average MSE/MAE of 0.686/0.442, outperforming DLinear (0.728/0.530) and ETSformer (0.734/0.505). In zero-shot cross-station transfer, the proposed model attains an average MSE/MAE of 0.529/0.403, demonstrating improved generalization compared with existing baselines. In long-term forecasting with full training data, GPT4AP remains competitive, achieving an average MAE of 0.429, while specialized time-series models show slightly lower errors. These results indicate that GPT4AP provides a data-efficient forecasting approach that performs robustly under limited supervision and domain shift, while maintaining competitive accuracy in data-rich settings.
Figures & tables
| Hyperparameter | Value |
| Backbone | GPT-2 (pre-trained) |
| Fine-tuning method | rsLoRA |
| rsLoRA initialization | Gaussian |
| Optimizer | Adam |
| Rank | 32 |
| Learning rate |
| Dataset | Horizon | GPT4AP | DLinear | ETSformer | FiLM | Informer | Pyraformer | Transformer |
| AZ | 24 | 0.546/0.338 | 0.628/0.491 | 0.589/0.440 | 0.657/0.442 | 0.853/0.592 | 0.724/0.544 | 0.766/0.556 |
| 36 | 0.633/0.431 | 0.692/0.525 | 0.685/0.492 | 0.701/0.466 | 1.147/0.742 | 0.824/0.585 | 0.874/0.615 | |
| 48 | 0.686/0.457 | 0.724/0.543 | 0.756/0.528 | 0.747/0.486 | 1.014/0.718 | 0.930/0.642 | 0.938/0.658 | |
| 60 | 0.731/0.477 | 0.753/0.557 | 0.788/0.552 | 0.784/0.500 | 1.254/0.787 | 0.901/0.634 | 1.029/0.693 | |
| Avg | 0.649 / 0.426 | 0.699 /0.529 | 0.705/0.503 | 0.722/ 0.474 | 1.067/0.710 | 0.845/0.601 | 0.902/0.631 | |
| DS | 24 | 0.662/0.400 | 0.725/0.497 | 0.694/0.447 | 0.788/0.458 | 0.896/0.571 | 0.786/0.530 | 0.803/0.535 |
| Dataset | Horizon | GPT4AP | DLinear | ETSformer | FiLM | Informer | Pyraformer | Transformer |
| AZ | 24 | 0.530/0.376 | 0.532/0.409 | 0.525/0.397 | 0.557/0.393 | 0.470/0.357 | 0.467/0.360 | 0.456/0.366 |
| 36 | 0.613/0.419 | 0.605/0.455 | 0.611/0.443 | 0.634/0.433 | 0.565/0.403 | 0.548/0.403 | 0.607/0.428 | |
| 48 | 0.671/0.447 | 0.649/0.482 | 0.656/0.486 | 0.685/0.457 | 0.615/0.434 | 0.594/0.441 | 0.607/0.433 | |
| 60 | 0.712/0.467 | 0.684/0.503 | 0.705/0.515 | 0.726/0.476 | 0.644/0.449 | 0.650/0.464 | 0.688/0.467 | |
| Avg | 0.632/0.427 | 0.618/0.462 | 0.624/0.460 | 0.651/0.440 | 0.574 / 0.411 | 0.565 / 0.417 | 0.590/0.424 | |
| DS | 24 | 0.636/0.389 | 0.636/0.420 | 0.612/0.398 | 0.668/0.405 | 0.578/0.385 | 0.579/0.390 | 0.597/0.395 |
| Dataset | Horizon | GPT4AP | DLinear | ETSformer | FiLM | Informer | Pyraformer | Transformer |
| AZ DS | 24 | 0.455/0.351 | 0.536/0.553 | 0.522/0.556 | 0.574/0.563 | 0.623/0.594 | 0.442/0.488 | 0.640/0.585 |
| 36 | 0.502/0.380 | 0.616/0.590 | 0.591/0.579 | 0.691/0.612 | 0.679/0.617 | 0.510/0.537 | 0.646/0.620 | |
| 48 | 0.541/0.403 | 0.690/0.622 | 0.686/0.621 | 0.795/0.654 | 0.627/0.596 | 0.572/0.576 | 0.588/0.567 | |
| 60 | 0.583/0.424 | 0.753/0.649 | 0.789/0.660 | 0.894/0.694 | 0.719/0.677 | 0.618/0.617 | 0.601/0.596 | |
| Avg | 0.520 / 0.390 | 0.649/0.604 | 0.647/0.604 | 0.739/0.631 | 0.662/0.621 | 0.536 / 0.555 | 0.619/0.592 | |
| DS AZ | 24 | 0.420/0.366 | 0.617/0.562 | 0.587/0.552 | 0.663/0.571 | 0.700/0.610 | 0.536/0.509 | 0.689/0.596 |
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Horizon | R4 | R8 | R16 | R32 | R64 |
| AZ | 24 | 0.575/0.406 | 0.55/0.392 | 0.550/0.389 | 0.546/0.338 | 0.559/0.387 |
| 36 | 0.656/0.444 | 0.650/0.438 | 0.632/0.432 | 0.633/0.431 | 0.633/0.430 | |
| 48 | 0.717/0.470 | 0.705/0.463 | 0.694/0.459 | 0.686/0.457 | 0.685/0.456 | |
| 60 | 0.763/0.493 | 0.740/0.482 | 0.741/0.479 | 0.731/0.477 | 0.726/0.475 | |
| Avg | 0.678/0.453 | 0.661/0.444 | 0.654/0.440 | 0.649 / 0.426 | 0.651 / 0.437 | |
| DS | 24 | 0.690/0.419 | 0.661/0.405 | 0.663/0.401 | 0.662/0.400 | 0.677/0.398 |
| Dataset | Horizon | R4 | R8 | R16 | R32 | R64 |
| AZ | 24 | 0.545/0.387 | 0.536/0.381 | 0.530/0.376 | 0.530/0.376 | 0.524/0.373 |
| 36 | 0.630/0.430 | 0.624/0.427 | 0.615/0.421 | 0.613/0.419 | 0.611/0.416 | |
| 48 | 0.687/0.457 | 0.677/0.452 | 0.671/0.448 | 0.671/0.447 | 0.666/0.445 | |
| 60 | 0.728/0.476 | 0.719/0.472 | 0.717/0.470 | 0.712/0.467 | 0.709/0.465 | |
| Avg | 0.648/0.438 | 0.639/0.433 | 0.633/0.429 | 0.632 / 0.427 | 0.628 / 0.425 | |
| DS | 24 | 0.650/0.398 | 0.641/0.393 | 0.636/0.388 | 0.636/0.389 | 0.644/0.391 |
| Dataset | Horizon | R4 | R8 | R16 | R32 | R64 |
| AZ DS | 24 | 0.465/0.364 | 0.457/0.351 | 0.455/0.348 | 0.455/0.351 | 0.453/0.350 |
| 36 | 0.509/0.389 | 0.506/0.385 | 0.501/0.382 | 0.502/0.380 | 0.504/0.381 | |
| 48 | 0.555/0.414 | 0.542/0.405 | 0.543/0.403 | 0.541/0.403 | 0.547/0.406 | |
| 60 | 0.592/0.438 | 0.582/0.426 | 0.585/0.427 | 0.583/0.424 | 0.583/0.424 | |
| Avg | 0.530/0.401 | 0.522/0.392 | 0.521 / 0.390 | 0.520 / 0.390 | 0.522/0.390 | |
| DS AZ | 24 | 0.457/0.386 | 0.439/0.373 | 0.447/0.376 | 0.420/0.366 | 0.452/0.376 |
Air Quality Prediction
Air quality prediction estimates future pollutant levels and maps local exposure to support health warnings, planning, and pollution control.
See all 42 papers