PROLA: Principal-Orthogonal Low-rank Adaptation for Predictive Spatiotemporal Weather Downscaling
Organizations: Korea Advanced Institute of Science and Technology (KAIST)
Abstract
High-resolution weather prediction is important for resolving local atmospheric patterns missed by coarse global forecasts. Large pretrained weather backbones have improved global forecasting, but adapting them to produce future high-resolution fields from recent coarse states remains challenging and costly. We formulate this setting as Predictive Spatiotemporal Weather Downscaling (PSWD), a task that jointly refines spatial and temporal resolution while predicting future high-resolution trajectories. To this end, we propose a framework that decomposes prediction into spatial and temporal stages using a shared pretrained backbone, fixed numerical scaffolds, and lightweight residual heads. For backbone adaptation, we introduce PROLA (PRincipal-Orthogonal Low-rank Adaptation), which allocates a fixed low-rank budget between pretrained weights' principal directions and their orthogonal complement. PROLA further rescales rank-1 optimizer updates using gradient signals, improving adaptation without increasing trainable rank. Experiments in the PSWD setting show that PROLA outperforms representative low-rank adaptation baselines. Further experiments across diverse datasets and tasks support the broader applicability of PROLA, showing that its benefits extend beyond the primary weather downscaling setting.
Figures & tables
| Method | Trainable Params | Z500 | T850 | T2M | U10 | V10 | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| RMSE | ACC | RMSE | ACC | RMSE | ACC | RMSE | ACC | RMSE | ACC | ||
| 3h lead | |||||||||||
| Full FT | 1.26B | 60.13 [-1pt] 1.61 | 0.998 [-1pt] 0.000 | 0.813 [-1pt] 0.002 | 0.987 [-1pt] 0.000 | 1.064 [-1pt] 0.009 | 0.979 [-1pt] 0.000 | 0.902 [-1pt] 0.004 | 0.975 [-1pt] 0.000 | 0.925 [-1pt] 0.009 | 0.974 [-1pt] 0.001 |
| Head only | 1.07M | 94.93 [-1pt] 0.49 | 0.996 [-1pt] 0.000 | 1.049 [-1pt] 0.006 | 0.978 [-1pt] 0.000 | 2.083 [-1pt] 0.009 | 0.915 [-1pt] 0.001 | 1.648 [-1pt] 0.018 | 0.914 [-1pt] 0.001 | 1.563 [-1pt] 0.018 | 0.924 [-1pt] 0.002 |
| LoRA | 3.92M | 54.72 [-1pt] 0.36 | 0.999 [-1pt] 0.000 | 0.898 [-1pt] 0.002 | 0.984 [-1pt] 0.000 | 1.199 [-1pt] 0.007 | 0.973 [-1pt] 0.000 | 1.219 [-1pt] 0.008 | 0.954 [-1pt] 0.000 | 1.181 [-1pt] 0.002 | 0.957 [-1pt] 0.000 |
| DoRA | 4.16M | 54.32 [-1pt] 0.35 | 0.999 [-1pt] 0.000 | 0.893 [-1pt] 0.002 | 0.984 [-1pt] 0.000 | 1.191 [-1pt] 0.006 | 0.973 [-1pt] 0.000 | 1.209 [-1pt] 0.008 | 0.955 [-1pt] 0.000 | 1.173 [-1pt] 0.001 | 0.958 [-1pt] 0.000 |
| 3h lead | 6h lead | |||||||||
| Method / Variant | Z500 | T850 | T2M | U10 | V10 | Z500 | T850 | T2M | U10 | V10 |
| Direct prediction | ||||||||||
| LoRA | 58.91 | 0.979 | 1.256 | 1.481 | 1.484 | 129.60 | 1.214 | 1.394 | 1.755 | 1.781 |
| PiSSA | 56.47 | 0.942 | 1.191 | 1.272 | 1.239 | 128.00 | 1.152 | 1.292 | 1.392 | 1.405 |
| OPLoRA | 58.15 | 0.983 | 1.254 | 1.463 | 1.457 | 131.16 | 1.219 | 1.388 | 1.741 | 1.735 |
| PROLA | 53.13 | 0.898 | 1.165 | 1.211 | 1.177 | 125.07 | 1.095 | 1.289 | 1.358 | 1.344 |
| 3h lead | 6h lead | ||||||||||||
| Configuration | Scaling | Z500 | T850 | T2M | U10 | V10 | Z500 | T850 | T2M | U10 | V10 | ||
| (i) Branch composition and update scaling | |||||||||||||
| Principal | 8 | 0 | – | 53.66 | 0.894 | 1.193 | 1.209 | 1.178 | 78.55 | 0.987 | 1.248 | 1.294 | 1.290 |
| Principal | 8 | 0 | 53.06 | 0.887 | 1.184 | 1.198 | 1.166 | 77.71 | 0.979 | 1.241 | 1.289 | 1.285 | |
| Orthogonal | 0 | 8 | – | 54.54 | 0.899 | 1.197 | 1.216 | 1.188 | 79.83 | 0.994 | 1.272 | 1.314 | 1.314 |
| Orthogonal | 0 | 8 | 52.81 | 0.886 | 1.175 | 1.189 | 1.157 | 76.67 | 0.977 | 1.260 | 1.284 | 1.278 | |
Appendix figures & tables32 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | Trainable parameters |
|---|---|
| Head only | 1,068,684 |
| LoRA | 3,919,500 |
| DoRA | 4,157,068 |
| AdaLoRA | 5,346,060 |
| PiSSA | 3,919,500 |
| OPLoRA | 3,919,500 |
| Method | Trainable Params | Variable | 1h | 2h | 3h | 4h | 5h | 6h |
|---|---|---|---|---|---|---|---|---|
| Z500 | 48.22 0.67 | 49.44 1.39 | 60.13 1.61 | 72.88 1.82 | 85.63 2.27 | 106.68 2.63 | ||
| T850 | 0.903 0.011 | 0.834 0.004 | 0.813 0.002 | 0.874 0.002 | 0.966 0.003 | 1.114 0.008 | ||
| T2M | 1.472 0.003 | 1.225 0.004 | 1.064 0.009 | 0.955 0.015 | 1.044 0.019 | 1.323 0.022 | ||
| U10 | 1.239 0.013 | 1.052 0.008 | 0.902 0.004 | 0.873 0.003 | 0.941 0.007 | 1.135 0.012 | ||
| Full FT | 1.26B | V10 | 1.310 0.024 | 1.094 0.017 | 0.925 0.009 | 0.894 0.003 | 0.984 0.003 | 1.209 0.010 |
| Head only | 1.07M | Z500 | 68.16 0.72 | 78.45 0.40 | 94.93 0.49 | 112.98 0.68 | 128.19 0.97 | 144.03 1.47 |
| Method | Trainable Params | Variable | 1h | 2h | 3h | 4h | 5h | 6h |
|---|---|---|---|---|---|---|---|---|
| Z500 | 0.999 0.000 | 0.999 0.000 | 0.998 0.000 | 0.997 0.000 | 0.996 0.000 | 0.995 0.000 | ||
| T850 | 0.984 0.000 | 0.986 0.000 | 0.987 0.000 | 0.985 0.000 | 0.981 0.000 | 0.976 0.000 | ||
| T2M | 0.958 0.001 | 0.972 0.000 | 0.979 0.000 | 0.983 0.000 | 0.980 0.001 | 0.970 0.001 | ||
| U10 | 0.952 0.001 | 0.966 0.000 | 0.975 0.000 | 0.976 0.000 | 0.973 0.000 | 0.961 0.001 | ||
| Full FT | 1.26B | V10 | 0.947 0.002 | 0.964 0.001 | 0.974 0.001 | 0.976 0.000 | 0.971 0.000 | 0.956 0.001 |
| Head only | 1.07M | Z500 | 0.998 0.000 | 0.997 0.000 | 0.996 0.000 | 0.994 0.000 | 0.992 0.000 | 0.990 0.000 |
| ERA5 CERRA: regional spatiotemporal downscaling | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 3h lead | 6h lead | |||||||||
| Method | Z500 | T850 | T2M | U10 | V10 | Z500 | T850 | T2M | U10 | V10 |
| LoRA | 72.14 | 1.076 | 2.150 | 1.370 | 1.327 | 86.33 | 1.092 | 2.147 | 1.373 | 1.340 |
| PiSSA | 71.80 | 1.072 | 2.126 | 1.354 | 1.311 | 85.72 | 1.084 | 2.114 | 1.352 | 1.323 |
| OPLoRA | 72.56 | 1.081 | 2.132 | 1.371 | 1.332 | 86.13 | 1.099 | 2.146 | 1.375 | 1.348 |
| PROLA | 68.44 | 1.028 | 2.021 | 1.316 | 1.274 | 81.41 | 1.036 | 2.019 | 1.319 | 1.283 |
| Pretrained | LoRA | PiSSA | OPLoRA | PROLA | |
| EX (%) | 45.50 | 49.54 | 51.96 | 49.41 | 52.87 |
| 6h lead | 12h lead | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Method | Z500 | T850 | T2M | U10 | V10 | Z500 | T850 | T2M | U10 | V10 |
| LoRA | 138.22 | 1.241 | 1.967 | 2.055 | 2.079 | 123.23 | 1.291 | 1.757 | 2.368 | 3.175 |
| PiSSA | 134.41 | 1.205 | 1.904 | 2.556 | 2.112 | 121.61 | 1.247 | 2.062 | 2.457 | 2.272 |
| OPLoRA | 139.20 | 1.237 | 1.839 | 2.093 | 2.097 | 124.38 | 1.296 | 1.760 | 2.519 | 3.102 |
| PROLA | 133.03 | 1.192 | 1.531 | 2.031 | 2.077 | 119.97 | 1.246 | 1.614 | 2.749 | 2.838 |
| 18h lead | 24h lead | |||||||||
| 3h lead | 6h lead | |||
|---|---|---|---|---|
| Method | Z500 | T2M | Z500 | T2M |
| LoRA | 52.27 | 1.148 | 76.14 | 1.141 |
| PiSSA | 54.62 | 1.171 | 82.54 | 1.159 |
| OPLoRA | 52.08 | 1.145 | 76.25 | 1.134 |
| PROLA | 48.01 | 1.054 | 69.58 | 0.998 |
| Method | DJF | MAM | JJA | SON |
|---|---|---|---|---|
| LoRA | 82.78 | 79.28 | 75.29 | 79.70 |
| PiSSA | 81.70 | 78.54 | 74.37 | 79.25 |
| OPLoRA | 83.28 | 79.88 | 75.70 | 79.99 |
| PROLA | 75.99 | 73.22 | 69.73 | 73.65 |
| Method | Upper RMSE | Upper BSS | Lower RMSE | Lower BSS |
|---|---|---|---|---|
| LoRA | 56.87 | 0.514 | 129.19 | 0.780 |
| PiSSA | 55.50 | 0.513 | 126.28 | 0.781 |
| OPLoRA | 56.63 | 0.512 | 127.55 | 0.776 |
| PROLA | 50.80 | 0.545 | 116.36 | 0.795 |
| 3h lead | 6h lead | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Method | Z500 | T850 | T2M | U10 | V10 | Z500 | T850 | T2M | U10 | V10 |
| LoRA | 0.301 | 0.627 | 0.272 | 0.396 | 0.372 | 0.077 | 0.349 | 0.081 | 0.286 | 0.260 |
| PiSSA | 0.304 | 0.619 | 0.277 | 0.392 | 0.369 | 0.082 | 0.331 | 0.079 | 0.291 | 0.263 |
| OPLoRA | 0.302 | 0.628 | 0.277 | 0.402 | 0.378 | 0.077 | 0.350 | 0.087 | 0.290 | 0.264 |
| PROLA | 0.294 | 0.589 | 0.229 | 0.329 | 0.312 | 0.072 | 0.293 | 0.062 | 0.248 | 0.204 |
| Method | SNR | Tendency RMSE | 850-hPa wind RMSE | |||
|---|---|---|---|---|---|---|
| Z500 | T2M | Z500 | T2M | Divergence | Vorticity | |
| LoRA | 30.78 | 21.46 | 40.48 | 0.418 | 8.70 | 15.53 |
| PiSSA | 30.87 | 21.62 | 40.26 | 0.417 | 8.61 | 15.45 |
| OPLoRA | 30.73 | 21.45 | 40.53 | 0.418 | 8.71 | 15.55 |
| PROLA | 31.48 | 21.87 | 39.89 | 0.415 | 8.46 | 15.18 |
| Method / backbone LR | Z500 | T850 | T2M | U10 | V10 |
|---|---|---|---|---|---|
| Full FT, | 116.94 | 1.116 | 1.306 | 1.178 | 1.235 |
| Full FT, | 108.90 | 1.105 | 1.304 | 1.122 | 1.203 |
| Full FT, | 106.06 | 1.096 | 1.324 | 1.093 | 1.165 |
| PROLA | 73.07 | 0.958 | 1.211 | 1.236 | 1.234 |
| Method | Training Time / Epoch | Peak GPU Memory |
|---|---|---|
| LoRA | 54m 38s | 15.56 GiB |
| DoRA | 71m 19s | 15.56 GiB |
| AdaLoRA | 58m 45s | 15.63 GiB |
| PiSSA | 55m 52s | 15.56 GiB |
| OPLoRA | 60m 56s | 15.57 GiB |
| PROLA | 67m 18s | 15.56 GiB |
| Method | Variable | RMSE | |||||
|---|---|---|---|---|---|---|---|
| 1h | 2h | 3h | 4h | 5h | 6h | ||
| Z500 | 101.90 | 72.91 | 58.91 | 72.14 | 98.30 | 129.60 | |
| T850 | 1.009 | 0.973 | 0.979 | 1.044 | 1.118 | 1.214 | |
| T2M | 1.419 | 1.303 | 1.256 | 1.230 | 1.294 | 1.394 | |
| U10 | 1.426 | 1.438 | 1.481 | 1.567 | 1.652 | 1.755 | |
| LoRA | V10 | 1.376 | 1.417 | 1.484 | 1.585 | 1.677 | 1.781 |
| Method / Variant | Variable | RMSE | ACC | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1h | 2h | 3h | 4h | 5h | 6h | 1h | 2h | 3h | 4h | 5h | 6h | ||
| Z500 | 104.37 | 72.09 | 53.13 | 64.82 | 92.24 | 125.07 | 0.995 | 0.997 | 0.999 | 0.998 | 0.996 | 0.992 | |
| T850 | 0.991 | 0.924 | 0.898 | 0.941 | 1.002 | 1.095 | 0.980 | 0.983 | 0.984 | 0.982 | 0.980 | 0.976 | |
| T2M | 1.432 | 1.245 | 1.165 | 1.109 | 1.174 | 1.289 | 0.960 | 0.970 | 0.974 | 0.976 | 0.974 | 0.968 | |
| U10 | 1.308 | 1.247 | 1.211 | 1.233 | 1.271 | 1.358 | 0.946 | 0.951 | 0.954 | 0.952 | 0.949 | 0.942 | |
| Direct (PROLA) | V10 | 1.245 | 1.202 | 1.177 | 1.203 | 1.244 | 1.344 | 0.953 | 0.956 | 0.958 | 0.956 | 0.953 | 0.945 |
| Configuration | Scaling | Variable | RMSE | ACC | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1h | 2h | 3h | 4h | 5h | 6h | 1h | 2h | 3h | 4h | 5h | 6h | |||||
| Z500 | 55.57 | 50.45 | 53.66 | 58.98 | 63.55 | 78.55 | 0.999 | 0.999 | 0.999 | 0.998 | 0.998 | 0.997 | ||||
| T850 | 0.941 | 0.910 | 0.894 | 0.918 | 0.937 | 0.987 | 0.982 | 0.983 | 0.984 | 0.983 | 0.982 | 0.980 | ||||
| T2M | 1.383 | 1.258 | 1.193 | 1.125 | 1.152 | 1.248 | 0.963 | 0.969 | 0.973 | 0.976 | 0.975 | 0.970 | ||||
| U10 | 1.288 | 1.240 | 1.209 | 1.225 | 1.241 | 1.294 | 0.948 | 0.952 | 0.954 | 0.953 | 0.952 | 0.947 | ||||
| Principal | 8 | 0 | – | V10 | 1.227 | 1.194 | 1.178 | 1.204 | 1.228 | 1.290 | 0.954 | 0.957 | 0.958 | 0.956 | 0.954 | 0.949 |
| Configuration | Scaling | Variable | RMSE | ACC | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1h | 2h | 3h | 4h | 5h | 6h | 1h | 2h | 3h | 4h | 5h | 6h | |||||
| Z500 | 53.90 | 48.22 | 51.46 | 56.54 | 60.16 | 74.19 | 0.999 | 0.999 | 0.999 | 0.998 | 0.998 | 0.997 | ||||
| T850 | 0.933 | 0.897 | 0.876 | 0.899 | 0.919 | 0.970 | 0.982 | 0.984 | 0.985 | 0.984 | 0.983 | 0.981 | ||||
| T2M | 1.385 | 1.229 | 1.153 | 1.086 | 1.127 | 1.225 | 0.962 | 0.971 | 0.974 | 0.977 | 0.976 | 0.971 | ||||
| U10 | 1.245 | 1.193 | 1.161 | 1.176 | 1.193 | 1.249 | 0.951 | 0.955 | 0.958 | 0.956 | 0.955 | 0.951 | ||||
| Principal-heavy | 6 | 2 | ✓ | V10 | 1.189 | 1.152 | 1.136 | 1.165 | 1.189 | 1.244 | 0.957 | 0.960 | 0.961 | 0.958 | 0.957 | 0.953 |
| Method | 3h lead | 6h lead | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Z500 | T850 | T2M | U10 | V10 | Z500 | T850 | T2M | U10 | V10 | |
| Total rank | ||||||||||
| PiSSA | 53.66 | 0.894 | 1.193 | 1.209 | 1.178 | 78.55 | 0.987 | 1.248 | 1.294 | 1.290 |
| OPLoRA | 54.54 | 0.899 | 1.197 | 1.216 | 1.188 | 79.83 | 0.994 | 1.272 | 1.314 | 1.314 |
| PROLA (4+4) | 50.30 | 0.866 | 1.128 | 1.140 | 1.119 | 73.07 | 0.958 | 1.211 | 1.236 | 1.234 |
| Total rank | ||||||||||
Low-Rank Adaptation
Low-rank adaptation (LoRA) fine-tunes pretrained models with compact weight updates, reducing trainable parameters and deployment costs.
See all 579 papers