HOPD: Historical On-Policy Distillation for SAR MLLM Continual Learning
Organizations: Singapore Management University · Nanyang Technological University · Nankai University · Xi’an Jiaotong University
Abstract
Continual SAR interpretation requires multimodal large language models to learn incoming sources while retaining earlier task abilities. This is difficult when acquisition and geographic shifts alter visual evidence, regionally concentrated categories bias answers, and historical images are scarce. We address this problem with Historical On-Policy Distillation (HOPD), which reuses source-acquisition LoRA states as persistent teachers for one evolving student. Archiving each state before consolidation preserves a stable source reference. The teachers supervise student-generated prefixes, transferring historical behavior to the student's changing generation contexts. To enrich sparse replay, original-view teachers guide augmented historical student views while preserving spatial coordinates. This knowledge is consolidated into a single deployed adapter that requires no source identity. On SARVQA-CL, our source-incremental evaluation protocol, HOPD achieves the highest final mean score of 54.51 among the compared methods, exceeding the strongest baseline by 2.96 points. It also improves the final source-macro score in every evaluated task over shared-memory Replay-SFT, with gains across source orders. These results support source-acquisition history as supervision for retaining earlier task performance during continual SAR adaptation. Code and data: https://github.com/zcablii/HOPD.
Figures & tables
| Method | Tunable Params. (M) | MFN | MAA | BWT | MFT | DCIR | NFR | JFR | JFA |
|---|---|---|---|---|---|---|---|---|---|
| Zero-shot [ 52 ] | 0 | 20.50 | — | — | — | — | — | — | — |
| Sequential LoRA [ 19 ] | 19.50 | 33.09 | 44.83 | 43.47 | 84.98 | 91.06 | 20.68 | 9.31 | |
| Replay-SFT [ 9 ] | 19.50 | 47.06 | 50.23 | 26.99 | 71.05 | 49.98 | 55.29 | 47.98 | 15.43 |
| EWC-LoRA [ 21 ] | 19.50 | 33.74 | 45.22 | 42.36 | 71.39 | 82.97 | 90.00 | 22.49 | 9.90 |
| LwF [ 37 ] | 19.50 | 33.01 | 45.73 | 42.96 | 71.19 | 89.36 | 94.92 | 29.60 | 9.73 |
| O-LoRA [ 66 ] | 175.47 | 35.95 | 46.01 | 37.38 | 69.18 | 77.88 | 83.39 | 36.51 | 12.10 |
| Variant | Supervision | Prefix | View (T/S) | Stage 4: HRSID | Stage 9: SIVED | Full 3-sequence | ||
|---|---|---|---|---|---|---|---|---|
| Old | Current | Old | Current | Overall MFN | ||||
| No consolidation | None | – | – | +0.00 | +0.00 | +0.00 | +0.00 | 33.09 |
| SFT consolidation | CE | GT | –/G | +1.11 | -1.08 | +21.19 | -1.11 | 49.81 |
| w/o historical gamma | History | Student | O/O | +3.30 | -0.80 | +20.54 | 51.23 | |
| w/ previous-stage teacher | Previous | Student | O/G | +2.57 | -1.79 | +21.26 | -0.27 | 50.56 |
| Symmetric gamma views | History | Student | G/G | +3.52 | -0.80 | +21.39 | -0.98 | 52.98 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Source | Train QA | Eval QA |
|---|---|---|
| OGSOD | 41,301 | 891 |
| Shipdataset | 106,167 | 830 |
| SSDD | 5,221 | 468 |
| HRSID | 26,500 | 725 |
| SAR-AIRcraft | 53,456 | 896 |
| MSAR | 110,226 | 896 |
| Task | Primary score | Output interpretation |
|---|---|---|
| Caption | Core Fact F1 | Deterministic category-presence and explicit scene-count claims scored against scoped object annotations. |
| Count | Accuracy | Exact integer after unambiguous numerical parsing. |
| Presence | Accuracy | Boolean answer with explicit yes/no and true/false aliases. |
| Classification | Exact-set accuracy | Canonical category set. |
| Detection | Class-aware GRIT | One-to-one category-compatible box matching. Matched IoU sum normalized by predicted and reference object counts. |
| Referring expression | [email protected] | One target box with IoU . |
| Method | Order | DCIR/NFR QA | JFR pairs | JFA pairs | Empty DCIR cells | Empty JFR cells |
|---|---|---|---|---|---|---|
| HOPD | 1 | 1677 | 435 | 1195 | 10 | 9 |
| HOPD | 2 | 1547 | 384 | 1472 | 7 | 4 |
| HOPD | 3 | 1233 | 270 | 1180 | 9 | 10 |
| Replay-SFT | 1 | 1324 | 387 | 1195 | 10 | 9 |
| Replay-SFT | 2 | 1316 | 339 | 1472 | 8 | 5 |
| Replay-SFT | 3 | 1101 | 246 | 1180 | 9 | 11 |
| Method | MFN | MAA | BWT | MFT | DCIR | NFR | JFR | JFA |
|---|---|---|---|---|---|---|---|---|
| Zero-shot | 20.50 | — | — | — | — | — | — | — |
| Sequential LoRA | ||||||||
| Replay-SFT | ||||||||
| EWC-LoRA | ||||||||
| LwF | ||||||||
| O-LoRA |
| Replay-SFT | HOPD | Reduction | |
|---|---|---|---|
| 1 | |||
| 2 | |||
| 3 | |||
| 4 | |||
| 5 | |||
| 6 |
Continual Learning for LLMs
Continual learning for large language models studies how models absorb new knowledge and skills while retaining previously learned capabilities.
See all 256 papers