Organizations: School of Artificial Intelligence, Wuhan University · Hangzhou Institute of Technology, Xidian University · Institute for Math and AI, Wuhan
Remote-sensing foundation models increasingly unify representations across sensing modalities and downstream tasks, yet their prediction mechanisms remain largely task-specific. Separate decoders, prediction heads, and objectives leave prediction fragmented across tasks. This naturally raises a question: can heterogeneous remote-sensing tasks share the prediction process? UniFlow-RS answers this question with a single latent flow-matching predictor shared across tasks. This requires addressing two key challenges: unifying heterogeneous prediction spaces and harmonizing joint optimization across tasks. First, we cast heterogeneous task outputs into structured visual targets in a common latent space, allowing them to be modeled by the same pretrained flow predictor. Task-dependent conditioning further accommodates differences in input configurations. Second, we optimize all tasks with a common Flow Matching objective, eliminating task-specific prediction losses. To mitigate optimization mismatch across tasks and modalities, we introduce Task-Modality Adaptive Timestep Sampling (TMAS), which dynamically allocates training timesteps according to the online learning state of each task-modality group. Using a single shared model, UniFlow-RS handles six tasks and thirteen datasets, surpassing recent task-specific state-of-the-art methods on several benchmarks while remaining competitive on the others. Beyond these benchmarks, the same checkpoint demonstrates strong zero-shot transfer across unseen datasets, task settings, occlusion conditions, and near-infrared false-color imagery, without target-specific adaptation. Codes are available at https://github.com/MindVision-AI/UniFlow-RS.
Figures & tables
Figure 1: UniFlow-RS unifies heterogeneous remote-sensing perception through a shared generative prediction process. Unlike prior multi-task models that share representations but retain task-specific predictors, UniFlow-RS uses a single generative predictor across six tasks and achieves competitive performance on thirteen datasets.
Figure 2: Overview of UniFlow-RS. Native task outputs are rendered as structured visual targets and encoded by a frozen VAE. A pretrained MMDiT with one shared trainable LoRA predicts the conditional velocity field, while TMAS adapts timestep sampling from online Flow Matching errors. At inference, transport starts from the task-dependent source latent; the shared VAE decoder and deterministic recovery operators then produce native task predictions.
Figure 3: Task–Modality Adaptive Timestep Sampling. Each group is indexed by task and modality. TMAS tracks group-specific Flow Matching error at two timescales, converts the resulting learning state into a sampling utility, and adjusts the timestep distribution after uniform warm-up. A persistent uniform component ensures continued exploration across the flow trajectory.
Figure 4: Main benchmark comparison across six tasks and thirteen datasets. Each panel reports the dataset-native metric indicated above the bars, with arrows showing the preferred direction. All UniFlow-RS results are obtained from the same shared checkpoint.
Figure 5: Qualitative comparison of task-specific baselines and UniFlow-RS across six tasks. Baseline names are shown below the corresponding task-specific predictions.
Setting
Building IoU ↑
Change IoU ↑
Semantic mIoU ↑
Height δ1↑
Ref. gIoU ↑
Det. mAP50 ↑
UTS ↑
Single-task
75.63
73.91
49.26
72.13
72.54
61.39
67.48
Joint
76.38
74.75
51.12
72.06
73.61
62.80
68.45
Joint + TMAS
76.93
74.63
52.57
73.31
73.93
64.75
69.35
Table 1: Comparison of task-specific and joint adaptation. Single-task uses one LoRA per task, whereas the joint variants share one LoRA.
Sampler
Building IoU ↑
Change IoU ↑
Semantic mIoU ↑
Height δ1↑
Ref. gIoU ↑
Det. mAP50 ↑
UTS ↑
Steps ↓
Uniform
75.92
74.42
50.31
71.84
73.20
61.95
67.94
NR
Fixed Beta (2.5,1)
76.38
74.75
51.12
72.06
73.61
62.80
68.45
60k
Adaptive non-uniform
75.98
74.36
50.47
72.13
73.72
62.20
68.14
58k
Global adaptive
76.54
74.61
51.68
72.33
73.52
63.03
68.62
53k
TMAS w/o activity
76.74
74.48
52.23
72.96
73.87
63.88
69.03
44k
TMAS
76.93
74.63
52.57
73.31
73.93
64.75
69.35
37k
Table 2: Comparison of timestep-sampling strategies under joint training. All results are measured after 60k updates. Steps denotes the first update reaching the final UTS of Fixed Beta, NR means not reached.
Table 3: Zero-shot transfer to unseen datasets under visual and visual–language out-of-distribution (OOD) shifts. EarthReason includes both visual and language shifts, whereas RISBench mainly introduces a visual shift. All UniFlow-RS results use the same jointly trained checkpoint.
Table 4: Zero-shot transfer to unseen tasks under occlusion. Chesapeake RSC evaluates road extraction under tree-canopy occlusion (IoU, F1, and canopy-road recall Rcanopy ), while PIE evaluates four-class semantic segmentation under clear and synthetic-cloud conditions. All models are evaluated without target-domain adaptation.
Figure 7: Source-state initialization ablation.
Appendix figures & tables23 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Fixed text prompts for building extraction, change detection, height estimation, and rotated detection. Prompts are shown verbatim.
Figure 9: Dataset- and modality-specific semantic-segmentation prompts. RGB triplets and hexadecimal colors are explicitly included in the prompts.
Task
Dataset
Reported metric(s)
Building extraction
WHU Building
IoU ↑
SpaceNet 6 (SAR)
IoU ↑
Change detection
WHU-CD
IoU ↑
LEVIR-CD+
IoU ↑
UrbanSARFloods (SAR)
mIoU ↑
Semantic segmentation
LoveDA
mIoU ↑
Appendix
Table 5: Evaluation metrics for the thirteen datasets. Arrows indicate the preferred direction.
Figure 10: Frozen-VAE round-trip analysis of the structured visual target space. Columns show building, change, semantic, height, referring, and rotated detection targets; rows show the native target, visual target, VAE round trip, recovered target, and recovery error. The experiment measures whether task semantics are preserved by the frozen VAE independently of flow-model learning.
Semantic target palette
LoveDA ↑
OEM-RGB ↑
OEM-SAR ↑
Mean ↑
Dataset official palettes
99.41
96.57
90.79
95.59
Shared canonical palette
99.73
93.28
87.91
93.64
VAE-aware canonical (ours)
99.72
98.83
99.11
99.22
Appendix
Table 6: Semantic-palette ablation under frozen-VAE round-trip evaluation. Results are reported as native mIoU.
Height encoding
Potsdam RMSE (m) ↓
DFC23 δ1 (%) ↑
Linear grayscale
0.378
72.22
Log1p grayscale (ours)
0.176
97.42
Appendix
Table 7: Height-encoding ablation under frozen-VAE round-trip evaluation. Both variants use the same frozen VAE and fixed 0.2-m ground-recovery threshold.
Palette
Recovery path
Raster mIoU ↑
mAP50 ↑
Split ↓
VAE-aware hybrid
Direct recovery
100.00
89.13
0.14
Initial palette
VAE + recovery
97.90
89.44
0.50
VAE-aware hybrid palette (ours)
VAE + recovery
98.43
90.10
0.44
Appendix
Table 8: DIOR-R raster and instance recovery fidelity. Direct recovery is compared with frozen-VAE round-trip recovery using ground-truth visual targets.
Figure 11: Recovery fidelity for small instances and semantic boundaries. (a,b) DIOR-R recall at polygon IoU 0.75 , grouped by instance area and rotated-box short-edge length, comparing direct and VAE round-trip recovery. (c) Class-macro boundary IoU after VAE round-trip recovery at 1-, 2-, and 4-pixel widths. Direct recovery achieves 100% boundary IoU and is omitted from (c).
Source zs
Bldg. IoU ↑
Sem. mIoU ↑
Height δ1↑
Ref. gIoU ↑
Det. mAP50 ↑
ϵ∼N(0,I)
74.17
49.93
69.87
71.22
61.65
Current image latent za
76.93
52.57
73.31
73.93
64.75
Appendix
Table 9: Full numerical results for the source-state initialization ablation. Visual conditioning is fixed within each panel.
Figure 12: Transport-state trajectories across six tasks. All rows use the same shared UniFlow-RS checkpoint. From top to bottom: semantic segmentation (LoveDA), rotated detection (DIOR-R), height estimation (Potsdam), building extraction (SpaceNet 6 SAR), referring segmentation (LaSeRS), and change detection (WHU-CD). Columns show the input, decoded source state, selected intermediate states D(zt) , final decoded state, and ground-truth structured visual target.
Figure 13: Pairwise gradient cosine similarities between tasks on the shared LoRA at the same initialization checkpoint: (a) task-specific heads and (b) UniFlow-RS. Diagonal entries are 1 by construction.
Table 10: Detailed zero-shot transfer results on two unseen datasets. Pr@α denotes the percentage of queries with IoU ≥α , and “–” indicates that the corresponding result is not reported.
Table 11: Zero-shot building extraction on Vaihingen IRRG imagery. Results are aggregated over 17 held-out areas (398 patches). All metrics are reported as percentages.
Variant
Bldg. ↑
Change ↑
Sem. ↑
Height ↑
Ref. ↑
Det. ↑
UTS ↑
Steps ↓
Improvement-only activity
76.79
74.47
52.24
73.06
73.77
64.50
69.14
42k
No uniform warm-up
76.58
74.29
51.96
72.84
73.51
64.22
68.90
49k
No persistent exploration ( λ=0 )
76.71
74.38
52.08
72.96
73.64
64.36
69.02
45k
TMAS
76.93
74.63
52.57
73.31
73.93
64.75
69.35
37k
Appendix
Table 12: Ablation of learning activity and exploration in TMAS. Improvement-only activity considers only decreasing errors. No uniform warm-up removes the initial uniform sampling phase, while no persistent exploration removes the uniform mixture after warm-up. Steps denotes the updates required to reach the final UTS of Fixed Beta.
Figure 14: Task–modality-specific timestep allocation and validation-error changes. (a) TMAS sampling probabilities relative to uniform sampling ( 1/K ). (b) Per-bin reduction in validation Flow Matching error relative to Fixed Beta. Warmer colors indicate higher relative sampling probabilities in (a) and larger error reductions in (b).
Rank
Trainable params ↓
UTS ↑
Peak memory ↓
Images/s ↑
8
12.3M
67.24
40.92 GiB
6.69
16
24.6M
68.91
41.21 GiB
6.69
32
49.1M
69.35
41.53 GiB
6.69
64
98.3M
69.37
42.23 GiB
6.69
Appendix
Table 13: LoRA capacity and efficiency. Rank 32 is the default used in all other experiments and is highlighted in the Rank column. Bold metric values indicate the best result in each column.
Method
Params (B) ↓
Mean ↓
P95 ↓
Memory ↓
Samples/s ↑
RS2-SAM2-RRSISD
0.935
234.90
238.07
4.018
4.257
LISA-7B-v1
7.705
638.48
721.41
15.992
1.566
GeoPixel-7B-RES
11.338
6352.46
7691.27
27.193
0.157
UniFlow-RS
8.032
321.22
322.35
13.291
3.113
Appendix
Table 14: End-to-end referring-segmentation efficiency on RRSIS-D. Parameters include all loaded components. Latency is reported in milliseconds and peak memory in GiB. All methods process expressions online with batch size one.
Table 15: Benchmark comparison for change detection, semantic segmentation, and rotated detection. Gray cells denote results reported by prior work, and bold indicates the best result in each column.
Table 17: Comparison with InstructCV across six tasks and thirteen datasets. All metrics are percentages except Potsdam RMSE (m). Referring-segmentation results are reported as cIoU/gIoU.
Figure 15: Additional qualitative comparisons across six tasks. Rows show the input, ground truth, two baseline predictions, and UniFlow-RS. Baseline methods are labeled below each prediction and cited in the text.
Figure 16: Qualitative zero-shot referring-segmentation results on EarthReason and RISBench. Each row shows the same image, query, ground truth, and model predictions. Comparisons include FLUX.2-Klein Base, RS2-SAM2-RRSISD ( Rong et al., 2026 ) on EarthReason, and VisionReasoner ( Liu et al., 2026 ) on RISBench. White denotes foreground. VisionReasoner predictions are reproduced using its released checkpoint and official pipeline on the selected examples.
Figure 17: Qualitative zero-shot transfer under occlusion and spectral shifts. Two examples are shown for each dataset. Comparisons include S5 ( Lv et al., 2026 ) , the RoadGIE automatic baseline ( Peng et al., 2026 ) , MM-OVSeg ( Wei et al., 2026b ) , SegEarth-OV ( Li et al., 2025b ) , and CorrCLIP ( Zhang et al., 2025a ) . PIE uses synthetic thick-cloud conditions, with MM-OVSeg additionally receiving uncorrupted SAR. Semantic colors indicate road (yellow), water (blue), forest (green), farmland (brown), and Other (gray); binary foreground is shown in white.
Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, China · School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing, China