DroneSegEdit: Adapting Pretrained Generative Editors for UAV Segmentation with Language-Grounded Verification
Organizations: Fudan University, China
Abstract
Recent work has shown that pretrained image generators can support diverse visual tasks. Yet directly applying them to language-guided unmanned aerial vehicle (UAV) segmentation exposes a key limitation: a plausible mask may correspond to the wrong instance among small and repeated objects. We call this mismatch between mask rendering and expression-level target selection the Grounding-Rendering Gap. We present DroneSegEdit, which combines RGB proposal rendering with language-grounded, proposal-constrained verification to select supported foreground while retaining rendered geometry. DroneSegEdit adapts the open-weight LongCat-Image-Edit model using task-specific LoRA and fixed color-to-label decoding. For referring segmentation, a Language-Grounded Spatial Verifier (LGSV) reads hierarchical states from the frozen multimodal large language model backbone to derive expression-conditioned spatial evidence, which is mapped back to the rendered proposal without introducing foreground outside it. On RIS-LAD, DroneSegEdit reaches a new state of the art at 52.78 mIoU, 7.16 points above the previous best. It remains competitive overall on UAVid-RIS and VDD-RIS, with its clearest class-wise gains on compact, application-relevant categories, notably vehicles and humans. A separate verifier-free experiment extends the RGB interface to full-scene parsing: a task-specific adapter jointly renders all classes from one task-level instruction without enumerating class names or palette colors.
Figures & tables
| Method | Core backbone(s) | [email protected] | [email protected] | [email protected] | [email protected] | [email protected] | mIoU | oIoU |
| Traditional RIS methods | ||||||||
| LAVT [ 45 ] | Swin-B/BERT | 31.67 | 27.04 | 23.02 | 17.55 | 8.58 | 30.14 | 41.97 |
| FIANet [ 19 ] | Swin-B/BERT | 40.21 | 35.22 | 29.93 | 23.71 | 13.21 | 37.44 | 43.39 |
| RMSIN [ 22 ] | Swin-B/BERT | 43.36 | 38.11 | 32.36 | 25.08 | 13.75 | 39.60 | 48.82 |
| RSRefSeg [ 4 ] | SigLIP/SAM | 44.73 | 40.21 | 36.30 | 30.91 | 21.46 | 41.16 | 47.71 |
| SAARN [ 46 ] | Swin-B/BERT | 45.02 | 39.34 | 33.37 | 26.24 | 15.31 | 41.67 | 49.60 |
| Dataset | Method | Core backbone(s) | [email protected] | [email protected] | [email protected] | [email protected] | [email protected] | mIoU | oIoU |
| UAVid-RIS | LAVT [ 45 ] | Swin-B/BERT | 84.45 | 75.75 | 59.60 | 38.71 | 10.91 | 69.32 | 78.76 |
| RMSIN [ 22 ] | Swin-B/BERT | 85.71 | 77.91 | 64.06 | 46.52 | 17.76 | 72.05 | 81.10 | |
| MAFN [ 36 ] | Swin-B/BERT | 85.10 | 77.76 | 63.68 | 44.26 | 16.33 | 71.64 | 80.67 | |
| AeroReformer [ 21 ] | Swin-B/BERT | 86.34 | 79.07 | 65.60 | 47.12 | 18.52 | 72.79 | 81.53 | |
| LongCat-Image-Edit | Qwen2.5-VL/DiT | 81.37 | 72.26 | 59.24 | 42.23 | 15.05 | 69.82 | 73.01 | |
| DroneSegEdit | Qwen2.5-VL/DiT | 88.37 | 79.68 | 66.24 | 46.82 | 17.24 | 73.48 | 78.66 |
| Method | Backbone | mIoU |
| Mask2Former [ 7 ] | Swin-T | 74.32 |
| Mask2Former [ 7 ] | ResNet-50 | 79.90 |
| SegFormer-B0 [ 41 ] | MiT-B0 | 74.94 |
| SegFormer-B5 [ 41 ] | MiT-B5 | 78.54 |
| UPerNet [ 40 ] | Swin-T | 80.39 |
| UPerNet [ 40 ] | Swin-L | 82.17 |
| (a) LGSV Design and Language Conditioning | |||
| Configuration | Peak-cell | Peak-in-mask | mIoU |
| Single layer (L16) | 59.57 | 53.01 | 51.71 |
| Mismatched expression | 41.76 | 37.43 | 47.78 |
| Equal-capacity image-only | 36.43 | 31.48 | 44.93 |
| DroneSegEdit | 61.48 | 55.07 | 53.15 |
| Symbol | Definition |
| Task index, with . | |
| Referring tasks, partitioned as . | |
| Input RGB image, optional referring expression, and task target. | |
| Pixel lattice and task label set. | |
| Frozen editor parameters and task-specific LoRA update. | |
| Generated RGB canvas and deterministic task palette decoder. |
| Module / task | Optimization | Schedule | Selected checkpoint | Inference |
| Rendering LoRA: RIS-LAD | AdamW; LR ; batch 4. Continuation: | 2k constant-LR adaptation; 2k cosine-decay continuation | 2k adaptation + 2k continuation (val.) | ; 8 steps; CFG 2.5; fixed decoder |
| Rendering LoRA: UAVid-RIS | AdamW; LR ; batch 8 | 7k constant | 6k (val.) | ; 8 steps; CFG 2.5; fixed decoder |
| Rendering LoRA: VDD-RIS | AdamW; LR ; batch 8 | 7k constant | 5k (val.) | ; 8 steps; CFG 2.5; fixed decoder |
| Rendering LoRA: VDD dense | AdamW; LR ; batch 2 | 7k constant | 5k (val.) | ; 12 steps; CFG 1.5; nearest palette; no LGSV |
| LGSV readout: referring tasks | BF16 AdamW; batch 2; weight decay ; LR | At most 24k optimizer updates | Best on validation | Layers ; width 256 |
| Dataset | Output | mDice | Global Precision | Global Recall | Pred/GT |
| RIS-LAD | LongCat-Image-Edit | 54.37 | 52.93 | 81.04 | 1.531 |
| DroneSegEdit | 58.88 | 62.10 | 80.57 | 1.297 | |
| VDD-RIS | LongCat-Image-Edit | 85.69 | 80.44 | 94.20 | 1.171 |
| DroneSegEdit | 86.90 | 87.70 | 92.11 | 1.050 | |
| UAVid-RIS | LongCat-Image-Edit | 80.04 | 76.93 | 93.47 | 1.215 |
| DroneSegEdit | 83.05 | 85.84 | 90.38 | 1.053 |
Text-Guided Image Segmentation
Text-guided image segmentation maps natural-language descriptions or concepts to pixel-level regions in images.
See all 102 papers