RSGen: Enhancing Layout-Driven Remote Sensing Image Generation with Diverse Edge Guidance
Authors: Xianbao Hou, Yonghao He, Zeyd Boukhers, John See, Hu Su, Wei Sui, Cong Yang
Organizations: School of Future Science and Engineering, Soochow University, Suzhou, China · D-Robotics, Beijing, China · Fraunhofer Institute for Applied Information Technology, Sankt Augustin, Germany · School of Mathematical and Computing Sciences, Heriot-Watt University Malaysia, Putrajaya, Malaysia · Institute of Automation, Chinese Academy of Sciences, Beijing, China
Diffusion models have significantly mitigated the impact of annotated data scarcity in remote sensing (RS). Although recent approaches have successfully harnessed these models to enable diverse and controllable Layout-to-Image (L2I) synthesis, they still suffer from limited fine-grained control and fail to strictly adhere to bounding box constraints. To address these limitations, we propose RSGen, a plug-and-play framework that leverages diverse edge guidance to enhance layout-driven RS image generation. Specifically, RSGen employs a progressive enhancement strategy: 1) it first enriches the diversity of edge maps composited from retrieved training instances via Image-to-Image generation; and 2) subsequently utilizes these diverse edge maps as conditioning for existing L2I models to enforce pixel-level control within bounding boxes, ensuring the generated instances strictly adhere to the layout. Extensive experiments across three baseline models demonstrate that RSGen significantly boosts the capabilities of existing L2I models. For instance, with CC-Diff on the DOTA dataset for oriented object detection, we achieve remarkable gains of +9.8/+12.0 in YOLOScore mAP50/mAP50-95 and +1.6 in mAP on the downstream detection task. Our code will be publicly available: https://github.com/D-Robotics-AI-Lab/RSGen
Figures & tables
Figure 1: Visualization of controllability between existing L2I methods (MIGC [ 38 ] , CC-Diff [ 35 ] , FICGen [ 24 ] ) and the same methods equipped with RSGen. While the original methods (a) struggle to adhere to the specified bounding boxes, integrating our module (b) significantly enhances instance alignment and control precision.
Figure 2: Overview of RSGen, which consists of the Edge2Edge module (a) and the L2I FGControl module (b), where “CLS” denotes the class label. The Edge2Edge module enhances the diversity of retrieved edge maps through an I2I process. Subsequently, these diverse edges and layout inputs guide the L2I FGControl module, which interacts with the base L2I model to achieve precise pixel-level control. Our framework significantly increases structural diversity while ensuring fine-grained spatial alignment.
Figure 3: Visualization of diverse edge maps generated by the Edge2Edge module. Our method employs distinct random seeds to generate varied structural details within the specified bounding boxes, significantly enhancing the diversity of the structural priors.
Figure 4: Comparison of ControlNet [ 34 ] , ControlNet-XS [ 32 ] , and our FGControl. Standard global control methods suffer from feature entanglement, causing background chaos. Conversely, FGControl strictly confines high-frequency structural guidance within the layout bounding boxes, achieving fine-grained local control without interfering with the global semantic synthesis.
Method
DIOR-RSVG (HBB)
DOTA (OBB)
FID ↓
YOLOScore ↑
FID ↓
YOLOScore ↑
mAP50
mAP50−95
mAP50
mAP50−95
MIGC [ 38 ]
79.32
63.2
38.4
66.80
53.0
30.0
+ Ours
85.04
68.7
45.8
68.07
55.1
37.7
CC-Diff [ 35 ]
66.75
66.8
41.2
47.72
57.7
34.0
+ Ours
68.12
70.1
47.7
46.86
67.5
46.0
Table 1: Quantitative comparison of baseline models with and without our proposed RSGen on DIOR-RSVG and DOTA datasets. Models equipped with RSGen achieve substantial improvements in layout consistency (YOLOScore) alongside highly competitive generation fidelity (FID).
Method
DIOR-RSVG (HBB)
DOTA (HBB)
DOTA (OBB)
mAP
mAP50
mAP75
mAP
mAP50
mAP75
mAP50
mAP75
MIGC [ 38 ]
54.3
78.9
59.8
38.3
64.5
39.2
53.55
19.68
+ Ours
55.0
78.7
61.1
38.9
64.5
40.5
53.19
21.92
CC-Diff [ 35 ]
54.7
78.4
60.1
37.4
63.2
38.6
54.80
20.11
+ Ours
55.0
78.8
60.7
39.0
64.2
40.4
54.92
21.65
FICGen [ 24 ]
54.5
78.7
60.7
39.2
64.8
40.8
55.81
22.66
Table 2: Comparison of downstream object detection performance. By mixing synthetic and real data at a 1:1 ratio, models trained with data generated by RSGen achieve overall accuracy improvements across both HBB and OBB settings.
Method
Ratio (Real:Syn)
DOTA (HBB)
DOTA (OBB)
mAP
mAP50
mAP75
mAP50
mAP75
CC-Diff [ 35 ]
1:3
39.0
65.2
39.8
55.72
21.11
+ Ours
39.0
64.2
40.0
56.07
24.36
CC-Diff [ 35 ]
1:2
38.8
65.4
39.7
54.75
21.44
+ Ours
39.3
64.4
41.2
55.98
22.34
CC-Diff [ 35 ]
1:1
37.4
63.2
38.6
54.80
20.11
Table 3: Comparison of downstream object detection performance under different real-to-synthetic data ratios. All experiments are conducted on the DOTA dataset utilizing the CC-Diff [ 35 ] baseline.
Figure 5: Qualitative comparison of generated instances with and without the Edge2Edge module. Incorporating the Edge2Edge module introduces rich structural variations, significantly enhancing both the structural and overall diversity of the generated instances within the specified bounding boxes.
Method
FID ↓
YOLOScore ↑
mAP50
mAP50−95
w/o Edge2Edge
68.40
69.5
46.9
w/ Edge2Edge
68.12
70.1
47.7
Table 4: Ablation of the Edge2Edge module on DIOR-RSVG. While maintaining nearly identical generation fidelity (FID), the module significantly boosts layout consistency (YOLOScore), particularly in the stricter IoU threshold.
Method
vehicle
ship
basketballcourt
groundtrackfield
harbor
tenniscourt
airplane
w/o Edge2Edge
6.8
8.3
14.9
13.6
9.6
42.6
8.2
w/ Edge2Edge
7.1
8.3
15.5
13.5
10.2
44.6
8.4
Table 5: Cross-dataset validation. Models are trained on DIOR-RSVG and evaluated on DOTA. The Edge2Edge module improves AP across most categories, demonstrating its strong generalization capability, which is crucial for practical applications.
Method
FID ↓
YOLOScore ↑
mAP50
mAP50−95
Baseline (Global Control)
112.43
62.1
45.5
+ Spatially Gated
64.38
68.7
45.7
+ High-pass Filter (Full)
68.40
69.5
46.9
Table 6: Ablation of the FGControl module on the DIOR-RSVG dataset. The spatially gated mechanism resolves background confusion, while the high-pass filter further decouples features to achieve the highest layout consistency.
In this paper, we explore text-guided image editing in the remote sensing domain using generative modeling. We propose \rsedit, a collection of models from U-Net to DiT with various configurations. Specifically, we present the first comprehensive study of conditioning strategies for building image editing models from off-the-shelf text-to-image ones. Our experiments show that \rsedit achieves the best instruction-faithful edits while preserving geospatial structure. We release the code at https://github.com/Bili-Sakura/RSEdit-Preview and checkpoints at https://huggingface.co/collections/BiliSakura/rsedit.
Chen Zhenyuan, Zhang Zechuan, Zhang Feng
School of Earth Sciences, Zhejiang University, Hangzhou, 310058 China, and Zhejiang Provincial Key Laboratory of Geographic Information Science. · ReLER, CCAI, Zhejiang University, Hangzhou, China. · Key Laboratory of Spatio-temporal Information and Intelligent Services (LSIIS), Ministry of Natural Resources of the People’s Republic of China.
Diffusion models have emerged as powerful tools for a wide range of vision tasks, including text-guided image generation and editing. In this work, we explore their potential for object grounding in remote sensing imagery. We propose a hybrid pipeline that integrates diffusion-based localization cues with state-of-the-art segmentation models such as RemoteSAM and SAM3 to obtain more accurate bounding boxes. By leveraging the complementary strengths of generative diffusion models and foundational segmentation models, our approach enables robust and adaptive object localization across complex scenes. Experiments demonstrate that our pipeline significantly improves localization performance, achieving over a 14% increase in [email protected] compared to existing state-of-the-art methods.
Unified remote sensing multimodal models exhibit a pronounced spatial reversal curse: Although they can accurately recognize and describe object locations in images, they often fail to faithfully execute the same spatial relations during text-to-image generation, where such relations constitute core semantic information in remote sensing. Motivated by this observation, we propose Uni-RS, the first unified multimodal model tailored for remote sensing, to explicitly address the spatial asymmetry between understanding and generation. Specifically, we first introduce explicit Spatial-Layout Planning to transform textual instructions into spatial layout plans, decoupling geometric planning from visual synthesis. We then impose Spatial-Aware Query Supervision to bias learnable queries toward spatial relations explicitly specified in the instruction. Finally, we develop Image-Caption Spatial Layout Variation to expose the model to systematic geometry-consistent spatial transformations. Extensive experiments across multiple benchmarks show that our approach substantially improves spatial faithfulness in text-to-image generation, while maintaining strong performance on multimodal understanding tasks like image captioning, visual grounding, and VQA tasks.
Weiyu Zhang, Yuan Hu, Yong Li +1
Institute of Remote Sensing and Geographic Information System, School of Earth and Space Sciences, Peking University, Beijing, 100871, China · Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology, Hong Kong