Controllable object insertion provides a targeted approach to augmenting object detection datasets that require category--count fidelity, annotation alignment, and source-content preservation beyond visual realism. Several approaches have been proposed to address this grand challenge, but most suffer from different constraints in precise category--count requests and visual consistency, limited coordination across categories and instances, and poor localization of multiple objects of specific category and count. We design SceneP2F, a two-stage trainable framework that connects scene-aware joint planning with layout-aligned generation leveraging explicit instance-level layouts. SceneP2F consists of two novel designs. The first is ScenePlan which allocates one class-labeled query per requested instance, jointly predicts scene-compatible object centers, and separately estimates box sizes conditioned on the planned centers and scene context. The second is SceneAlign-Fill which combines geometry- and appearance-aware conditioning, box--class-guided regional attention, and an annotation-region-weighted flow-matching objective to render the planned objects while limiting unintended changes to source content. Extensive experiments on three challenging benchmarks UAVDT, VisDrone, and BDD100K show that SceneP2F outperforms representative general-purpose editors and object-insertion methods consistently, demonstrating the value of coordinating instance-level planning and generation for category-targeted detection augmentation.
Figures & tables
Figure 1: Given a source image and an object insertion request, representative image editing and object insertion approaches may fail to satisfy the requested categories or counts, alter existing objects, or generate implausible placements, scales, and appearances, as highlighted by the red boxes. SceneP2F, a two-stage framework comprising ScenePlan and SceneAlign-Fill, first predicts a scene-compatible, class-labeled box layout and then jointly synthesizes the requested instances within the planned regions, aiming to improve those questions in multi-object insertion task.
Figure 2: Overview of the SceneP2F framework, comprising ScenePlan (left) and SceneAlign-Fill (right). ScenePlan first jointly predicts object centers from image features, existing boxes, and category–count queries, while a size predictor estimates the corresponding box extents. Given the resulting class-labeled layout, SceneAlign-Fill uses geometric, categorical, and scene-style conditioning to synthesize the requested objects while limiting changes outside the target regions.
Figure 3: Removal dataset construction (left) and pretrained size predictor (right). The removal-data construction pipeline uses SAM2 to generate masks and ObjectClear to remove the target objects, while a verification module triggers iterative reprocessing when the check fails. Size predictor sub-model estimates target-box dimensions from image features, context boxes, and input center.
Figure 4: Qualitative comparison of SceneP2F with image-editing models (FLUX.2-dev and Qwen-Image-Edit) and object-insertion baselines (Add-SD and Add-it) on the joint multi-object insertion task. The first three rows present results on UAVDT, the subsequent two rows on VisDrone, and the final row on BDD100K. The leftmost column shows the requested categories and counts. For BDD100K (last row), Diffree replaces Add-SD, which was not trained on this dataset.
Dataset
Method Category
Method
Fine- tuned
Request Fidelity
Visual Consistency
Outside Preservation
mTCE ↓
mCCE ↓
LPIPS ↓
DINO Sim ↑
CLIP-I ↑
PSNR ↑
SSIM ↑
UAVDT ( Du et al., 2018 )
Image Editing
FLUX.2-dev ( Labs, 2025 )
7.001
3.076
0.5561
0.4753
0.8131
18.9585
0.8405
Qwen-Image-Edit ( Wu et al., 2025 )
5.596
2.458
0.5588
0.4635
0.8188
20.6050
0.8973
✓
14.959
7.603
0.5051
0.4499
0.8452
19.3728
0.8408
Object Insertion
Add-it ( Tewel et al., 2025 )
✓
8.529
4.594
0.5164
0.4508
0.8410
21.5469
0.8987
Add-SD ( Yang et al., 2024 )
✓
2.744
2.105
0.5751
0.4295
0.8356
22.2962
0.9010
Table 1: Quantitative comparison of general image-editing and object-insertion methods on UAVDT, VisDrone, and BDD100K. The best results are shown in bold, and the second-best are underlined.
Dataset
Inserted Objects
Method
Inserted Class
All Classes
Recall
mAP50
mAP50:95
mAP50
mAP50:95
UAVDT ( Du et al., 2018 )
2 buses
Original dataset
0.403
0.229
0.168
0.341
0.208
Copy-and-Paste
0.408
0.268
0.190
0.358
0.211
X-Paste
0.464
0.283
0.210
0.351
0.212
Ours (SceneP2F)
0.522
0.310
0.226
0.371
0.224
2 trucks
Original dataset
0.246
0.0718
0.045
0.341
0.208
Table 2: Downstream detection performance of rare-class augmentation methods on UAVDT using YOLO11-L. In two separate settings, each method augments 40% of the training images with either two buses or two trucks per image, and the resulting samples are added to the original training set.
Figure 7
Figure 7: Qualitative comparison of Random, Dataset-Prior, and ScenePlan object placements on VisDrone and UAVDT, illustrating differences in predicted locations and scales.
Table 9
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Visualization of Removal Dataset Processed images. The left figure is the original image with target removed bbox and corresponding class. while the right is processed image.
Figure 9: Visualization of plausible region mask among three datasets generated by SAM3. The black and white image on the right of each RGB image is the binary map of the result segmentation.
Group
ID
Class-balanced log-area L1 ↓
Class-balanced aspect-ratio L1 ↓
Average ↓
Description
Ours
0.4651
0.2730
0.3691
Selected configuration
Global feature
GF0
0.5057
0.2732
0.3895
All global features
GF1
0.4621
0.2933
0.3777
Distance feature
GF2
0.4774
0.2876
0.3825
Median-area feature
Relative feature
RF0
0.5149
0.3550
0.4350
All relative features
RF1
0.4834
0.2715
0.3775
Class embedding
Appendix
Table 5: Ablation study of the ScenePlan size predictor. We evaluate alternative global-feature and relative-feature designs, MLP head architectures, and training settings using class-balanced log-area and aspect-ratio L1 errors. The average is the arithmetic mean of the two errors.
Figure 10: Visual Comparison of different Image Inpainting Methods with SceneAlign-Fill.
Dataset
Recall
mAP50
mAP50–95
BDD100K ( Yu et al., 2020 )
94.91
65.54
38.04
UAVDT ( Du et al., 2018 )
96.64
91.71
70.75
VisDrone ( Zhu et al., 2021 )
78.88
48.29
28.59
Appendix
Table 6: Evaluation results of GroundingDINO on BDD100K, UAVDT and Visdrone test dataset
Figure 11: Visualization of inserted object detected by GroundingDINO. The figure in the left in original image and right is the generated. Green, red, and blue boxes indicate matched, unpreserved, and newly inserted objects, respectively; inserted-object metrics are computed on the blue boxes.
Figure 12: Qualitative results for the two-bus and two-truck insertion settings on UAVDT.
Figure 13: Visualized Comparison between different methods.
Remote sensing recognition is often constrained by scarce observations of rare targets and costly annotations, making realistic synthetic augmentation particularly valuable for few-shot and long-tailed scenarios. Object insertion provides an efficient way to increase target diversity while preserving authentic background scenes, but realistic insertion in overhead imagery requires the generated target to adapt coherently to its surrounding environment. To this end, we propose PDA++, a unified environment-aware object insertion framework organized as Plan, Decouple, and Assimilate. Planning determines scene-compatible poses through an affordance field that combines geometric clearance with structure- and scale-aware cues. Decoupling introduces a pose-conditioned background that provides precise spatial guidance together with target-scene context, allowing the reference object to preserve its identity while adapting to the target observation. This construction also naturally provides pixel-level masks for segmentation augmentation. Assimilation further improves local coherence by aligning multi-scale texture distributions through optimal transport. On the optical benchmark, PDA++ achieves a whole-image FID of 6.28 and improves average few-shot recognition mAP50 by 17.69 points, corresponding to a 28.8% relative gain over the real-data baseline. On SAR imagery, it improves ship detection by 4.10 mAP50 points and remains effective under cross-dataset transfer and amorphous-target insertion. Code is available at https://github.com/lisheyu972/PDA_PLUS.
We present InsertFuse, a unified framework for multi-category reference-guided image insertion. Its key idea is to decouple category-specific expertise learning from cross-category capability consolidation. InsertFuse first trains specialized experts for different insertion categories and then introduces Insertion On-Policy Distillation (IOPD) to consolidate their capabilities into a single student. By querying the matched expert at states visited by the student, IOPD preserves category-specific insertion behavior while mitigating the cross-category interference caused by direct joint training. To improve spatial control, we propose Token-Aligned Geometry Conditioning (TAGC), which maps mask-derived geometric cues to the visual token grid, and Region-Balanced Flow Matching, which separately normalizes prediction errors inside and outside the insertion region to prevent background-dominated and scale-dependent supervision. We further introduce Reference CFG to isolate and strengthen the guidance induced by the visual reference under fixed scene and geometry conditions, with IOPD transferring this enhanced supervision into the unified student. Extensive experiments on the public AnyInsertion benchmark and our multi-category test set demonstrate state-of-the-art performance on most metrics, showing strong reference fidelity and generation quality across diverse insertion categories.
Guangzhao Li, Qingyan Wei, Huayu Zheng +7
1Shanghai Jiao Tong University · 2Shanghai Innovation Institute · 4Xi’an Jiaotong University +2
Object insertion aims to seamlessly composite a reference object into a specified region of a background image. Recent diffusion-based methods achieve high visual quality but formulate insertion as a simple 2D inpainting task, providing no explicit control over the object's 3D pose and limiting their practical applicability. We propose DIRECT (Decomposed Injection for Reference Composition and Target-integration), a novel framework that integrates interactive pose manipulation with high-fidelity 2D image synthesis to enable pose-controllable object insertion. Our method decomposes the insertion conditions into three complementary components: appearance guidance capturing visual details from the reference object, geometry guidance derived from the user-adjusted 3D proxy, and context guidance from the target background. By injecting them through separate pathways, DIRECT avoids feature entanglement and simultaneously preserves reference appearance, follows the user-specified pose, and adapts the object to the target scene. We also introduce an automated data construction pipeline to improve the diversity and quality of training data. Experiments show that DIRECT outperforms previous methods in both geometric controllability and visual quality.
Jingbo Gong, Yikai Wang, Yushi Lan +6
VCIP, School of Computer Science, Nankai University · Zhongguancun Academy · S-Lab, Nanyang Technological University +2