We propose a method to learn explicit, class-conditioned spatial priors for object placement in natural scenes by distilling the implicit placement knowledge encoded in text-conditioned diffusion models. Prior work relies either on manually annotated data, which is inherently limited in scale, or on inpainting-based object-removal pipelines, whose artifacts promote shortcut learning. To address these limitations, we introduce a fully automated and scalable framework that evaluates dense object placements on high-quality real backgrounds using a diffusion-based inpainting pipeline. With this pipeline, we construct HiddenObjects, a large-scale dataset comprising 27M placement annotations, evaluated across 27k distinct scenes, with ranked bounding box insertions for different images and object categories. Experimental results show that our spatial priors outperform sparse human annotations on a downstream image editing task (3.90 vs. 2.68 VLM-Judge), and significantly surpass existing placement baselines and zero-shot Vision-Language Models for object placement. Furthermore, we distill these priors into a lightweight model for fast practical inference (230,000x faster).
Figures & tables
Figure 2: Background artifacts in PIPE [ 52 ] . Object removal pipelines are effective at removing the requested object but leave visible traces that can facilitate shortcut learning, e.g., (left) blurry area and the shadow presence, or (right) larger artifacts.
Dataset
# Annotation
# Positive Box
# Negative Box
Real
Dense
Scalable
Artifact-free
Boxes
(per instance)
(per instance)
Images
Priors
Background
DreamEditBench [ 26 ]
220
1
0
✓
✗
✗
✓
MureCom [ 32 ]
640
1
0
✓
✗
✗
✓
OPA [ 31 ]
0.07M
1.9
3.8
✓
✗
✗
✓
OPA-ext [ 31 ]
0.15M
2.2
4.3
✓
✗
✗
✓
PIPE [ 52 ]
0.89M
1
0
✗
✗
✓
✗
Table 1: Object Placement Datasets. Comparison of common datasets for learning object placement models. Manually annotated datasets are limited in scale, while automated approaches often introduce visual artifacts via inpainting. Our proposed HiddenObjects dataset achieves both scalability, high-quality dense bounding box annotations, and artifact-free backgrounds.
Figure 3: Spatial Prior Extraction. Given a background image x , the inpainter I synthesizes an image x^ij for target class cj at a bounding box bi . The verifier V evaluates the inpainted images x^ij , yielding a detected bounding box and a human-aligned preference score rij . Repeating this process across multiple sampled boxes yields the ground-truth spatial prior S(x,cj) for the j-th class. The heatmap visualizes the aggregated dense distribution of preferences over spatial locations ( no , mid , good ).
Figure 4: Spatial Prior ( HiddenObjects ) aggregated as a heatmap. ( Left ) We visualize the mean aggregated spatial prior across all images of HiddenObjects for different classes listed on the left. Classes such as bench and boat tend to appear near the bottom of images, while kite occurs more frequently near the top. These patterns reflect both semantic regularities (e.g., kite in the sky) and inherent photographic biases in image datasets. ( Right ) We visualize per-instance random example of spatial priors.
Figure 5: Verifier Comparison. For each verifier, we display the inpainting generation of the highest-scoring location. CLIP Score often prefers insertions irrespective of global scene context. In contrast, Image and Vision Reward outperform aesthetic score in judging realism and semantic coherence, leading to more human-aligned spatial priors.
Figure 6: Inpainter Comparison. We compare cross-attention-based inpainters with ControlNet-based variants. Cross-attention-based inpainters are prone to (i) inpainting artifacts or (ii) inpaint objects that don’t respect scene semantics. While ControlNet-based methods more reliably enforce scene constraints: they either (iii) refuse implausible insertions or generate (valid) semantically coherent, context-aware objects.
Inpainter
Ranker
AP ↑
FLUX
Random
52.8
FLUX
Aesthetics
52.6
FLUX
CLIP Score
53.8
FLUX
ImageReward
57.1
Table 2: Component Analysis. We measure inpainting and ImageReward performance (Left) We evaluate different rankers for the same inpainter FLUX. (Right) We evaluate different inpainters for the same Ranker. ImageReward and Qwen perform best. IR: ImageReward, CN: ControlNet.
Method
Test set
PC ↑
VN ↑
PDC ↑
Avg ↑
Raw background
HiddenObjects
1.04
1.03
1.03
1.04
Full Mask
HiddenObjects
1.62
1.60
1.59
1.60
Random Bounding Box
HiddenObjects
2.73
2.62
2.62
2.65
Ours (Annotation Pipeline)
HiddenObjects
3.83
3.63
3.63
3.69
Raw background
OPA
1.00
1.00
1.00
1.00
Full Mask
OPA
1.66
1.66
1.66
1.66
Table 3: Downstream image editing quality evaluated by ImgEdit-Judge [ 57 ] (1-5). We compare our annotation pipeline against various placement strategies.
Figure 7: Image inpainting with object placement priors. We evaluate four object placement approaches on the OPA [ 31 ] and HiddenObjects backgrounds. Full Mask inpaints the entire image. Human Annot. uses ground-truths from OPA. Random BBox samples a uniformly random placement region, serving as a controlled baseline. Ours uses the top-ranked box of our dataset. Dashed boxes show the inpainting region ( HiddenObject annotation (ours) , OPA annotation , Random ). Corner scores are ImgEdit-Judge [ 57 ] . Our pipeline consistently produces well-placed, photorealistic composites across object classes and image scenes, outperforming sparse human placement.
Figure 8: Analysis of Spatial and Scale Distributions. (a) Object center distributions across datasets: while traditional benchmarks like COCO [ 29 ] , VOC [ 14 ] , and ImageNet [ 11 ] exhibit a pronounced center-crop bias, our pipeline (evaluated on HiddenObjects and OPA backgrounds) yields a more diffuse distribution with higher spatial variance. (b) Distribution of bounding box areas: HiddenObjects and PIPE [ 52 ] lack small objects (0.01%–0.1% image area), which reveals a shared resolution bottleneck of underlying diffusion backbones in both generating or removing small objects.
Method
Train Set
mAP
IoU50@1
IoU@1
IoU50@5
IoU@5
(%)
(%)
(%)
(%)
(%)
Qwen2.5-VL-72B
Zero-shot
1.3
7.9
14.0
12.7
18.8
Qwen2.5-VL-7B
Zero-shot
1.4
8.6
13.9
10.1
16.5
InternVL3-8B
Zero-shot
0.3
3.3
11.2
9.2
18.7
LLaVA-OneVision-7B
Zero-shot
1.0
11.7
20.9
16.3
24.7
LLaVA-1.5-7B
Zero-shot
1.0
6.4
17.0
8.8
22.8
Table 4: Comparison of object placement baselines on the HiddenObjects test set. Our baseline distilled model outperforms zero-shot VLMs and other methods for object placement trained on OPA.
Method
PC. ↑
VN. ↑
PDC. ↑
Avg. ↑
OPA Human Annotation
2.72
2.66
2.66
2.68
Ours
Model trained on OPA
3.39
3.33
3.33
3.35
Model trained on HiddenObjects
3.53
3.38
3.38
3.43
Diffusion Annotation (Sec. 3 )
4.05
3.83
3.83
3.90
Table 5: We compare object insertions on OPA using ImgEdit-Judge. A model trained on HiddenObjects (ours) outperforms both the original OPA human annotations as well as a model trained on OPA. The slow pipeline of Sec. 3 , provides the best results.
Figure 9: Image inpainting with object placement priors. We evaluate three object placement approaches on the OPA backgrounds. Human Annot. uses ground-truths from OPA. Random BBox samples a uniformly random placement region, serving as a controlled baseline. Ours uses the top-ranked box of our dataset. Dashed boxes show the inpainting region ( HiddenObject annotation (ours) , OPA annotation , Random ).
Figure 10: Composite Overlay of Inpaintings. Multiple valid inpaintings are aggregated into a single composite scene. Accepted objects are segmented and overlaid on the source background. Insertions are ordered by preference ranking so the highest-ranked object remains unoccluded at the top of the stack.
Figure 11: Placement Distillation Model. Given a background image and a target object class, the model predicts plausible insertion locations using a DETR-style architecture. The transformer decoder operates on a fixed set of learned queries that are conditioned on the target class embedding. Each query predicts a candidate bounding box with its plausibility score. During inference, predicted boxes are ranked by their plausibility scores to obtain the most likely placement locations for the target object.
Figure 12: Speedup Analysis. Top : Distribution of Δ(xt,∅) for 10040 inpaintings. Bottom: Plotting recall versus time across time steps shows that using only the second denoising iteration seems to be sufficient to get a 2x speed-up improvement on an 80% recall. The dotted red line denotes the cost of skipping the first step all along N⋅τ20 .
Figure 13: Bounding Box Proposals. Left : The 12×12 anchor grid. Middle : Full coverage set for an interior anchor. Right : Pruned set for a corner anchor due to boundary constraints.
Figure 14: Spatial Prior . Object placements are computed for the following classes: airplane, apple, bench, bicycle, boat, book, bottle, cake.
Figure 15: Spatial Prior . Object placements are computed for the following classes: car, cat, chair, cow, cup, dog, elephant, fire hydrant.
Figure 16: Spatial Prior . Object placements are computed for the following classes: horse, keyboard, laptop, motorcycle, person, pizza, potted plant, sandwich.
Figure 17: Inpainting with Object Placement Priors. We evaluate three object placement approaches on the OPA backgrounds. Human Annot. uses ground-truths from OPA. Random BBox samples a uniformly random placement region, serving as a controlled baseline. Ours uses the top-ranked box of our dataset. Dashed boxes show the inpainting region ( HiddenObject annotation (ours) , OPA annotation , Random ).
Figure 18: Inpainting with Object Placement Priors. We evaluate three object placement approaches on the OPA backgrounds. Human Annot. uses ground-truths from OPA. Random BBox samples a uniformly random placement region, serving as a controlled baseline. Ours uses the top-ranked box of our dataset. Dashed boxes show the inpainting region ( HiddenObject annotation (ours) , OPA annotation , Random ).
Figure 19: Inpainting with Object Placement Priors. We evaluate three object placement approaches on the OPA backgrounds. Human Annot. uses ground-truths from OPA. Random BBox samples a uniformly random placement region, serving as a controlled baseline. Ours uses the top-ranked box of our dataset. Dashed boxes show the inpainting region ( HiddenObject annotation (ours) , OPA annotation , Random ).
Figure 20: Inpainting with Object Placement Priors. We evaluate three object placement approaches on the OPA backgrounds. Human Annot. uses ground-truths from OPA. Random BBox samples a uniformly random placement region, serving as a controlled baseline. Ours uses the top-ranked box of our dataset. Dashed boxes show the inpainting region ( HiddenObject annotation (ours) , OPA annotation , Random ).
Spatial reasoning studies how systems represent geometry, relations, viewpoints, and motion to understand scenes, plan actions, and solve spatial tasks.
Object placement is critical in image composition, requiring spatially and semantically coherent positioning of objects within diverse scenes. Existing approaches typically rely on hand-crafted rules or supervised learning on limited datasets, which restricts their generalization and interpretability, especially in open-world scenarios involving novel objects and scenes. In this work, we reformulate open-world object placement as a heuristic search task guided by reasoning from a Multimodal Large Language Model (MLLM). We introduce presto, a zero-shot, training-free framework that operates within an imaginary action space to iteratively refine object position and scale. Our coarse-to-fine search strategy ensures fast convergence, and we evaluate two decision-making variants: Metric-guided Selection and MLLM-as-a-judge. Experiments across multiple benchmarks show that presto~achieves state-of-the-art performance, particularly in previously unseen, open-world settings. Human studies further reveal that the MLLM-as-a-judge variant produces more perceptually coherent placements than metric-driven approaches, highlighting a gap between standard evaluation metrics and human visual judgment.
Weixuan Ding, Shang Liu, Hanyu Pei +1
Wuhan University, Wuhan Hubei Province, China · University of Louisville, Louisville KY, USA
As an important operation of image composition, object placement aims to predict the plausible placement (location, scale) for the inserted foreground object. Previous object placement methods can be divided into generative methods and discriminative methods, both of which cannot balance efficiency and effectiveness well. In this work, we propose a semi-generative method in the middle ground between them. In particular, we assign uniformly distributed anchors on the background. Then, we fuse foreground and background features to predict the rationality score for each anchor and predict plausible placement sets for positive anchors. Extensive experiments on the OPA dataset show that our method can strike a good balance between efficiency and effectiveness.
Video object insertion is fundamental to video editing, yet existing diffusion methods often produce visually plausible but physically inconsistent results. We present Place-it-R1, an end-to-end framework for physically plausible video object insertion driven by environment-aware MLLM reasoning. Rather than treating reasoning as a generic text prompt, Place-it-R1 uses the MLLM to analyze the target environment, infer object-scene interactions, and determine where an insertion is physically valid. The resulting reasoning is translated into two complementary forms of guidance for video diffusion: semantic guidance that describes the intended physical interaction and spatial guidance that provides a valid insertion region in each frame. To further align generation with local physical realism, we introduce Spatial Direct Preference Optimization, which leverages an MLLM to rank generated candidates, and introduces a region-aware preference objective that explicitly localizes physical-violation penalties to the inserted-object region. Place-it-R1 further offers flexible and standard modes to trade off environment adaptation and scene preservation. Extensive experiments show that Place-it-R1 produces more physically coherent and visually natural insertions than state-of-the-art methods and achieves competitive results against commercial systems.
Bohai Gu, Taiyi Wu, Dazhao Du +5
The Hong Kong University of Science and Technology · AI Technology Center, Tencent Video, Tencent · Peking University