cs.CVApr 12, 2026
SaveHiddenObjects: Scalable Diffusion-Distilled Spatial Priors for Object Placement
Organizations: Technical University of Denmark · Pioneer Centre for AI · University of Copenhagen
Abstract
We propose a method to learn explicit, class-conditioned spatial priors for object placement in natural scenes by distilling the implicit placement knowledge encoded in text-conditioned diffusion models. Prior work relies either on manually annotated data, which is inherently limited in scale, or on inpainting-based object-removal pipelines, whose artifacts promote shortcut learning. To address these limitations, we introduce a fully automated and scalable framework that evaluates dense object placements on high-quality real backgrounds using a diffusion-based inpainting pipeline. With this pipeline, we construct HiddenObjects, a large-scale dataset comprising 27M placement annotations, evaluated across 27k distinct scenes, with ranked bounding box insertions for different images and object categories. Experimental results show that our spatial priors outperform sparse human annotations on a downstream image editing task (3.90 vs. 2.68 VLM-Judge), and significantly surpass existing placement baselines and zero-shot Vision-Language Models for object placement. Furthermore, we distill these priors into a lightweight model for fast practical inference (230,000x faster).
Figures & tables
Figure 2: Background artifacts in PIPE [ 52 ] . Object removal pipelines are effective at removing the requested object but leave visible traces that can facilitate shortcut learning, e.g., (left) blurry area and the shadow presence, or (right) larger artifacts.
| Dataset | # Annotation | # Positive Box | # Negative Box | Real | Dense | Scalable | Artifact-free |
|---|---|---|---|---|---|---|---|
| Boxes | (per instance) | (per instance) | Images | Priors | Background | ||
| DreamEditBench [ 26 ] | 220 | 1 | 0 | ✓ | ✗ | ✗ | ✓ |
| MureCom [ 32 ] | 640 | 1 | 0 | ✓ | ✗ | ✗ | ✓ |
| OPA [ 31 ] | 0.07M | 1.9 | 3.8 | ✓ | ✗ | ✗ | ✓ |
| OPA-ext [ 31 ] | 0.15M | 2.2 | 4.3 | ✓ | ✗ | ✗ | ✓ |
| PIPE [ 52 ] | 0.89M | 1 | 0 | ✗ | ✗ | ✓ | ✗ |
Table 1: Object Placement Datasets. Comparison of common datasets for learning object placement models. Manually annotated datasets are limited in scale, while automated approaches often introduce visual artifacts via inpainting. Our proposed HiddenObjects dataset achieves both scalability, high-quality dense bounding box annotations, and artifact-free backgrounds.
Figure 3: Spatial Prior Extraction. Given a background image , the inpainter synthesizes an image for target class at a bounding box . The verifier evaluates the inpainted images , yielding a detected bounding box and a human-aligned preference score . Repeating this process across multiple sampled boxes yields the ground-truth spatial prior for the j-th class. The heatmap visualizes the aggregated dense distribution of preferences over spatial locations ( no , mid , good ).
Figure 4: Spatial Prior ( HiddenObjects ) aggregated as a heatmap. ( Left ) We visualize the mean aggregated spatial prior across all images of HiddenObjects for different classes listed on the left. Classes such as bench and boat tend to appear near the bottom of images, while kite occurs more frequently near the top. These patterns reflect both semantic regularities (e.g., kite in the sky) and inherent photographic biases in image datasets. ( Right ) We visualize per-instance random example of spatial priors.
Figure 5: Verifier Comparison. For each verifier, we display the inpainting generation of the highest-scoring location. CLIP Score often prefers insertions irrespective of global scene context. In contrast, Image and Vision Reward outperform aesthetic score in judging realism and semantic coherence, leading to more human-aligned spatial priors.
Figure 6: Inpainter Comparison. We compare cross-attention-based inpainters with ControlNet-based variants. Cross-attention-based inpainters are prone to (i) inpainting artifacts or (ii) inpaint objects that don’t respect scene semantics. While ControlNet-based methods more reliably enforce scene constraints: they either (iii) refuse implausible insertions or generate (valid) semantically coherent, context-aware objects.
| Inpainter | Ranker | AP |
|---|---|---|
| FLUX | Random | 52.8 |
| FLUX | Aesthetics | 52.6 |
| FLUX | CLIP Score | 53.8 |
| FLUX | ImageReward | 57.1 |
Table 2: Component Analysis. We measure inpainting and ImageReward performance (Left) We evaluate different rankers for the same inpainter FLUX. (Right) We evaluate different inpainters for the same Ranker. ImageReward and Qwen perform best. IR: ImageReward, CN: ControlNet.
| Method | Test set | PC | VN | PDC | Avg |
|---|---|---|---|---|---|
| Raw background | HiddenObjects | 1.04 | 1.03 | 1.03 | 1.04 |
| Full Mask | HiddenObjects | 1.62 | 1.60 | 1.59 | 1.60 |
| Random Bounding Box | HiddenObjects | 2.73 | 2.62 | 2.62 | 2.65 |
| Ours (Annotation Pipeline) | HiddenObjects | 3.83 | 3.63 | 3.63 | 3.69 |
| Raw background | OPA | 1.00 | 1.00 | 1.00 | 1.00 |
| Full Mask | OPA | 1.66 | 1.66 | 1.66 | 1.66 |
Table 3: Downstream image editing quality evaluated by ImgEdit-Judge [ 57 ] (1-5). We compare our annotation pipeline against various placement strategies.
Figure 7: Image inpainting with object placement priors. We evaluate four object placement approaches on the OPA [ 31 ] and HiddenObjects backgrounds. Full Mask inpaints the entire image. Human Annot. uses ground-truths from OPA. Random BBox samples a uniformly random placement region, serving as a controlled baseline. Ours uses the top-ranked box of our dataset. Dashed boxes show the inpainting region ( HiddenObject annotation (ours) , OPA annotation , Random ). Corner scores are ImgEdit-Judge [ 57 ] . Our pipeline consistently produces well-placed, photorealistic composites across object classes and image scenes, outperforming sparse human placement.
Figure 8: Analysis of Spatial and Scale Distributions. (a) Object center distributions across datasets: while traditional benchmarks like COCO [ 29 ] , VOC [ 14 ] , and ImageNet [ 11 ] exhibit a pronounced center-crop bias, our pipeline (evaluated on HiddenObjects and OPA backgrounds) yields a more diffuse distribution with higher spatial variance. (b) Distribution of bounding box areas: HiddenObjects and PIPE [ 52 ] lack small objects (0.01%–0.1% image area), which reveals a shared resolution bottleneck of underlying diffusion backbones in both generating or removing small objects.
| Method | Train Set | mAP | IoU50@1 | IoU@1 | IoU50@5 | IoU@5 |
|---|---|---|---|---|---|---|
| (%) | (%) | (%) | (%) | (%) | ||
| Qwen2.5-VL-72B | Zero-shot | 1.3 | 7.9 | 14.0 | 12.7 | 18.8 |
| Qwen2.5-VL-7B | Zero-shot | 1.4 | 8.6 | 13.9 | 10.1 | 16.5 |
| InternVL3-8B | Zero-shot | 0.3 | 3.3 | 11.2 | 9.2 | 18.7 |
| LLaVA-OneVision-7B | Zero-shot | 1.0 | 11.7 | 20.9 | 16.3 | 24.7 |
| LLaVA-1.5-7B | Zero-shot | 1.0 | 6.4 | 17.0 | 8.8 | 22.8 |
Table 4: Comparison of object placement baselines on the HiddenObjects test set. Our baseline distilled model outperforms zero-shot VLMs and other methods for object placement trained on OPA.
| Method | PC. | VN. | PDC. | Avg. | |
|---|---|---|---|---|---|
| OPA Human Annotation | 2.72 | 2.66 | 2.66 | 2.68 | |
| Ours | Model trained on OPA | 3.39 | 3.33 | 3.33 | 3.35 |
| Model trained on HiddenObjects | 3.53 | 3.38 | 3.38 | 3.43 | |
| Diffusion Annotation (Sec. 3 ) | 4.05 | 3.83 | 3.83 | 3.90 |
Table 5: We compare object insertions on OPA using ImgEdit-Judge. A model trained on HiddenObjects (ours) outperforms both the original OPA human annotations as well as a model trained on OPA. The slow pipeline of Sec. 3 , provides the best results.
Figure 9: Image inpainting with object placement priors. We evaluate three object placement approaches on the OPA backgrounds. Human Annot. uses ground-truths from OPA. Random BBox samples a uniformly random placement region, serving as a controlled baseline. Ours uses the top-ranked box of our dataset. Dashed boxes show the inpainting region ( HiddenObject annotation (ours) , OPA annotation , Random ).
Figure 10: Composite Overlay of Inpaintings. Multiple valid inpaintings are aggregated into a single composite scene. Accepted objects are segmented and overlaid on the source background. Insertions are ordered by preference ranking so the highest-ranked object remains unoccluded at the top of the stack.
Figure 11: Placement Distillation Model. Given a background image and a target object class, the model predicts plausible insertion locations using a DETR-style architecture. The transformer decoder operates on a fixed set of learned queries that are conditioned on the target class embedding. Each query predicts a candidate bounding box with its plausibility score. During inference, predicted boxes are ranked by their plausibility scores to obtain the most likely placement locations for the target object.
Figure 12: Speedup Analysis. Top : Distribution of for 10040 inpaintings. Bottom: Plotting recall versus time across time steps shows that using only the second denoising iteration seems to be sufficient to get a 2x speed-up improvement on an 80% recall. The dotted red line denotes the cost of skipping the first step all along .
Figure 13: Bounding Box Proposals. Left : The anchor grid. Middle : Full coverage set for an interior anchor. Right : Pruned set for a corner anchor due to boundary constraints.
Figure 14: Spatial Prior . Object placements are computed for the following classes: airplane, apple, bench, bicycle, boat, book, bottle, cake.
Figure 15: Spatial Prior . Object placements are computed for the following classes: car, cat, chair, cow, cup, dog, elephant, fire hydrant.
Figure 16: Spatial Prior . Object placements are computed for the following classes: horse, keyboard, laptop, motorcycle, person, pizza, potted plant, sandwich.
Figure 17: Inpainting with Object Placement Priors. We evaluate three object placement approaches on the OPA backgrounds. Human Annot. uses ground-truths from OPA. Random BBox samples a uniformly random placement region, serving as a controlled baseline. Ours uses the top-ranked box of our dataset. Dashed boxes show the inpainting region ( HiddenObject annotation (ours) , OPA annotation , Random ).
Figure 18: Inpainting with Object Placement Priors. We evaluate three object placement approaches on the OPA backgrounds. Human Annot. uses ground-truths from OPA. Random BBox samples a uniformly random placement region, serving as a controlled baseline. Ours uses the top-ranked box of our dataset. Dashed boxes show the inpainting region ( HiddenObject annotation (ours) , OPA annotation , Random ).
Figure 19: Inpainting with Object Placement Priors. We evaluate three object placement approaches on the OPA backgrounds. Human Annot. uses ground-truths from OPA. Random BBox samples a uniformly random placement region, serving as a controlled baseline. Ours uses the top-ranked box of our dataset. Dashed boxes show the inpainting region ( HiddenObject annotation (ours) , OPA annotation , Random ).
Figure 20: Inpainting with Object Placement Priors. We evaluate three object placement approaches on the OPA backgrounds. Human Annot. uses ground-truths from OPA. Random BBox samples a uniformly random placement region, serving as a controlled baseline. Ours uses the top-ranked box of our dataset. Dashed boxes show the inpainting region ( HiddenObject annotation (ours) , OPA annotation , Random ).
More in this topic
Spatial Reasoning
Spatial reasoning studies how systems represent geometry, relations, viewpoints, and motion to understand scenes, plan actions, and solve spatial tasks.
See all 157 papers