PICS: Pairwise Image Compositing with Spatial Interactions

Hang Zhou; Xinxin Zuo; Sen Wang; Li Cheng

cs.CVMar 6, 2026

PICS: Pairwise Image Compositing with Spatial Interactions

Authors: Hang Zhou, Xinxin Zuo, Sen Wang, Li Cheng

Organizations: University of Alberta · Concordia University

Abstract

Despite strong single-turn performance, diffusion-based image compositing often struggles to preserve coherent spatial relations in pairwise or sequential edits, where subsequent insertions may overwrite previously generated content and disrupt physical consistency. We introduce PICS, a self-supervised composition-by-decomposition paradigm that composes objects in parallel while explicitly modeling the compositional interactions among (fully-/partially-)visible objects and background. At its core, an Interaction Transformer employs mask-guided Mixture-of-Experts to route background, exclusive, and overlap regions to dedicated experts, with an adaptive α-blending strategy that infers a compatibility-aware fusion of overlapping objects while preserving boundary fidelity. To further enhance robustness to geometric variations, we incorporate geometry-aware augmentations covering both out-of-plane and in-plane pose changes of objects. Our method delivers superior pairwise compositing quality and substantially improved stability, with extensive evaluations across virtual try-on, indoor, and street scene settings showing consistent gains over state-of-the-art baselines. Code and data are available at https://github.com/RyanHangZhou/PICS

Figures & tables

Figure 2: Visual comparison of pairwise support relations across Paint-by-Paint, ControlCom, ObjectStitch, AnyDoor, FreeCompose, OmniPaint and InsertAnything . Left: backgrounds and two objects; right: compositing results. The first row shows composites with the basket, and the second row shows subsequent composites obtained by adding the bread on top. Unlike prior methods that suffer from contact artifacts and fidelity loss, our approach performs parallel compositing, effectively handling spatial occlusions and yielding consistent results with preserved fine-grained structure.

Figure 3: Overview of PICS. Input data are constructed by decomposing the target image into a background and pairwise objects with their designated regions. (a) The interaction diffusion network composites the objects into the background. (b) The interaction transformer block, shared across both branches, models interactions among objects and with the background. (c) Expert blocks focus on distinct spatial regions. All notations are defined in the main text for clarity.

Method	mPSNR $\uparrow$	mSSIM $\uparrow$	mLPIPS $\downarrow$	PSNR $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	CLIP-score $\uparrow$	DINOV2-score $\uparrow$	DreamSim $\downarrow$
PbE (CVPR’23)	$10.24$	$0.4241$	$0.4535$	$15.29$	$34.93$	$0.4138$	$81.42$	$0.4320$	$0.4896$
ControlCom (arXiv’23)	$11.82$	$0.3185$	$\underline{0.3986}$	$\underline{17.61}$	$26.93$	$0.3375$	$\mathbf{85.39}$	$0.5264$	$0.3248$
ObjectStitch (CVPR’23)	$10.84$	$0.3471$	$0.4203$	$16.55$	$29.68$	$0.3572$	$85.01$	$0.5574$	$0.3458$
AnyDoor (CVPR’24)	$11.62$	$\underline{0.5283}$	$0.4185$	$17.12$	$27.17$	$\underline{0.3302}$	$84.99$	$\mathbf{0.6089}$	$\underline{0.2820}$
OmniPaint (ICCV’25)	$\underline{12.20}$	$0.3096$	$0.4618$	$16.09$	$\underline{26.25}$	$0.3542$	$83.11$	$0.5673$	$0.2774$
PICS (ours)	$\mathbf{13.88}$	$\mathbf{0.5823}$	$\mathbf{0.3221}$	$\mathbf{18.27}$	$\mathbf{24.99}$	$\mathbf{0.2530}$	$\underline{85.25}$	$\underline{0.5713}$	$\mathbf{0.2659}$

Table 1: Quantitative comparison of object recompositing against prior methods on the LVIS validation set. The prefix “m-” indicates evaluation restricted to the intersection regions. Bold numbers denote the best performance, and underlined numbers indicate the second best.

Figure 4: Qualitative comparison on the LVIS validation set. Source images, backgrounds, and the two decomposed objects are shown on the left. On the right are the recompositing results from different methods. Our approach is the only one that produces composites with realistic spatial interactions between scene objects while maintaining scene consistency and object identity.

Figure 5: Qualitative comparison of different composition orders on the DreamBooth test set. Left: backgrounds and two objects. Right: results from different methods. Our approach better preserves natural contacts and occlusions, while implicitly learning the correct occlusion order.

Table 6

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 11: Examples of pairwise object relations from the LVIS validation set, with object instances visualized alongside their bounding boxes.

Figure 12: Heatmap of spatial relations of pairwise bounding boxes.

Datasets	${\#}$ Training	Type
LVIS ( Gupta et al., 2019 )	34,160	General
VITON-HD ( Choi et al., 2021 )	11,647	Try-on
Objects365 ( Shao et al., 2019 )	940,764	General
Cityscapes ( Cordts et al., 2016 )	536	Street
Mapillary Vistas ( Neuhold et al., 2017 )	603	Street
BDD100K ( Yu et al., 2020 )	1,012	Street

Appendix

Table 5: Statistics and description of our training datasets for pairwise object compositing.

Figure 13: Details of multi-view shape prior.

Figure 14: Inference process of PICS. The background and object embeddings, together with their masks, are fused in latent space and decoded by the VAE to produce the final composite $\mathbf{x}_{comp}$ .

Figure 15: Effect of different classifier-free guidance (CFG) scales on image recompositing, comparing CFG values of 1, 5, and 9.

Figure 16: Four exemplars for (non-)partial 3D reconstruction. From left to right: source image, image with object bounding boxes, 6-view reconstructed object images.

Figure 17: Four exemplars for (non-)partial 3D reconstruction. From left to right: source image, image with object bounding boxes, 6-view reconstructed object images.

Figure 18: CLIP encoder vs DINOv2 encoder. Models are separately trained on LVIS dataset.

Figure 19: Additional visualization of our pairwise compositing with unoccluded inputs.

Figure 20: Additional visualization of w vs w/o 3D augmentation of our method.

Methods	Params	Time (min)	GFLOPs	GPU Memory
PbE ( Yang et al., 2023 )	1.31 G	0.722	1048	8.70 G
ControlCom ( Zhang et al., 2023 )	1.37 G	0.684	1123	21.5 G
ObjectStitch ( Song et al., 2023 )	1.31 G	0.241	852	7.04 G
AnyDoor ( Chen et al., 2024b )	2.45 G	0.342	2450	15.8 G
FreeCompose ( Chen et al., 2024c )	1.07 G	1.912	678	5.79 G
OmniPaint ( Yu et al., 2025 )	12.1 G	1.516	33035	23.1 G

Appendix

Table 6: Comparison of inference-time computation cost for different methods.

Figure 21: Qualitative comparisons for in-the-wild compositing, highlighting spatial relations: support, containment, occlusion, and deformation, corresponding to the teaser figure.

Sample	Paint-by-Example	ControlCom	ObjectStitch	AnyDoor
1	Seat surface missing	Seat distorted, fused with backpack	No physical placement supported	Seat surface disappeared
2	Holder shortened, pen style changed	Pen attached on side of holder	Pen attached on side of holder	Pen attached on side of holder
3	Artifacts between backpacks	Back one occludes front one	Right backpack shows left backpack’s color	Back one occludes front one
4	Toy style changed, backpack deformed	Contact region distorted	Toy style changed, contact region unrealistic	Toy not physically touching blanket
5	Vase shape altered	Vase not placed upright	Vase not placed upright	Vase and flowers both tilted
6	Banana turned into apple, no basket shadow	Banana fused with basket front	Banana placement caused basket gap	Banana partly fused with basket

Appendix

Table 7: Typical artifacts observed in the pairwise compositing results of Figure 21 .

Figure 22: Qualitative comparison on the VITON-HD.

Figure 23: Additional qualitative comparison on the LVIS validation set.

Image Editing

Image editing changes selected visual content while preserving the parts of an image that should remain unchanged.

See all 313 papers

Interact3D: Compositional 3D Generation of Interactive Objects
Mar 17, 2026Hui Shan, Keyang Luo, Ming Li +43D Asset Generation 3D Scene Generation
Zhejiang University, China · Shanghai Innovation Institute, China · Westlake University, China +4
Depth-to-RGB: Repurposing a Frozen Depth Estimator for Geometry-Guided Compositing
Oct 6, 2026Sanghyun Jo, Chae Yeon Lim, Donghwan Lee +3Image Editing Compositional Image Generation
OGQ · Seoul National University · Adobe Research
Direct 3D-Aware Object Insertion via Decomposed Visual Proxies
Jun 4, 2026Jingbo Gong, Yikai Wang, Yushi Lan +6Image Editing Diffusion Models
VCIP, School of Computer Science, Nankai University · Zhongguancun Academy · S-Lab, Nanyang Technological University +2

Mar 17, 2026J/K move · Enter open · S save

Interact3D: Compositional 3D Generation of Interactive Objects

Hui Shan, Keyang Luo, Ming Li, Sizhe Zheng+3

Zhejiang University, China · Shanghai Innovation Institute, China · Westlake University, China+4

3D Asset Generation 3D Scene Generation 3D Shape Generation

Open paper AskPDF arXiv

Recent breakthroughs in 3D generation have enabled the synthesis of high-fidelity individual assets. However, generating 3D compositional objects from single images--particularly under occlusions--remains challenging. Existing methods often degrade geometric details in hidden regions and fail to preserve the underlying object-object spatial relationships (OOR). We present a novel framework Interact3D designed to generate physically plausible interacting 3D compositional objects. Our approach first leverages advanced generative priors to curate high-quality individual assets with a unified 3D guidance scene. To physically compose these assets, we then introduce a robust two-stage composition pipeline. Based on the 3D guidance scene, the primary object is anchored through precise global-to-local geometric alignment (registration), while subsequent geometries are integrated using a differentiable Signed Distance Field (SDF)-based optimization that explicitly penalizes geometry intersections. To reduce challenging collisions, we further deploy a closed-loop, agentic refinement strategy. A Vision-Language Model (VLM) autonomously analyzes multi-view renderings of the composed scene, formulates targeted corrective prompts, and guides an image editing module to iteratively self-correct the generation pipeline. Extensive experiments demonstrate that Interact3D successfully produces promising collsion-aware compositions with improved geometric fidelity and consistent spatial relationships.

Abstract

Image Editing

Explore similar work

Interact3D: Compositional 3D Generation of Interactive Objects

Depth-to-RGB: Repurposing a Frozen Depth Estimator for Geometry-Guided Compositing

Direct 3D-Aware Object Insertion via Decomposed Visual Proxies