Reflection Generation for Composite Image Using Diffusion Model

Haonan Zhao; Qingyang Liu; Jiaxuan Chen; Li Niu

cs.CVApr 2, 2026

Reflection Generation for Composite Image Using Diffusion Model

Authors: Haonan Zhao, Qingyang Liu, Jiaxuan Chen, Li Niu

Organizations: Shanghai Jiao Tong University

Abstract

Image composition involves inserting a foreground object into the background while synthesizing environment-consistent effects such as shadows and reflections. Although shadow generation has been extensively studied, reflection generation remains largely underexplored. In this work, we focus on reflection generation. We inject the prior information of reflection placement and reflection appearance into foundation diffusion model. We also divide reflections into two types and adopt type-aware model design. To support training, we construct the first large-scale object reflection dataset DEROBA. Experiments demonstrate that our method generates reflections that are physically coherent and visually realistic, establishing a new benchmark for reflection generation.

Figures & tables

Fig. 1: A composite image can be obtained by placing a foreground object onto the background. Reflection generation seeks to synthesize plausible reflection for the inserted object, thereby enhancing the overall realism and perceptual consistency of the composite image.

Fig. 2: The pipeline of DEROBA dataset construction. Foreground objects are automatically detected and segmented, with their associated reflection masks manually annotated. Inpainting is performed on the source image $I_{src}$ within both foreground and reflection regions to generate a background image $I_{b}$ . To correct color discrepancies, inpainting is again performed on $I_{src}$ with a black mask, leading to the ground-truth image $I_{g}$ . Finally, $I_{b}$ and $I_{g}$ are combined to get the composite image $I_{c}$ . The details can be found in Section III .

Fig. 3: The illustration of our method. Besides denoising U-Net and ControlNet encoder $E_{c}$ , we introduce an auxiliary encoder $E_{b}$ to predict the bounding box regression coefficients $\tilde{l}_{r}$ and reflection type $\tilde{t}_{r}$ . According to the predicted reflection type, we extract the corresponding reference features and use the corresponding reflection type embedding.

Fig. 4: Visual comparison of different methods on DEROBA dataset. From left to right are input composite image, foreground mask, results of SD-ControlNet, FLUX-ControlNet, ICEdit, FLUX-Kontext, Qwen-Edit, our method, and ground-truth.

Method	GR $\downarrow$	LR $\downarrow$	GS $\uparrow$	LS $\uparrow$
SD-ControlNet [ 5 ]	15.726	57.110	0.903	0.103
FLUX-ControlNet [ 6 ]	15.451	57.882	0.898	0.115
ICEdit [ 7 ]	19.464	59.643	0.862	0.095
FLUX-Kontext [ 8 ]	13.839	55.231	0.913	0.134
Qwen-Edit [ 9 ]	12.390	54.797	0.916	0.129
Ours	11.522	53.430	0.923	0.144

TABLE I: The results of different methods on DEROBA test set. The best results are highlighted in boldface.

Fig. 5: Multiple results for one test image on DEROBA test set. From left to right are input composite image (a), foreground object mask (b), results of our method using different random seeds (c)-(g) and ground-truth (h).

Row	Base	Box	Features	Type	GR $\downarrow$	LR $\downarrow$	GS $\uparrow$	LS $\uparrow$
1	+	-	-	-	15.726	57.110	0.903	0.103
2	+	+	-	-	13.917	54.873	0.904	0.131
3	+	-	+	-	12.275	55.354	0.915	0.127
4	+	+	+	-	11.739	53.922	0.918	0.139
5	+	+	+	+	11.522	53.430	0.923	0.144

TABLE II: Ablation studies of our method on DEROBA test set.

Fig. 6: Visual results of failure cases. From left to right, we show composite image, foreground object mask, our result, and ground-truth.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Fig. 3: The distribution of images with different numbers of detected foreground object in DEROBA dataset. Each bar further contains the distribution of images with different numbers of valid instances.

Fig. 4: Examples of DEROBA dataset in six super-categories (“animal”, “vehicle”, “plant”, “building”, “people”, “sundry”). For each super-category, we present two tuples of composite image $I_{c}$ , foreground object mask $M_{fo}$ , foreground reflection mask $M_{r}$ , and ground-truth target image $I_{g}$ .

Fig. 5: The visual results of our ablated versions. From left to right are input composite image (a), foreground mask (b), basic ControlNet (c), only using reference features (d), only using reflection bounding box (e), full version (f), and ground-truth (g).

Metric	Training set	Test set
Acc $\uparrow$	0.962	0.945

Appendix

TABLE I: The classification accuracy of predicted reflection types.

Metric	Training set	Test set	Upper
KFIoU $\uparrow$	0.263	0.246	0.333

Appendix

TABLE II: Quantitative results of the rotated bounding box regression. “Upper” denotes the upper bound of KFIoU.

Fig. 6: Visualization results of rotated bounding box regression. Mask means foreground mask. In the third column, we present the predicted bounding box, while in the fourth column, we overlay it on the ground-truth image.

Fig. 7: Examples from the DEROBA dataset illustrating two types of reflections: “vertical” and “others”. The first two rows show samples of the “vertical” type, while the last two rows correspond to the “others” type. For each example, we present a tuple consisting of the composite image $I_{c}$ , the foreground object mask $M_{fo}$ , the foreground reflection mask $M_{r}$ , and the ground-truth target image $I_{g}$ .

Fig. 8: Visual comparison of different methods on DEROBA dataset. From left to right are input composite image (a), foreground object mask (b), results of SD-ControlNet [ 5 ] (c), FLUX-ControlNet [ 6 ] (d), ICEdit [ 7 ] (e), FLUX-Kontext [ 8 ] (f), Qwen-Edit [ 9 ] (g), our method (h), and ground-truth (i).

Method	Param	Inference Time
Qwen-Edit [ 9 ]	20B	20s
FLUX-Kontext [ 8 ]	12B	13s
Ours	1.3B	2.8s

Appendix

TABLE III: The comparison of model size and inference time with state-of-the-art open-source image editing models.

Compositional Image Generation

Compositional image generation studies how to combine objects, attributes, spatial relations, and visual references into coherent, controllable images.

See all 85 papers

MirrorWorld: Taming Video Diffusion Models for Mirror Reflection Generation
Aug 7, 2026Youjun Zhao, Alex Warren, Gary K. L. Tam +1Video Diffusion Models Video Generation
City University of Hong Kong · Swansea University
PhysMirror: Physics-Aware Mirror Object Generation
Jul 3, 2026Xuan-Bach Mai, Duy-Phuc Nguyen, Quoc-Van Le +6Conditional Image Generation Image Generation
University of Science, Ho Chi Minh City, Vietnam · University of Dayton, Dayton, Ohio, US · Monash University, Melbourne, Victoria, Australia +1
PhysReflect: Geometry and Perception Guided Diffusion for Physically-Plausible Mirror Reflections
Sep 20, 2026Shuheng Ge, Hongwei Ren, Li Zhang +1Diffusion Model Guidance Image Generation
Harbin Institute of Technology, Harbin, China

Aug 7, 2026J/K move · Enter open · S save

MirrorWorld: Taming Video Diffusion Models for Mirror Reflection Generation

Youjun Zhao, Alex Warren, Gary K. L. Tam, Rynson W. H. Lau

City University of Hong Kong · Swansea University

Video Diffusion Models Video Generation Image Inpainting

Open paper AskPDF arXiv

Recent advances in video diffusion models (VDMs) have enabled high-fidelity video synthesis. However, generating mirror reflections remains challenging because the content within a mirror must remain consistent with the surrounding scene. Existing VDMs are not specifically designed to model scene-to-mirror relationships, which can lead to reflections with incorrect content or inconsistent spatial arrangements. We observe that mirror reflection generation involves two complementary challenges: determining what scene content should be reflected and how the reflected content should be spatially arranged within the mirror region. Motivated by this observation, we propose MirrorWorld, a reflection-aware video inpainting framework that models scene-to-mirror relationships during generation. Specifically, we introduce Semantic Relation Distillation (SRD), which transfers relational information from a frozen visual foundation model to encourage semantic associations between visible scene content and mirror regions. We further propose Geometric Transformation Alignment (GTA), which learns a transformation that guides the spatial arrangement of reflected content. The two components play complementary roles, with SRD modeling what should be reflected and GTA modeling how it should be arranged. To facilitate research on this problem, we construct a benchmark for video mirror reflection generation by repurposing four existing video mirror datasets into a unified reflection reconstruction task. Experimental results show that MirrorWorld achieves improved reflection reconstruction quality over representative image-based reflection generation methods and strong video inpainting baselines.

Abstract

Compositional Image Generation

Explore similar work

MirrorWorld: Taming Video Diffusion Models for Mirror Reflection Generation

PhysMirror: Physics-Aware Mirror Object Generation

PhysReflect: Geometry and Perception Guided Diffusion for Physically-Plausible Mirror Reflections