Realistic shadow generation is crucial for achieving seamless image compositing, yet existing methods primarily focus on single-object insertion and often fail to generalize when multiple foreground objects are composited into a background scene. In practice, however, modern compositing pipelines and real-world applications often insert multiple objects simultaneously, necessitating shadows that are jointly consistent in terms of geometry, attachment, and location. In this paper, we address the under-explored problem of multi-object shadow generation, aiming to synthesize physically plausible shadows for multiple inserted objects. Our approach exploits the multimodal capabilities of a pre-trained text-to-image diffusion model. An image pathway injects dense, multi-scale features to provide fine-grained spatial guidance, while a text-based pathway encodes per-object shadow bounding boxes as learned positional tokens and fuses them via cross-attention. An attention-alignment loss further grounds these tokens to their corresponding shadow regions. To support this task, we augment the DESOBAv2 dataset by constructing composite scenes with multiple inserted objects and automatically derive prompts combining object category and shadow positioning information. Experimental results demonstrate that our method achieves state-of-the-art performance in both single and multi-object shadow generation settings.
Figures & tables
Fig. 1: Architectural Overview. From a composite image Ic and corresponding object masks Om , a Shadow-Box Predictor estimates per-object shadow bounding boxes, which are quantized into grid bins and converted into shadow positional tokens. These positional tokens are inserted into the prompt and processed through a CLIP encoder to provide text-conditioned spatial grounding. On the image pathway, a Feature Encoder extracts multi-scale visual features and injects them into the diffusion UNet. Both conditioning streams jointly guide the diffusion model to generate the final shadowed image Ig .
Fig. 2: ViP-LLaVA object naming from bounding-box prompts. Given the prompt “Name the objects in the bounding boxes.”, the model generates: (Left: “Girl riding a motorbike”), (Center: “Man in blue shirt.” and “Woman in white dress.”), (Right: “The pole.” and “Second pole from the left.”).
Fig. 3: Visual comparison with state-of-the-art baseline methods for single object shadow generation. Our method (MultiShadow), given the same image inputs plus a compact text prompt of category terms and shadow positional tokens, produces better shadows for all objects (e.g., Row 1: “a woman casting shadow [sx_8][sy_9][sx_3][sy_8]”). Positional tokens are inserted automatically; see Sec. III-B .
Fig. 4: Visual comparison with state-of-the-art baseline methods for multiple object shadow generation. Our method (MultiShadow), given the same image inputs plus a compact text prompt of category terms and shadow positional tokens, produces better shadows for all objects (e.g., Row 1: “a girl casting shadow [sx_3][sy_8][sx_8][sy_9]; a boy casting shadow [sx_4][sy_9][sx_10][sy_11]”). Positional tokens are inserted automatically; see Sec. III-B .
Fig. 5: Visual comparison with state-of-the-art baseline methods on real composited images. Our method (MultiShadow), given the same image inputs plus a compact text prompt of category terms and shadow positional tokens, produces better shadows for all objects (e.g., Row 1: “a ball casting shadow […]; a bird casting shadow […], an cone object casting shadow […]”). Positional tokens are inserted automatically; see Sec. III-B .
Fig. 6: Incremental visual ablation from simple image conditioning (SIC) to the full configuration. From left to right: Shadow-free composite, Object Mask, SIC: Simple Image Conditioning, GAAM: Geometry-Aware Affine Modulation, GP: General Prompt, SPT: Shadow Positional Tokens, AAL: Attention Alignment Loss, GT: Ground Truth.
Method
BOS Test Images
BOS-free Test Images
GR ↓
LR ↓
GS ↑
LS ↑
GB ↓
LB ↓
GR ↓
LR ↓
GS ↑
LS ↑
GB ↓
LB ↓
SGRNet [ 26 ]
7.184
68.255
0.964
0.206
0.301
0.596
15.596
60.350
0.909
0.100
0.271
0.534
DMASNet [ 2 ]
8.256
59.380
0.961
0.228
0.276
0.547
18.725
86.694
0.913
0.055
0.297
0.574
SGDiffusion [ 5 ]
6.098
53.611
0.971
0.370
0.245
0.487
15.110
55.874
0.913
0.117
0.233
0.452
GPSDiffusion [ 6 ]
5.896
46.713
0.966
0.374
0.213
0.423
13.809
55.616
0.917
0.166
0.197
0.384
MetaShadow [ 7 ]
5.961
47.328
0.968
0.373
0.237
0.458
14.698
55.781
0.916
0.152
0.209
0.426
TABLE I: Quantitative comparison of single-object shadow generation performance on the original DESOBAv2 [ 13 ] test split.
Method
BOS Test Images (Multi-Object)
BOS-free Test Images (Multi-Object)
GR ↓
LR ↓
GS ↑
LS ↑
GB ↓
LB ↓
GR ↓
LR ↓
GS ↑
LS ↑
GB ↓
LB ↓
SGRNet [ 26 ]
9.842
85.327
0.938
0.152
0.412
0.723
18.936
78.452
0.872
0.068
0.385
0.698
DMASNet [ 2 ]
10.125
72.893
0.935
0.178
0.367
0.654
21.583
94.217
0.885
0.042
0.394
0.712
SGDiffusion [ 5 ]
8.327
67.425
0.952
0.285
0.324
0.592
17.892
68.327
0.894
0.089
0.315
0.567
GPSDiffusion [ 6 ]
7.894
58.642
0.948
0.301
0.294
0.528
16.427
67.893
0.901
0.124
0.278
0.489
MetaShadow [ 7 ]
8.293
60.519
0.951
0.293
0.304
0.573
17.253
68.032
0.904
0.117
0.293
0.526
TABLE II: Performance evaluation on the extended DESOBAv2 [ 13 ] dataset for multi-object shadow generation.
Method
K=1
K=2
K=3 – 4
K≥5
SGRNet [ 26 ]
0.746
0.762
0.852
0.930
DMASNet [ 2 ]
0.673
0.691
0.720
0.886
SGDiffusion [ 5 ]
0.645
0.674
0.785
0.842
GPSDiffusion [ 6 ]
0.514
0.573
0.617
0.662
MetaShadow [ 7 ]
0.539
0.582
0.630
0.687
MultiShadow (Ours)
0.395
0.403
0.412
0.426
TABLE III: Object count K vs. local BER (LB ↓ ; lower is better).
Method
Bradley–Terry Score ↑
Single-Object
Multi-Object
SGRNet [ 26 ]
0.191
0.013
DMASNet [ 2 ]
0.353
0.189
SGDiffusion [ 5 ]
0.552
0.378
GPSDiffusion [ 6 ]
0.912
0.734
MetaShadow [ 7 ]
0.889
0.701
TABLE IV: User study results on real composite images for both single-object and multi-object shadow generation.
Generating realistic cast shadows for inserted foreground objects is a crucial yet challenging problem in image composition, where maintaining geometric consistency of shadow and object in complex scenes remains difficult due to the ill-posed nature of shadow formation. To address this issue, we propose VSDiffusion, a visibility-constrained two-stage framework designed to narrow the solution space by incorporating visibility priors. In Stage I, we predict a coarse shadow mask to localize plausible shadow generated regions. And in Stage II, conditional diffusion is performed guided by lighting and depth cues estimated from the composite to generate accurate shadows. In VSDiffusion, we inject visibility priors through two complementary pathways. First, a visibility control branch with shadow-gated cross attention that provides multi-scale structural guidance. Then, a learned soft prior map that reweights training loss in error-prone regions to enhance geometric correction. Additionally, we also introduce high-frequency guided enhancement module to sharpen boundaries and improve texture interaction with the background. Experiments on widely used public DESOBAv2 dataset demonstrated that our proposed VSDiffusion can generate accurate shadow, and establishes new SOTA results across most evaluation metrics.
Jing Li, Jing Zhang
East China University of Science and Technology, Shanghai, China
Generating realistic cast shadows for inserted foreground objects requires reasoning about scene geometry and illumination. However, most learning-based approaches treat shadow generation as an image translation problem and capture these physical relationships only implicitly. This often results in misaligned or implausible shadows. Motivated by the physics of shadow formation, we introduce explicit geometric guidance for outdoor shadow generation. Given a composite image and a foreground object mask, we recover approximate scene geometry and estimate a dominant light direction to derive a coarse shadow estimate via geometric reasoning. While coarse, this estimate provides a spatial anchor for shadow placement. Because illumination cannot always be uniquely inferred from a single image, we predict confidence scores for both lighting and shadow cues and use them to regulate their influence during generation. These cues (shadow mask, light direction, and their confidence scores) condition a diffusion-based generator that refines the estimate into a realistic shadow. Experiments on DESOBAV2 show substantially improved shadow-region fidelity and localization, with an overall 23% lower shadow-region RMSE and 30% lower shadow-mask BER than the prior state-of-the-art method.
Shilin Hu, Jingyi Xu, Akshat Dave +2
Stony Brook University, Stony Brook, NY 11794, USA · UNC Charlotte, Charlotte, NC 28223, USA
Image composition involves inserting a foreground object into the background while synthesizing environment-consistent effects such as shadows and reflections. Although shadow generation has been extensively studied, reflection generation remains largely underexplored. In this work, we focus on reflection generation. We inject the prior information of reflection placement and reflection appearance into foundation diffusion model. We also divide reflections into two types and adopt type-aware model design. To support training, we construct the first large-scale object reflection dataset DEROBA. Experiments demonstrate that our method generates reflections that are physically coherent and visually realistic, establishing a new benchmark for reflection generation.