SpatialDiff: 3D-Aware Object Movement via Implicit Spatial Modeling
Authors: Zheng Liu, Zijian He, Huiguo He, Weizhi Zhong, Yejun Tang, Huan Yang, Kun Gai, Guanbin Li
Organizations: Sun Yat-sen University · South China University of Technology · Kuaishou Technology · Shenzhen Loop Area Institute · Guangdong Key Laboratory of Big Data Analysis and Processing
Recent advances in image editing allow impressive manipulation of objects, existing methods still struggle to handle spatial movement in complex scenes, such as objects span different depth layers or are partially occluded. Most image editing methods focus solely on prior information from 2D datasets, emphasizing planar features while lacking support for spatial structures. Even approaches that incorporate explicit positional information fail to capture true 3D spatial relationships, thus limiting accurate object movement in complex scenes. In this paper, we present SpatialDiff, a method that effectively captures 3D spatial structures, enabling precise and consistent object movements in complex scenes. Our core innovations are twofold: (1) Implicit 3D Spatial Modeling, which introduces 3D prior knowledge and enables the model to internally build a comprehensive understanding of the three-dimensional spatial structure; and (2) Global Spatial Supervision, which constrains the latent spatial features to enable the model to perceive changes in object spatial positions caused by editing operations. Experimental results demonstrate that our method significantly improves the accuracy and fidelity of spatial movement in complex scenes.
Figures & tables
Figure 2: (a) Examples of failure cases from other methods: (1) fail to maintain natural appearance and 3D consistency; (2) fail to perform object movement correctly in complex scenes. (b) Examples of successful edits by SpatialDiff.
Figure 3: The pipeline of SpatialDiff. SpatialDiff leverages a single image along with its latent 3D scene representation to achieve precise control over object positions in 2D image scenes, while preserving the global spatial structure and semantic consistency.
Figure 4: Qualitative comparison . The leftmost image represents the input image, with the target object to be edited shown below it. Each example is accompanied by the corresponding editing instruction, where the blue text indicates the spatial position involved in the editing operation.
Method
GPT-SC ↑
GPT-PQ ↑
GPT-O ↑
Qwen-SC ↑
Qwen-PQ ↑
Qwen-O ↑
Flux-Kontext 2
0.292
0.848
0.498
0.261
0.767
0.447
OmniGen2 40
0.301
0.661
0.446
0.305
0.687
0.458
Step1X-Edit 21
0.484
0.785
0.616
0.492
0.691
0.583
BAGEL 47
0.368
0.709
0.511
0.390
0.642
0.500
Qwen-Image-Editing 39
0.666
0.882
0.766
0.632
0.813
0.717
SpatialDiff (Ours)
0.803
0.886
0.843
0.778
0.838
0.807
Table 1: Quantitative comparison on SpatialBench. GPT-SC, GPT-PQ, and GPT-O refer to the metrics evaluated by GPT-5, while Qwen-SC, Qwen-PQ, and Qwen-O refer to the metrics evaluated by Qwen3-VL-32B-Instruct. Scores range from 0 to 1. ↑ : higher is better.
Method
FT
ISM
EDS
LDS
GPT-SC ↑
GPT-PQ ↑
GPT-O ↑
Qwen-SC ↑
Qwen-PQ ↑
Qwen-O ↑
Baseline
0.236
0.831
0.443
0.310
0.723
0.473
Model A
✓
0.398
0.795
0.563
0.427
0.739
0.562
Model B
✓
✓
0.518
0.743
0.620
0.513
0.684
0.592
Model C
✓
✓
✓
0.566
0.801
0.673
0.527
0.745
0.627
SpatialDiff
✓
✓
✓
0.804
0.871
0.837
0.796
0.835
0.815
Table 2: Ablation study on SpatialBench. FT denotes fine-tuning the attention modules of DiT using LoRA. ISM (Implicit 3D Spatial Modeling) represents introducing 3D tokens from the input image through a 3D foundation model, and aligning them to the DiT feature space. EDS (Explicit Depth Supervision) indicates applying supervision on 3D tokens in the pixel space, while LDS (Latent Depth Supervision) applies supervision in the VAE latent space. GPT-SC, GPT-PQ, and GPT-O refer to the metrics evaluated by GPT-5, while Qwen-SC, Qwen-PQ, and Qwen-O refer to the metrics evaluated by Qwen3-VL-32B-Instruct. Scores range from 0 to 1. ↑ : higher is better.
Figure 5: Ablation study . Model A denotes LoRA fine-tuning applied on the Baseline using the movement dataset; Model B denotes the introduction of Implicit 3D Spatial Modeling only; Model C denotes the incorporation of both Implicit 3D Spatial Modeling and Explicit Depth Supervision.
Method
H-SC ↑
H-PQ ↑
H-O ↑
Flux-Kontext 2
1.874
3.433
2.536
OmniGen2 40
2.258
2.692
2.465
Step1X-Edit 21
2.271
2.616
2.437
BAGEL 47
2.759
3.351
3.041
Qwen-Image-Editing 39
3.513
3.611
3.562
SpatialDiff (Ours)
4.711
4.720
4.715
Table 3: User Study . H-SC refers to the assessment of instruction-following ability by humans, H-PQ refers to the evaluation of image quality by humans, and H-O represents the combined metric of both. Scores range from 1 to 5. ↑ : higher is better.
Figure 6: Screenshot of our user study rating interface.
Figure 7: Limitations of the approach.
Method
GPT-SC ↑
GPT-PQ ↑
GPT-O ↑
Qwen-SC ↑
Qwen-PQ ↑
Qwen-O ↑
Flux-Kontext 2
0.424
0.831
0.594
0.253
0.858
0.466
OmniGen2 40
0.416
0.587
0.494
0.407
0.691
0.530
Step1X-Edit 21
0.528
0.637
0.580
0.552
0.765
0.650
BAGEL 47
0.697
0.518
0.601
0.643
0.700
0.671
Qwen-Image-Editing 39
0.783
0.807
0.795
0.746
0.839
0.791
SpatialDiff (Ours)
0.891
0.833
0.862
0.776
0.847
0.811
Table 4: Quantitative comparison on OBJect-3DIT. GPT-SC, GPT-PQ, and GPT-O refer to the metrics evaluated by GPT-5, while Qwen-SC, Qwen-PQ, and Qwen-O refer to the metrics evaluated by Qwen3-VL-32B-Instruct. Scores range from 0 to 1. ↑ : higher is better.
Method
GPT-SC ↑
GPT-PQ ↑
GPT-O ↑
Qwen-SC ↑
Qwen-PQ ↑
Qwen-O ↑
Nano Banana 2
0.711
0.892
0.796
0.594
0.840
0.706
SpatialDiff (Ours)
0.803
0.886
0.843
0.778
0.838
0.807
Table 5: Quantitative results compared with a commercial baseline.
Method
GPT-SC ↑
GPT-PQ ↑
GPT-O ↑
Qwen-SC ↑
Qwen-PQ ↑
Qwen-O ↑
Full model w/o LDS
0.518
0.743
0.620
0.513
0.684
0.592
Full model w/o ISM
0.656
0.882
0.761
0.635
0.775
0.702
Full model (SpatialDiff)
0.804
0.871
0.837
0.796
0.835
0.815
Table 6: Additional Ablation Studies on the ISM Module.
Figure 8: Qualitative comparison on OBJect-3DIT. The leftmost image represents the input image, with the target object to be edited shown below it. Each example is accompanied by the corresponding editing instruction, where the blue text indicates the spatial position involved in the editing operation.
Figure 9: More results of SpatialDiff . Each example is accompanied by the corresponding editing instruction, where the blue text indicates the spatial position involved in the editing operation.