DecoEdit: Beyond Flat Control for 3D-Aware Unified Image Editing
Organizations: Zhejiang University · Alibaba Group · University of Illinois at Urbana-Champaign
Abstract
While recent advances in multimodal image editing have achieved impressive results, current frameworks remain largely confined to ``flat" 2D controls (e.g., text prompts, bounding boxes, or masks). Precise 3D-aware geometric manipulation -- such as coordinating an object's location, physical scale, and 3D orientation while strictly preserving its fine-grained identity -- remains an elusive challenge, typically suffering from severe detail degradation or requiring expensive per-instance fine-tuning. In this paper, we present DecoEdit, an all-in-one image editing framework that goes beyond flat control by seamlessly integrating 1D textual, 2D spatial, and 3D volumetric inputs under a unified architecture. First, to power this unified model, we establish a data curation pipeline and curate a large-scale dataset, namely DecoDataset, containing 1.6 million samples, encompassing diverse multi-dimensional (1D/2D/3D) tasks and their complex compositions. Furthermore, we establish a comprehensive image editing benchmark named DecoBench to systematically evaluate multi-task editing performance. Finally, to address the fundamental trade-off between complex spatial transformations and identity preservation, we propose a novel decoupled pipeline: DecoEdit factorizes the editing process by utilizing structural guidance from SAM-3D to anchor the target geometry, scale, and spatial orientation, while concurrently leveraging a detail-preservation path to lock in high-fidelity textures. Extensive experiments demonstrate that DecoEdit achieves state-of-the-art results. Notably, on the challenging 3D spatial pose and orientation control tasks, DecoEdit significantly outperforms leading closed-source models such as Seedream 5.0 in both geometric accuracy and identity consistency.
Figures & tables
| Dataset | Size | Tasks | MS | 2D Spatial | 2D Object | 3D View | 3D Object | Joint 1D/2D/3D |
|---|---|---|---|---|---|---|---|---|
| InstructPix2Pix | 313K | - | – | – | – | – | – | – |
| HQ-Edit | 200K | - | – | – | – | – | – | – |
| Pico-Banana-400K | 400K | 35 | – | – | – | – | – | – |
| Emu Edit | 10M | 16 | – | – | – | – | – | – |
| OmniEdit | 1.2M | 7 | – | – | – | – | – | |
| MagicBrush | 10K | - | – | – | – | – | – |
| Model | IA | SCA | PVA | IAF | NCP | IEQ | DCP | Macro | DiT Param. |
|---|---|---|---|---|---|---|---|---|---|
| Open-source models | |||||||||
| HiDream-O1-Image-Dev | 1.17 | 1.57 | 2.21 | 2.14 | 1.17 | 2.04 | 0.27 | 2.34 | 8B |
| HiDream-O1-Image | 2.08 | 3.15 | 3.51 | 2.88 | 3.11 | 2.85 | 1.08 | 3.52 | 8B |
| BAGEL | 2.54 | 3.35 | 4.24 | 3.20 | 3.10 | 2.61 | 1.69 | 3.55 | 7B |
| OmniGen | 0.53 | 0.78 | 0.85 | 0.71 | 0.97 | 1.48 | 0.29 | 1.26 | 3.8B |
| OmniGen2 | 2.75 | 3.46 | 4.70 | 4.53 | 5.06 | 5.01 | 2.47 | 4.82 | 4B |
| Rater | Ours | Seedream 5.0 | HY-Image 3.0 | Gemini 2.5 Flash | Qwen-Image-Edit | Spearman | -value | Kendall |
|---|---|---|---|---|---|---|---|---|
| Rater 1 | 72.50 | 62.75 | 44.50 | 44.75 | 25.50 | 0.644 | 0.547 | |
| Rater 2 | 61.25 | 71.25 | 48.25 | 41.25 | 28.00 | 0.451 | 0.383 | |
| Rater 3 | 74.25 | 66.00 | 40.00 | 42.25 | 27.50 | 0.676 | 0.592 | |
| Rater 4 | 73.00 | 65.25 | 44.50 | 41.00 | 26.25 | 0.732 | 0.655 | |
| Average | 70.25 | 66.31 | 44.31 | 42.31 | 26.81 | 0.733 | 0.648 |
| Task | Configuration | IA | SCA | PVA | IAF | NCP | IEQ | DCP | Avg. |
|---|---|---|---|---|---|---|---|---|---|
| Object addition | Baseline | 5.73 3.52 | 6.00 3.07 | 6.35 2.81 | 6.22 2.59 | 8.91 0.64 | 6.77 2.52 | 5.08 3.99 | 6.60 2.58 |
| w/o Decoupling | 8.53 0.72 | 8.66 0.41 | 8.47 0.69 | 8.16 0.65 | 9.32 0.23 | 8.87 0.42 | 8.57 0.50 | 8.65 0.53 | |
| w/o Check | 8.80 0.45 | 8.98 0.09 | 8.86 0.30 | 8.44 0.37 | 9.23 0.32 | 8.84 0.45 | 8.88 0.19 | 8.85 0.33 | |
| Ours | 9.25 | 9.07 | 9.16 | 8.81 | 9.55 | 9.29 | 9.07 | 9.18 | |
| Appearance editing | Baseline | 5.58 3.12 | 8.76 0.76 | 8.70 0.75 | 5.39 3.03 | 9.61 | 8.61 | – | 7.77 1.22 |
| w/o Decoupling | 8.55 0.15 | 9.45 0.07 | 9.48 | 8.18 0.24 | 9.30 0.22 | 8.21 0.12 | – | 8.86 0.13 |
Image Editing
Image editing changes selected visual content while preserving the parts of an image that should remain unchanged.
See all 337 papers