Referring Layer Decomposition

Fangyi Chen; Yaojie Shen; Lu Xu; Ye Yuan; Shu Zhang; Yulei Niu; Longyin Wen

cs.CVFeb 22, 2026

Referring Layer Decomposition

Authors: Fangyi Chen, Yaojie Shen, Lu Xu, Ye Yuan, Shu Zhang, Yulei Niu, Longyin Wen

Organizations: Intelligent Editing Team, Intelligent Creation, ByteDance

Abstract

Precise, object-aware control over visual content is essential for advanced image editing and compositional generation. Yet, most existing approaches operate on entire images holistically, limiting the ability to isolate and manipulate individual scene elements. In contrast, layered representations, where scenes are explicitly separated into objects, environmental context, and visual effects, provide a more intuitive and structured framework for interpreting and editing visual content. To bridge this gap and enable both compositional understanding and controllable editing, we introduce the Referring Layer Decomposition (RLD) task, which predicts complete RGBA layers from a single RGB image, conditioned on flexible user prompts, such as spatial inputs (e.g., points, boxes, masks), natural language descriptions, or combinations thereof. At the core is the RefLade, a large-scale dataset comprising 1.11M image-layer-prompt triplets produced by our scalable data engine, along with 100K manually curated, high-fidelity layers. Coupled with a perceptually grounded, human-preference-aligned automatic evaluation protocol, RefLade establishes RLD as a well-defined and benchmarkable research task. Building on this foundation, we present RefLayer, a simple baseline designed for prompt-conditioned layer decomposition, achieving high visual fidelity and semantic alignment. Extensive experiments show our approach enables effective training, reliable evaluation, and high-quality image decomposition, while exhibiting strong zero-shot generalization capabilities.

Figures & tables

Figure 1: The RefLayer model trained on RefLade demonstrating the RLD task. Each row presents two different prompts and their corresponding layer outputs for the same input image. Given an image and diverse user prompts, RLD requires the model to generate targeted and complete RGBA layers. The figure showcases examples across prompting modes: Spatial Prompting (e.g., points, boxes, masks) and Linguistic Prompting (e.g., text descriptions like “the brown and white horse” or “background”). Coarse prompts such as a single point may lead to coarse-grained outputs (e.g., a combination of a walker and a dog), while more precise prompts yield accurate, object-specific layers, highlighting multi-granularity capabilities of RefLayer and its strong controllability and generalization.

Figure 2: Overview of the data engine. The pipeline decomposes a natural image into multiple prompt-aligned RGBA layers through six automatic stages: pre-filtering, scene understanding, layer completion, post-completion, prompt generation, and post-filtering.

Panel of Figure 3 — Figure 3: Analysis of RefLade dataset. (a) Instance distribution: Most training images contain 1–3 instances, covering a wide range of object categories. (b) Size comparison: RefLade includes significantly more small instances (by area ratio) than MuLAn.

Dataset	Task	# Images	Average Resolutions	# Cls	# Instances	Occlusion Rate	Image Source
SAIL-VOS ( Hu et al., 2019 )	Amodal	111,654	800 $\times$ 1280	162	1,896,296	56.3%	Synthetic
OVD ( Yan et al., 2019 )	Amodal	34,100	500 $\times$ 375	196	-	-	Real
WALT ( Reddy et al., 2022 )	Amodal	15M	-	2	36M	-	Real
AHP ( Zhou et al., 2021 )	Amodal	56,599	-	1	56,599	-	Real
DYCE ( Ehsani et al., 2018 )	Amodal	5,500	1000 $\times$ 1000	79	85,975	27.7%	Real
OMLD ( Dhamo et al., 2019 )	Amodal	13,000	384 $\times$ 512	40	-	-	Synthetic

Table 1: Comparison of RefLade with related existing datasets.

Panel of Figure 4 — Figure 4: Comparison of model evaluation metrics. (a) The HPA score shows strong alignment with Human ELO rankings. (b) In contrast, none of the individual metrics ( $S_{\text{vis}}$ , $S_{\text{fid}}$ , $S_{\text{gen}}$ ) consistently align with human preferences across models. Model A-I are anonymous for ELO.

Figure 5: RefLayer Model architecture. The model supports prompt-conditioned layer generation using spatial (box, point, mask) and/or textual inputs.

	HPA	$S_{vis}$	$S_{gen}$	$S_{fid}$	$S_{vis}$ + $S_{gen}$	$S_{fid}$ + $S_{gen}$	$S_{vis}$ + $S_{fid}$
Pearson correlation	0.96	0.90	0.96	0.94	0.95	0.95	0.94
Spearman correlation	1	0.60	0.98	0.67	0.92	0.97	1

Table 2: Pearson and Spearman correlations with human ELO across different metrics

Dataset	#layers	Foreground				Background
Dataset	#layers	HPA $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	DIR $\uparrow$	HPA $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	DIR $\uparrow$
MuLAn	50K	0.3852	22.68	0.1403	0.2031	0.3459	21.84	0.1588	0.6385
RefLade	50K	0.4629	10.98	0.1411	0.2543	0.5932	16.87	0.0520	0.7206
RefLade	100K	0.4621	11.27	0.1428	0.2589	0.5935	16.73	0.0530	0.7213
RefLade	200K	0.4631	10.99	0.1434	0.2547	0.5461	19.84	0.0552	0.6950
RefLade	400K	0.4678	10.66	0.1404	0.2575	0.5792	18.36	0.0493	0.7129

Table 3: Benchmarking RefLade with Different Training Set and Scale . The results are reported on the RefLade testing set, where prompts are given in a multimodal text+box format. RefLadeQ: high-quality tuning set. RefLade+Q: two-stage training of the pretraining plus quality fine-tuning. DIR is the image directional similarity specified in Sec. 3.3 .

Table 9

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Prompt engineering with gemini-2.0 for judging completeness.

Figure 7: Data Engine Inpainting Error Example. (a) The person is completed with extra length of leg. (b) The person’s identity is not preserved after processing.

Methods	HPA ${}_{\text{frgd}}$ $\uparrow$	Zero-Shot	mIoU full	mIoU occ
PCNet ( Zhan et al., 2020 )	-	✗	76.91	20.34
VRSP ( Xiao et al., 2021 )	-	✗	78.98	22.92
AISformer ( Tran et al., 2022 )	-	✗	72.69	13.75
C2F-Seg ( Gao et al., 2023 )	-	✗	80.28	27.71
Pix2Gestalt ∗ ( Ozguroglu et al., 2024 )	0.3397	✓	70.02	26.79
RefLayer- $\alpha$	0.4833	✓	73.28	32.31

Appendix

Table 6: Comparison with amodal segmentation methods on COCOA. RefLayer-

\alpha

\beta

, and

\gamma

denote our model trained on 100K high-quality set, 1M training set, and both sets, respectively, using text+mask prompts. The ∗ denotes that we reproduce the results based on a single inference attempt using the official checkpoint provided by the authors.

Method	HPA occ $\uparrow$	Foreground
Method	HPA occ $\uparrow$	HPA $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	DIR $\uparrow$
Pix2Gestalt ( Ozguroglu et al., 2024 )	0.2687	0.3397	18.23	0.1671	0.1198
MuLAn ( Tudosiu et al., 2024 )	0.3041	0.3852	22.68	0.1403	0.2031
RefLayer- $\alpha$	0.4403	0.4833	10.43	0.1298	0.2586

Appendix

Table 7: Comparison with amodal completion methods on RefLade test subset. RefLayer-

\alpha

denotes the same model as in Tab. 6 .

Figure 8: The challenge of object localization when the model is conditioned solely on a textual prompt . Text Prompt (e.g., "two red barns") is easier to cause a poor or incorrect layer extraction due to the model’s inability to precisely locate the target object from semantic information alone. Conversely, providing a simple Spatial Prompt (e.g., a box) instantly resolves the ambiguity, guiding the model to produce a high-fidelity RGBA layer that accurately isolates the target object(s).

Figure 9: RefLayer model qualitative result. For each testing image (top-left corner of each block), we conduct the RefLayer model eight times. The generated result shows a diversity in completion, while preserving the visible content robustly.

Figure 10: Limitations of existing general-purpose LMMs on the Referring Layer Decomposition (RLD) task . We observe that Google Gemini 3 (Nano Banana Pro) struggles to preserve object identity, often regenerating the appearance entirely, and fails to produce valid RGBA outputs.

Figure 11: Comprehensive Visual Diversity and Scene Category Distribution of the RefLade Dataset. The figure illustrates the dataset’s dual composition of 95% Real Images and 5% Stylized Images across nine distinct visual categories. The central pie chart quantifies the distribution of dominant scene types, showing strong coverage of Indoor (25.5%) and City (24.6%) scenes to ensure robust generalization across various photographic contexts.

Figure 12: Comparison between RefLayer trained on the MuLAn dataset and the same model trained on our RefLade dataset. We observe that the results on the MuLAn dataset are generally worse than those obtained when training on our RefLade dataset, showing lower accuracy in the RGB content and alpha channel along object boundaries.

Figure 13: More qualitative results of our RefLayer model with respect to diverse prompts. Each row shows two different prompts and the corresponding results for the same input image.

Compositional Representation Learning

Compositional representation learning studies how models encode reusable parts and relations to understand combinations they have not seen.

See all 79 papers

Editing Images into Reusable Layers via Compositional Edit-Prior Distillation
Oct 17, 2026Wen Xue, Yong Liu, Xueheng Li +5Image Editing Text-Guided Image Editing
South China University of Technology · Tsinghua University · University of Science and Technology of China +1
Stable-Layers: Fine-Tuning Image Layer Decomposition Models with VLM-Scored Reinforcement Learning
May 28, 2026Ciara Rowles, Reshinth Adithyan, Nikhil Pinnaparaju +2Reinforcement Learning
Stability AI
LimeCross: Context-Conditioned Layered Image Editing with Structural Consistency
May 11, 2026Ryugo Morita, Stanislav Frolov, Brian Bernhard Moser +4Text-Guided Image Editing Image Editing
RPTU Kaiserslautern-Landau & DFKI GmbH, Kaiserslautern, Germany · Faculty of Science and Engineering, Hosei University, Tokyo, Japan · EQUES, Tokyo, Japan

Oct 17, 2026J/K move · Enter open · S save

Editing Images into Reusable Layers via Compositional Edit-Prior Distillation

Wen Xue, Yong Liu, Xueheng Li, Si Wu+4

South China University of Technology · Tsinghua University · University of Science and Technology of China+1

Image Editing Text-Guided Image Editing

Open paper AskPDF arXiv

High-quality image editing requires precise modification of target regions while preserving the content of unrelated regions. Existing editing models produce flattened outputs that limit precise local manipulation and layer reuse, while decomposition methods typically optimize layer recovery separately from instruction-driven editing. To this end, we introduce LayerEditor, a unified framework that couples instruction-guided image editing with controllable generation of reusable RGBA layers. At its core, In-Context RGB-Alpha Token Modeling jointly represents the edited canvas and an ordered sequence of RGB-Alpha layers within a single token sequence, enabling semantic edits and explicit control over individual layer content. To retain the pretrained model's editing capability during adaptation to layered outputs, we propose Compositional Edit-Prior Distillation, which transfers its editing prior to both the edited canvas and the image composed from the predicted layers. To support training and evaluation, we construct LayerEditor-100K, a large-scale paired image-editing dataset with ordered RGBA layer annotations, and introduce dedicated benchmarks for layered editing. Extensive experiments demonstrate that LayerEditor maintains strong editing fidelity while generating accurate RGBA layers, supporting both flexible layer-level manipulation and multi-layer decomposition.

Abstract

Compositional Representation Learning

Explore similar work

Editing Images into Reusable Layers via Compositional Edit-Prior Distillation

Stable-Layers: Fine-Tuning Image Layer Decomposition Models with VLM-Scored Reinforcement Learning

LimeCross: Context-Conditioned Layered Image Editing with Structural Consistency