Organizations: School of Software, Beihang University · Igarashi Lab, The University of Tokyo · Bambu Lab · Department of Computer Science, The University of Hong Kong
We introduce AmodalSVG, a new framework for amodal image vectorization that produces semantically organized and geometrically complete SVG representations from natural images. Existing vectorization methods operate under a modal paradigm: tracing only visible pixels and disregarding occlusion. Consequently, the resulting SVGs are semantically entangled and geometrically incomplete, limiting SVG's structural editability. In contrast, AmodalSVG reconstructs full object geometries, including occluded regions, into independent, editable vector layers. To achieve this, AmodalSVG reformulates image vectorization as a two-stage framework, performing semantic decoupling and completion in the raster domain to produce amodally complete semantic layers, which are then independently vectorized. In the first stage, we introduce Semantic Layer Peeling (SLP), a VLM-guided strategy that progressively decomposes an image into semantically coherent layers. By hybrid inpainting, SLP recovers complete object appearances under occlusions, enabling explicit semantic decoupling. To vectorize these layers efficiently, we propose Adaptive Layered Vectorization (ALV), which dynamically modulates the primitive budget via an error-budget-driven adjustment mechanism. Extensive experiments demonstrate that AmodalSVG significantly outperforms prior methods in visual fidelity. Moreover, the resulting amodal layers enable object-level editing directly in the vector domain, capabilities not supported by existing vectorization approaches. Code will be released upon acceptance.
Figures & tables
Figure 1: AmodalSVG vectorizes a raster image into semantically decoupled and amodally complete vector layers ( e.g . , the unoccluded lighthouse and its shadow on the top left, and the full giraffe on the top right). The resulting vector layers enable object-level editing–such as Reposition , Recolor , Replace , Resize , Reorder , and Remove –directly in the vector domain while maintaining global visual consistency (bottom).
Figure 2: Overview of the AmodalSVG. Our framework adopts a two-stage pipeline: raster-level semantic decoupling and completion, followed by layer-wise vectorization. (Left) For semantic decoupling and completion, Semantic Layer Peeling (SLP) iteratively identifies entities via VLMs to guide SAM segmentation. A hybrid inpainting mechanism then reconstitutes occluded regions to produce structurally integral raster layers Lt . (Right) For layer-wise vectorization, Adaptive Layered Vectorization (ALV) vectorizes each layer Lk starting from a content-adaptive initialization. It dynamically manages SVG primitives Sk by pruning redundant primitives via visibility analysis and inserting new primitives based on the error map E(p) .
Metric
Global Optimization
Coarse-to-fine Layering
Path-by-path Layering
Amodal Layering
DiffVG [ 20 ]
O&R [ 14 ]
LIVE [ 24 ]
SGLIVE [ 59 ]
LayerTracer [ 35 ]
LayeredVec [ 43 ]
LayerPeeler [ 46 ]
AdaVec [ 58 ]
Ours
MSE ↓
0.00098
0.00423
0.00110
0.01809
0.00464
0.00099
0.14324
0.00891
0.00072
PSNR ↑
30.52
23.76
30.07
18.00
24.01
30.45
9.72
20.61
32.91
SSIM ↑
0.8860
0.7093
0.8863
0.6230
0.7533
0.8939
0.5591
0.6943
0.9342
LPIPS ↓
0.1075
0.3450
0.1635
0.3727
0.2161
0.1307
0.5211
0.3280
0.0554
Table 1: Quantitative comparison of SVG reconstruction fidelity. The best results are bolded and the second best are underlined .
Figure 3: Qualitative comparison of SVG reconstruction fidelity. We evaluate AmodalSVG against various state-of-the-art vectorization methods. See zoom-in regions (highlighted in boxes) for details. Our method achieves high reconstruction fidelity and preserves sharp edges and fine-grained textures than other methods.
Figure 4: Qualitative comparison of SVG layering quality. Compared with coarse-to-fine methods whose layers primarily serve the optimization process and often entangle multiple semantics and path-by-path approaches that generate fragmented segments, AmodalSVG produces structurally coherent layers. The generative Qwen-Image-Layered [ 55 ] often entangles multiple semantics and produces empty layers in complex scenes, whereas our method accurately decouples semantic entities and reconstructs occluded regions.
Figure 5: Ablation of Semantic Layer Peeling (SLP) strategies. (a) w/o VLM (SAM Auto): Native SAM segmentation relies solely on low-level features, causing severe over-segmentation. (b) w/o Iteration (One-shot): Attempting to identify all objects simultaneously causes attention dispersion, leading to missed objects and imprecise boundary localization (object in red box). (c) SLP (Ours): Our iterative peel-and-inpaint strategy ensures robust entity extraction and structurally complete background recovery, yielding distinct and coherent semantic layers.
Figure 7
Paradigm
Semantic Decoupling
Structural Completion
Recolor
Resize
Reposition
Reorder
Remove
Replace
Global Optimization [ 34 , 40 , 20 , 29 , 14 ]
✗
✗
✗
✗
✗
✗
✗
✗
Path-by-path Layering [ 58 , 46 ]
✗
➚
➚
✗
✗
✗
✗
✗
Coarse-to-Fine Layering [ 24 , 59 , 35 , 43 ]
➚
✗
➚
➚
➚
✗
➚
➚
Amodal Layering (Ours)
✓
✓
✓
✓
✓
✓
✓
✓
Table 3: Comparison of editing capabilities across different vectorization paradigms. ✓ denotes full support, ✗ denotes no support, and ➚ indicates limited or conditional support ( e.g . , only for simple icons or resulting in visual artifacts/holes).
Figure 7: Structural editability and downstream applications. With its structurally complete and semantically decoupled layers, AmodalSVG enables seamless object-level manipulations and scene modifications without background gaps.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1 : System Prompt used for Semantic Layer Peeling. This structured prompt guides the VLM to sequentially reason about occlusion relationships, isolate the frontmost semantic entity, and format its output as an easily parsable JSON tuple.
Figure S2 : Ablation of the hybrid inpainting strategy. We evaluate our synergistic inpainting pipeline against single-model baselines. While Only LaMa maintains structural continuity but lacks high-frequency detail (resulting in blurry textures or streaky artifacts), Only Flux (w/o VLM guidance) often fails to accurately localize and remove foreground entities. Our Hybrid inpainting strategy, guided by VLM semantic parsing, successfully reconstitutes complex backgrounds ( e.g . , bookshelf textures and cup geometry) with high fidelity, ensuring structural integrity for subsequent vectorization.
Prune ratio
Method
Quality
Runtime (s) ↓
MSE ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Area
1.61×10−4
40.24
0.9896
0.0104
0.533
Opacity
2.41×10−4
37.76
0.9856
0.0177
0.244
Oracle
5.43×10−6
∞
0.9997
0.0020
129.049
10%
Ours
4.75×10−5
52.49
0.9946
0.0065
6.468
Area
3.83×10−4
36.57
0.9769
0.0232
0.322
Appendix
Table S1: Quantitative comparison of primitive pruning strategies. We evaluate different pruning strategies under three reduction ratios ( 10% , 20% , and 30% ) and report the average performance over 10 SVGs. For metrics (MSE, PSNR, SSIM, and LPIPS) across different reduction ratios, the theoretical upper bound ( Oracle ) is shown in bold , and the best practical performance is underlined . Runtime is averaged per SVG. Our method significantly outperforms naive Area-based or Opacity-based heuristics, achieving comparable fidelity to the brute-force Oracle bound while offering a massive ∼20× speedup.
Figure S3 : Qualitative comparison of primitive pruning strategies. We evaluate our Occlusion-aware strategy against Area-based and Opacity-based heuristics, with Oracle serving as the theoretical ground truth upper bound. At progressive pruning ratios (10%–30%), the area-based method tends to eliminate fine structural details ( e.g. , sharp edges of the Opera House), while the opacity-based method disrupts complex color blending and gradients ( e.g. , the butterfly’s wing patterns). In contrast, our method identifies redundant primitives within the occlusion hierarchy, maintaining visual fidelity almost identical to the exhaustive Oracle bound.
Figure S4 : Effect of the temperature hyperparameter T on primitive addition. We visualize the final reconstruction results (top), detailed zoom-in regions (middle), and the corresponding primitive sampling distributions (bottom, where red dots denote newly sampled primitives over the residual error map). At low temperature ( T=0.1 ), the distribution sharpens, causing primitives to greedily concentrate on high-frequency error peaks ( e.g . , object contours), leaving smoother gradient regions structurally under-fitted. At high temperature ( T=2.0 ), the distribution flattens into nearly uniform exploration, overlooking localized structural details. This results in blurred textures and collapsed geometry, as evidenced in the car (red box) and architecture (yellow box). Our default intermediate value ( T=0.5 ) achieves an optimal balance, ensuring both crisp structural boundaries and regional color consistency.
Figure S5 : Failure cases of AmodalSVG. (Top) Transmissive objects and secondary effects. Peeling transparent entities ( e.g . , glassware) disrupts natural color blending, resulting in “baked-in” foreground colors and “orphaned” shadows on the background. (Bottom) Cyclic occlusions. Intertwined objects ( e.g . , a parrot gripping a branch) create depth cycles that cannot be natively rendered by the strict global z-ordering of the SVG format.
Scalable Vector Graphics (SVGs) function both as visual images and as structured code that encode rich geometric and layout information, yet most methods rasterize them and discard this symbolic organization. At the same time, recent sentence embedding methods produce strong text representations but do not naturally extend to visual or structured modalities. We propose a training-free, instruction-guided multimodal embedding framework that uses a Multimodal Large Language Model (MLLM) to map text, raster images, and SVG code into an aligned embedding space. We control the direction of embeddings through modality-specific instructions and structural SVG cues, eliminating the need for learned projection heads or contrastive training. Our method has two key components: (1) Multimodal Explicit One-word Limitation (mEOL), which instructs the MLLM to summarize any multimodal input into a single token whose hidden state serves as a compact semantic embedding. (2) A semantic SVG rewriting module that assigns meaningful identifiers and simplifies nested SVG elements through visual reasoning over the rendered image, exposing geometric and relational cues hidden in raw code. Using a repurposed VGBench, we build the first text-to-SVG retrieval benchmark and show that our training-free embeddings outperform encoder-based and training-based multimodal baselines. These results highlight prompt-level control as an effective alternative to parameter-level training for structure-aware multimodal retrieval. Project page: https://scene-the-ella.github.io/meol/
While Vision-Language Models (VLMs) excel at visual reasoning, generating structured, editable Scalable Vector Graphics (SVG) remains a fundamental challenge. Existing pipelines predominantly yield flat, semantically agnostic collections of paths, where editing a single object requires manually identifying its constituent paths. To address this, we propose a VLM-driven agentic framework for semantic compositional SVG generation. Our pipeline recursively parses visual scenes into semantic and geometric hierarchies via top-down decomposition, visual grounding, and prompt-driven amodal occlusion recovery, ensuring each component is geometrically complete. Furthermore, we introduce the Semantic SVG Benchmark with human-annotated semantic groups and novel sub-component metrics (Semantic Recall/Precision, PERE) to explicitly evaluate structural compositionality and functional editability. Experiments show that our natively predicted structures surpass the upper bounds of existing flat-generation methods in both grouping quality and editability, while maintaining state-of-the-art visual fidelity.
Amodal segmentation is a challenging task that aims to predict the complete geometric shape of objects, including their occluded regions. Although existing methods primarily focus on amodal segmentation within the training domain, these approaches often lack the generalization capacity to extend effectively to novel object categories and unseen contexts. This paper introduces Amodal SAM, a unified framework that leverages SAM (Segment Anything Model) for both amodal image and amodal video segmentation. Amodal SAM preserves the powerful generalization ability of SAM while extending its inherent capabilities to the amodal segmentation task. The improvements lie in three aspects: (1) a lightweight Spatial Completion Adapter that enables occluded region reconstruction, (2) a Target-Aware Occlusion Synthesis (TAOS) pipeline that addresses the scarcity of amodal annotations by generating diverse synthetic training data, and (3) novel learning objectives that enforce regional consistency and topological regularization. Extensive experiments demonstrate that Amodal SAM achieves state-of-the-art performance on standard benchmarks, while simultaneously exhibiting robust generalization to novel scenarios. We anticipate that this research will advance the field toward practical amodal segmentation systems capable of operating effectively in unconstrained real-world environments.
Bo Zhang, Zhuotao Tian, Xin Tao +3
Department of Computer Science, Harbin Institute of Technology at Shenzhen, Shenzhen 518057, China · Kuaishou Technology, Shenzhen, Guangdong, China