SemEllipsoid: Semantic-aware Shape Abstraction with Ellipsoid

Kuankuan Cheng; Haocheng Yuan; Changjian Li

cs.GROct 30, 2026

SemEllipsoid: Semantic-aware Shape Abstraction with Ellipsoid

Authors: Kuankuan Cheng, Haocheng Yuan, Changjian Li

Organizations: University of Edinburgh, United Kingdom

Abstract

Representing 3D shapes using a set of primitives - known as shape abstraction - is an actively studied problem in computer graphics with wide applications in shape understanding, analysis, and simulation due to the compactness and computational efficiency of the primitive-based representation. Prior work has primarily focused on geometry-aware shape abstraction, where the objective is to approximate the input shape with a minimal set of primitives while preserving geometric fidelity. However, these methods often overlook semantic consistency, inevitably leading to primitives that span across multiple semantic parts. Such inconsistency can cause undesirable artifacts in downstream tasks such as part-level shape editing or manipulation. In this paper, we introduce a new task - semantic-aware shape abstraction, where the goal is to produce abstractions that are both geometrically faithful and semantically consistent, ensuring that each primitive adheres strictly to a single semantic part without crossing semantic boundaries. We propose SemEllipsoid, a novel method to address this problem using ellipsoidal primitives. The core idea is to inject semantic constraints into the primitive optimization process leveraging the availability of semantic cues from large-scale foundation models. We design a fully differentiable projection-blending-evaluation scheme to measure semantic divergence against input semantic masks, and formulate an alternating optimization strategy to iteratively solve for optimal semantic assignment and primitive configurations. Our method is non-learning-based, requiring no data collection or network training, and generalizes well to shapes from arbitrary categories. It also supports different levels of abstraction according to user demand. Extensive experiments and evaluations validate the effectiveness of our approach across several 3D shape collections.

Figures & tables

Figure 1. Semantic-aware Shape Abstraction. Given a 3D shape from an arbitrary category, SemEllipsoid produces intuitive shape abstractions with a set of ellipsoids that are both geometrically faithful and semantically consistent. The gray mesh represents the input, while the adjacent models show different levels of abstraction. Each primitive adheres strictly to a single semantic part, avoiding any cross-semantic assignments.

Figure 2. Semantic violation hinders downstream applications. (a) Existing shape abstraction methods ( e.g. , Marching-Primitives ( Liu et al., 2023a ) ) consider shape geometry solely in their abstraction calculation, neglecting the importance of part semantics. See the abstracted box spans the leg and back of the chair. (b) The intent to widen the leg will inevitably impact the back part, while (c) our method considers both the geometry and semantics, achieving a semantic-aware multi-level shape abstraction to facilitate downstream applications ( e.g. , skeleton detection, shape retrieval).

Figure 3. Method Overview. Given an input mesh and multi-view semantic masks, we first estimate initial primitives using Gaussian Mixture Model (GMM) clustering. The label assignment and primitive optimization processes are then performed in an alternating manner to fit the primitives to the input mesh while adhering to semantic masks. An optional merging step can be applied to simplify the resulting abstraction.

Figure 4

Figure 5. Visual Comparison. We compare abstraction results across different methods, where colors indicate semantic labels. For semantically inconsistent primitives that are difficult to recognize, we highlight the conflict region with a transparent effect.

Method	Ours	APG	OP	SA-PF	SA-CH	SA-EMS	Ours	APG	OP	SA-PF	SA-CH	SA-EMS	Ours	APG	OP	SA-PF	SA-CH	SA-EMS	Ours	APG	OP	SA-PF	SA-CH	SA-EMS
Category	Human						Table						Chair						Bottle
$IoU$ ( $\uparrow$ )	0.922	0.496	0.446	0.852	0.821	0.729	0.679	0.517	0.406	0.549	0.428	0.591	0.783	0.767	0.467	0.701	0.618	0.619	0.928	0.696	0.658	0.903	0.801	0.858
$CD\text{-}L1$ ( $\downarrow$ )	0.016	0.035	0.032	0.018	0.014	0.015	0.025	0.018	0.022	0.017	0.025	0.015	0.015	0.046	0.049	0.014	0.021	0.019	0.036	0.012	0.012	0.040	0.031	0.019
$N$ ( $\downarrow$ )	67.6	-	-	292.9	85.2	47.3	38.4	-	-	49.6	47.5	21.7	70.9	-	-	130.6	87.9	36.1	54.9	-	-	197.0	48.6	21.1
SV p -95%( $\downarrow$ )	0.006	0.437	0.573	0.048	0.015	0.054	0.030	0.261	0.402	0.083	0.082	0.050	0.026	0.468	0.645	0.060	0.033	0.176	0.002	0.181	0.330	0.031	0.037	0.039
SV p -90%( $\downarrow$ )	0.004	0.391	0.495	0.038	0.008	0.029	0.024	0.239	0.359	0.069	0.058	0.038	0.016	0.357	0.560	0.047	0.023	0.125	0.001	0.181	0.268	0.026	0.024	0.028

Table 1. Quantitative comparison with semantic-aware methods . We compare with part-based generation and segmentation-abstraction methods in terms of geometric fidelity and semantic consistency at three different consistency levels. The best performance is bold.

Figure 6. Failure cases of SA-CH. Top : Failure in converting input mesh to watertight. (a) Input irregular mesh. (b) Incorrect closing holes result. (c) Watertight mesh. (d) Abstraction result. Bottom : Semantic violation introduced by MPS. (e) Input part meshes. (f) Abstraction (blue) exceeds the original boundary of input (red). (g) Semantic violation in abstraction. (h) Abstraction by our method.

Semantic Loss Term $\boldsymbol{\lambda}$
	w/o	10.0	200.0	Sched .	w/o	10.0	200.0	Sched .	w/o	10.0	200.0	Sched .	w/o	10.0	200.0	Sched .
Category	Human				Table				Chair				Bottle
$IoU$ ( $\uparrow$ )	0.854	0.879	0.908	0.922	0.700	0.721	0.636	0.679	0.767	0.790	0.750	0.783	0.958	0.965	0.879	0.928
$CD\text{-}L1$ ( $\downarrow$ )	0.015	0.015	0.014	0.016	0.012	0.012	0.027	0.024	0.014	0.014	0.014	0.015	0.024	0.029	0.041	0.036
SV p -95%( $\downarrow$ )	0.320	0.069	0.006	0.006	0.160	0.061	0.085	0.030	0.204	0.056	0.073	0.026	0.166	0.017	0.005	0.002
SV s -95%( $\downarrow$ )	1.000	1.000	0.160	0.300	0.750	0.492	0.308	0.125	0.958	0.650	0.275	0.233	0.696	0.464	0.071	0.089

Table 2. Statistical ablation and robustness study. We report the performance in terms of geometry fidelity and semantic consistency at the 95% consistency level on our test set, when ablating the semantic loss term, the alpha blending rendering scheme, and the multi-view masks.

Perturbation Type
	D	E	M	None	D	E	M	None	D	E	M	None	D	E	M	None
Category	Human				Table				Chair				Bottle
$IoU$ ( $\uparrow$ )	0.921	0.921	0.922	0.922	0.712	0.695	0.714	0.700	0.783	0.780	0.787	0.783	0.928	0.919	0.925	0.926
$CD\text{-}L1$ ( $\downarrow$ )	0.016	0.017	0.016	0.016	0.021	0.035	0.021	0.024	0.015	0.017	0.015	0.015	0.036	0.038	0.035	0.036
SV p -95%( $\downarrow$ )	0.008	0.008	0.009	0.009	0.025	0.032	0.040	0.030	0.019	0.024	0.026	0.026	0.002	0.003	0.002	0.002
SV s -95%( $\downarrow$ )	0.421	0.421	0.474	0.421	0.142	0.125	0.142	0.133	0.208	0.242	0.208	0.200	0.107	0.089	0.089	0.089

Table 3. Robustness study. We report the performance in terms of geometry fidelity and semantic consistency on our test set, when applying three kinds of perturbation to the ground-truth masks: Dilation (D), Erosion (E), Merging (M).

Figure 7. Visual results of ablation study. Top: semantic loss term controlled by the balancing weight $\lambda$ . Bottom: alpha blending scheme controlled by the temperature coefficient $\beta$ .

Figure 8. Effects of rendered views. A sofa is abstracted with our method, given different numbers of rendered views. Looking up at the bottom of the sofa, for 3-views and 6-views settings, primitives that span across different semantic parts are observed due to inadequate semantic guidance.

Figure 9. Semantic-aware abstraction with extra primitives. While our implementation utilizes ellipsoids with closed-form distance functions under projection, the semantically constrained primitive abstraction pipeline extends naturally to general support-function primitives, e.g. superellipsoids and zonotopes. Such primitives are more efficient when modeling shapes with angular features.

Method	Ours	MPS	AVS	SN	DAE	NP	PP	PC	Ours	MPS	AVS	SN	DAE	NP	PP	PC	Ours	MPS	AVS	SN	DAE	NP	PP	PC	Ours	MPS	AVS	SN	DAE	NP	PP	PC
Category	Human								Table								Chair								Bottle
$IoU$ ( $\uparrow$ )	0.922	0.932	0.580	0.612	0.630	0.780	0.464	0.360	0.679	0.836	0.471	0.330	0.407	0.367	0.389	0.198	0.783	0.856	0.461	0.446	0.399	0.379	0.337	0.245	0.928	0.950	0.750	0.848	0.790	0.846	0.694	0.614
$CD\text{-}L1$ ( $\downarrow$ )	0.016	0.013	0.023	0.026	0.020	0.016	0.035	0.034	0.025	0.014	0.022	0.099	0.036	0.030	0.046	0.071	0.015	0.016	0.026	0.079	0.043	0.030	0.035	0.070	0.036	0.036	0.035	0.029	0.032	0.028	0.013	0.018
$N$ ( $\downarrow$ )	67.6	40.9	11.8	7.2	-	-	-	-	38.4	39.3	12.0	6.2	-	-	-	-	70.9	62.8	13.5	6.1	-	-	-	-	54.9	30.7	5.4	3.7	-	-	-	-
SV p -90%( $\downarrow$ )	0.004	0.184	0.156	0.527	0.398	0.921	0.416	0.518	0.024	0.115	0.121	0.330	0.057	0.248	0.285	0.477	0.016	0.121	0.180	0.451	0.214	0.619	0.451	0.654	0.001	0.102	0.217	0.215	0.330	0.189	0.304	0.464
SV s -90%( $\downarrow$ )	0.211	1.000	1.000	1.000	1.000	1.000	1.000	1.000	0.108	0.552	0.608	0.800	0.317	0.883	0.773	0.636	0.167	0.815	0.833	0.900	0.636	0.992	1.000	0.955	0.036	0.732	0.714	0.661	0.661	0.643	0.522	0.870

Table 4. Quantitative comparison with non-semantic-aware methods . We compare with shape abstraction, co-segmentation and part-based generation methods in terms of geometric fidelity and semantic consistency at the 90% consistency level. The best performance is bold.

Figure 10. Failure cases. (a) When the input sword has many thin structures, the abstraction using the default primitive number is inferior. Increasing this number can mitigate this issue to some extent. (b) When the airplane’s semantic mask contains incorrect engine labels, our abstraction consequently suffers from the semantic conflict, leading to undesired geometry ( e.g. , wings).

Figure 11. Applications. (a) Part-wise shape editing using linear and non-linear deformation. (b) Shape part mixing. (c) Shape skeleton extraction.

Jul 6, 2026

Prompting Image Generators for Training-free Primitive Shape Abstraction

Compact primitive abstractions represent 3D shapes with a few geometric primitives while preserving recognizable components. Learned methods depend on their training classes, and optimization-based methods split shapes geometrically rather than into parts. We instead reuse the visual part knowledge of pretrained models without task-specific training or fine-tuning. A vision-language model names parts in multi-view renders, and an unmodified image generator paints color-coded part masks. Reprojection and spatial clustering recover 3D instances, and a classical optimizer fits one tapered and bent superquadric per part. With five to eight primitives per object, the abstractions match the Chamfer distance of the strongest learned baseline on HumanPrim, improve on it by 10% on Toys4K, and have the lowest overlap among compact methods, while chair legs, backrest bars and wheels remain separate primitives. Our accuracy also transfers better than theirs to objects outside the learned methods' ShapeNet training classes. Replacing the generated masks with part labels from the 3D segmentation methods P3-SAM or PartField lowers IoU by 7 to 17 points under the same fitter. Further studies relate the remaining volumetric error to part granularity and to parts that the rendered views observe from one side only.

Gregor Kobsik, Tim Elsner, Leif Kobbelt

3D Part Segmentation

Ask PDF

Mar 8, 2026

GLASS: Graph and Vision-Language Assisted Semantic Shape Correspondence

Establishing dense correspondence across 3D shapes is crucial for fundamental downstream tasks, including texture transfer, shape interpolation, and robotic manipulation. However, learning these mappings without manual supervision remains a formidable challenge, particularly under severe non-isometric deformations and in inter-class settings where geometric cues are ambiguous. Conventional functional map methods, while elegant, typically struggle in these regimes due to their reliance on isometry. To address this, we present GLASS, a framework that bridges the gap by integrating geometric spectral analysis with rich semantic priors from vision-language foundation models. GLASS introduces three key innovations: (i) a view-consistent strategy that enables robust multi-view visual feature extraction from powerful vision foundation models; (ii) the injection of language embeddings into vertex descriptors via zero-shot 3D segmentation, capturing high-level part semantics; and (iii) a graph-assisted contrastive loss that enforces structural consistency between regions (e.g., source's head''

\leftrightarrow

target's head'') by leveraging geodesic and topological relationships between regions. This design allows GLASS to learn globally coherent and semantically consistent maps without ground-truth supervision. Extensive experiments demonstrate that GLASS achieves state-of-the-art performance across all regimes, maintaining high accuracy on standard near-isometric tasks while significantly advancing performance in challenging settings. Specifically, it achieves average geodesic errors of 0.21, 4.5, and 5.6 on the inter-class benchmark SNIS and non-isometric benchmarks SMAL and TOPKIDS, reducing errors from URSSM baselines of 0.49, 6.0, and 8.9 by 57%, 25%, and 37%, respectively.

Qinfeng Xiao, Guofeng Mei, Qilong Liu +5

Hong Kong Polytechnic University, HK SAR · Fondazione Bruno Kessler, Italy · Laboratory for Artificial Intelligence in Design, HK SAR +1

3D Shape Matching Semantic Correspondence Non-Rigid Shape Correspondence

Ask PDF

Feb 22, 2026

Universal 3D Shape Matching via Coarse-to-Fine Language Guidance

Establishing dense correspondences between shapes is a crucial task in computer vision and graphics, while prior approaches depend on near-isometric assumptions and homogeneous subject types (i.e., only operate for human shapes). However, building semantic correspondences for cross-category objects remains challenging and has received relatively little attention. To achieve this, we propose UniMatch, a semantic-aware, coarse-to-fine framework for constructing dense semantic correspondences between strongly non-isometric shapes without restricting object categories. The key insight is to lift "coarse" semantic cues into "fine" correspondence, which is achieved through two stages. In the "coarse" stage, we perform class-agnostic 3D segmentation to obtain non-overlapping semantic parts and prompt multimodal large language models (MLLMs) to identify part names. Then, we employ pretrained vision language models (VLMs) to extract text embeddings, enabling the construction of matched semantic parts. In the "fine" stage, we leverage these coarse correspondences to guide the learning of dense correspondences through a dedicated rank-based contrastive scheme. Thanks to class-agnostic segmentation, language guiding, and rank-based contrastive learning, our method is versatile for universal object categories and requires no predefined part proposals, enabling universal matching for inter-class and non-isometric shapes. Extensive experiments demonstrate UniMatch consistently outperforms competing methods in various challenging scenarios.

Qinfeng Xiao, Guofeng Mei, Bo Yang +3

Hong Kong Polytechnic University, HK SAR · Fondazione Bruno Kessler, Italy · University of Technology Sydney, Australia

3D Shape Matching Semantic Correspondence Non-Rigid Shape Correspondence

Ask PDF

Abstract

Explore similar work

Prompting Image Generators for Training-free Primitive Shape Abstraction

GLASS: Graph and Vision-Language Assisted Semantic Shape Correspondence

Universal 3D Shape Matching via Coarse-to-Fine Language Guidance