SemEllipsoid: Semantic-aware Shape Abstraction with Ellipsoid

Kuankuan Cheng; Haocheng Yuan; Changjian Li

cs.GROct 30, 2026

SemEllipsoid: Semantic-aware Shape Abstraction with Ellipsoid

Authors: Kuankuan Cheng, Haocheng Yuan, Changjian Li

Organizations: University of Edinburgh, United Kingdom

Abstract

Representing 3D shapes using a set of primitives - known as shape abstraction - is an actively studied problem in computer graphics with wide applications in shape understanding, analysis, and simulation due to the compactness and computational efficiency of the primitive-based representation. Prior work has primarily focused on geometry-aware shape abstraction, where the objective is to approximate the input shape with a minimal set of primitives while preserving geometric fidelity. However, these methods often overlook semantic consistency, inevitably leading to primitives that span across multiple semantic parts. Such inconsistency can cause undesirable artifacts in downstream tasks such as part-level shape editing or manipulation. In this paper, we introduce a new task - semantic-aware shape abstraction, where the goal is to produce abstractions that are both geometrically faithful and semantically consistent, ensuring that each primitive adheres strictly to a single semantic part without crossing semantic boundaries. We propose SemEllipsoid, a novel method to address this problem using ellipsoidal primitives. The core idea is to inject semantic constraints into the primitive optimization process leveraging the availability of semantic cues from large-scale foundation models. We design a fully differentiable projection-blending-evaluation scheme to measure semantic divergence against input semantic masks, and formulate an alternating optimization strategy to iteratively solve for optimal semantic assignment and primitive configurations. Our method is non-learning-based, requiring no data collection or network training, and generalizes well to shapes from arbitrary categories. It also supports different levels of abstraction according to user demand. Extensive experiments and evaluations validate the effectiveness of our approach across several 3D shape collections.

Figures & tables

Figure 1. Semantic-aware Shape Abstraction. Given a 3D shape from an arbitrary category, SemEllipsoid produces intuitive shape abstractions with a set of ellipsoids that are both geometrically faithful and semantically consistent. The gray mesh represents the input, while the adjacent models show different levels of abstraction. Each primitive adheres strictly to a single semantic part, avoiding any cross-semantic assignments.

Figure 2. Semantic violation hinders downstream applications. (a) Existing shape abstraction methods ( e.g. , Marching-Primitives ( Liu et al., 2023a ) ) consider shape geometry solely in their abstraction calculation, neglecting the importance of part semantics. See the abstracted box spans the leg and back of the chair. (b) The intent to widen the leg will inevitably impact the back part, while (c) our method considers both the geometry and semantics, achieving a semantic-aware multi-level shape abstraction to facilitate downstream applications ( e.g. , skeleton detection, shape retrieval).

Figure 3. Method Overview. Given an input mesh and multi-view semantic masks, we first estimate initial primitives using Gaussian Mixture Model (GMM) clustering. The label assignment and primitive optimization processes are then performed in an alternating manner to fit the primitives to the input mesh while adhering to semantic masks. An optional merging step can be applied to simplify the resulting abstraction.

Figure 4

Figure 5. Visual Comparison. We compare abstraction results across different methods, where colors indicate semantic labels. For semantically inconsistent primitives that are difficult to recognize, we highlight the conflict region with a transparent effect.

Method	Ours	APG	OP	SA-PF	SA-CH	SA-EMS	Ours	APG	OP	SA-PF	SA-CH	SA-EMS	Ours	APG	OP	SA-PF	SA-CH	SA-EMS	Ours	APG	OP	SA-PF	SA-CH	SA-EMS
Category	Human						Table						Chair						Bottle
$IoU$ ( $\uparrow$ )	0.922	0.496	0.446	0.852	0.821	0.729	0.679	0.517	0.406	0.549	0.428	0.591	0.783	0.767	0.467	0.701	0.618	0.619	0.928	0.696	0.658	0.903	0.801	0.858
$CD\text{-}L1$ ( $\downarrow$ )	0.016	0.035	0.032	0.018	0.014	0.015	0.025	0.018	0.022	0.017	0.025	0.015	0.015	0.046	0.049	0.014	0.021	0.019	0.036	0.012	0.012	0.040	0.031	0.019
$N$ ( $\downarrow$ )	67.6	-	-	292.9	85.2	47.3	38.4	-	-	49.6	47.5	21.7	70.9	-	-	130.6	87.9	36.1	54.9	-	-	197.0	48.6	21.1
SV p -95%( $\downarrow$ )	0.006	0.437	0.573	0.048	0.015	0.054	0.030	0.261	0.402	0.083	0.082	0.050	0.026	0.468	0.645	0.060	0.033	0.176	0.002	0.181	0.330	0.031	0.037	0.039
SV p -90%( $\downarrow$ )	0.004	0.391	0.495	0.038	0.008	0.029	0.024	0.239	0.359	0.069	0.058	0.038	0.016	0.357	0.560	0.047	0.023	0.125	0.001	0.181	0.268	0.026	0.024	0.028

Table 1. Quantitative comparison with semantic-aware methods . We compare with part-based generation and segmentation-abstraction methods in terms of geometric fidelity and semantic consistency at three different consistency levels. The best performance is bold.

Figure 6. Failure cases of SA-CH. Top : Failure in converting input mesh to watertight. (a) Input irregular mesh. (b) Incorrect closing holes result. (c) Watertight mesh. (d) Abstraction result. Bottom : Semantic violation introduced by MPS. (e) Input part meshes. (f) Abstraction (blue) exceeds the original boundary of input (red). (g) Semantic violation in abstraction. (h) Abstraction by our method.

Semantic Loss Term $\boldsymbol{\lambda}$
	w/o	10.0	200.0	Sched .	w/o	10.0	200.0	Sched .	w/o	10.0	200.0	Sched .	w/o	10.0	200.0	Sched .
Category	Human				Table				Chair				Bottle
$IoU$ ( $\uparrow$ )	0.854	0.879	0.908	0.922	0.700	0.721	0.636	0.679	0.767	0.790	0.750	0.783	0.958	0.965	0.879	0.928
$CD\text{-}L1$ ( $\downarrow$ )	0.015	0.015	0.014	0.016	0.012	0.012	0.027	0.024	0.014	0.014	0.014	0.015	0.024	0.029	0.041	0.036
SV p -95%( $\downarrow$ )	0.320	0.069	0.006	0.006	0.160	0.061	0.085	0.030	0.204	0.056	0.073	0.026	0.166	0.017	0.005	0.002
SV s -95%( $\downarrow$ )	1.000	1.000	0.160	0.300	0.750	0.492	0.308	0.125	0.958	0.650	0.275	0.233	0.696	0.464	0.071	0.089

Table 2. Statistical ablation and robustness study. We report the performance in terms of geometry fidelity and semantic consistency at the 95% consistency level on our test set, when ablating the semantic loss term, the alpha blending rendering scheme, and the multi-view masks.

Perturbation Type
	D	E	M	None	D	E	M	None	D	E	M	None	D	E	M	None
Category	Human				Table				Chair				Bottle
$IoU$ ( $\uparrow$ )	0.921	0.921	0.922	0.922	0.712	0.695	0.714	0.700	0.783	0.780	0.787	0.783	0.928	0.919	0.925	0.926
$CD\text{-}L1$ ( $\downarrow$ )	0.016	0.017	0.016	0.016	0.021	0.035	0.021	0.024	0.015	0.017	0.015	0.015	0.036	0.038	0.035	0.036
SV p -95%( $\downarrow$ )	0.008	0.008	0.009	0.009	0.025	0.032	0.040	0.030	0.019	0.024	0.026	0.026	0.002	0.003	0.002	0.002
SV s -95%( $\downarrow$ )	0.421	0.421	0.474	0.421	0.142	0.125	0.142	0.133	0.208	0.242	0.208	0.200	0.107	0.089	0.089	0.089

Table 3. Robustness study. We report the performance in terms of geometry fidelity and semantic consistency on our test set, when applying three kinds of perturbation to the ground-truth masks: Dilation (D), Erosion (E), Merging (M).

Figure 7. Visual results of ablation study. Top: semantic loss term controlled by the balancing weight $\lambda$ . Bottom: alpha blending scheme controlled by the temperature coefficient $\beta$ .

Figure 8. Effects of rendered views. A sofa is abstracted with our method, given different numbers of rendered views. Looking up at the bottom of the sofa, for 3-views and 6-views settings, primitives that span across different semantic parts are observed due to inadequate semantic guidance.

Figure 9. Semantic-aware abstraction with extra primitives. While our implementation utilizes ellipsoids with closed-form distance functions under projection, the semantically constrained primitive abstraction pipeline extends naturally to general support-function primitives, e.g. superellipsoids and zonotopes. Such primitives are more efficient when modeling shapes with angular features.

Method	Ours	MPS	AVS	SN	DAE	NP	PP	PC	Ours	MPS	AVS	SN	DAE	NP	PP	PC	Ours	MPS	AVS	SN	DAE	NP	PP	PC	Ours	MPS	AVS	SN	DAE	NP	PP	PC
Category	Human								Table								Chair								Bottle
$IoU$ ( $\uparrow$ )	0.922	0.932	0.580	0.612	0.630	0.780	0.464	0.360	0.679	0.836	0.471	0.330	0.407	0.367	0.389	0.198	0.783	0.856	0.461	0.446	0.399	0.379	0.337	0.245	0.928	0.950	0.750	0.848	0.790	0.846	0.694	0.614
$CD\text{-}L1$ ( $\downarrow$ )	0.016	0.013	0.023	0.026	0.020	0.016	0.035	0.034	0.025	0.014	0.022	0.099	0.036	0.030	0.046	0.071	0.015	0.016	0.026	0.079	0.043	0.030	0.035	0.070	0.036	0.036	0.035	0.029	0.032	0.028	0.013	0.018
$N$ ( $\downarrow$ )	67.6	40.9	11.8	7.2	-	-	-	-	38.4	39.3	12.0	6.2	-	-	-	-	70.9	62.8	13.5	6.1	-	-	-	-	54.9	30.7	5.4	3.7	-	-	-	-
SV p -90%( $\downarrow$ )	0.004	0.184	0.156	0.527	0.398	0.921	0.416	0.518	0.024	0.115	0.121	0.330	0.057	0.248	0.285	0.477	0.016	0.121	0.180	0.451	0.214	0.619	0.451	0.654	0.001	0.102	0.217	0.215	0.330	0.189	0.304	0.464
SV s -90%( $\downarrow$ )	0.211	1.000	1.000	1.000	1.000	1.000	1.000	1.000	0.108	0.552	0.608	0.800	0.317	0.883	0.773	0.636	0.167	0.815	0.833	0.900	0.636	0.992	1.000	0.955	0.036	0.732	0.714	0.661	0.661	0.643	0.522	0.870

Table 4. Quantitative comparison with non-semantic-aware methods . We compare with shape abstraction, co-segmentation and part-based generation methods in terms of geometric fidelity and semantic consistency at the 90% consistency level. The best performance is bold.

Figure 10. Failure cases. (a) When the input sword has many thin structures, the abstraction using the default primitive number is inferior. Increasing this number can mitigate this issue to some extent. (b) When the airplane’s semantic mask contains incorrect engine labels, our abstraction consequently suffers from the semantic conflict, leading to undesired geometry ( e.g. , wings).

Figure 11. Applications. (a) Part-wise shape editing using linear and non-linear deformation. (b) Shape part mixing. (c) Shape skeleton extraction.

Prompting Image Generators for Training-free Primitive Shape Abstraction
Jul 6, 2026Gregor Kobsik, Tim Elsner, Leif Kobbelt3D Part Segmentation
GLASS: Graph and Vision-Language Assisted Semantic Shape Correspondence
Mar 8, 2026Qinfeng Xiao, Guofeng Mei, Qilong Liu +53D Shape Matching Semantic Correspondence
Hong Kong Polytechnic University, HK SAR · Fondazione Bruno Kessler, Italy · Laboratory for Artificial Intelligence in Design, HK SAR +1
Universal 3D Shape Matching via Coarse-to-Fine Language Guidance
Feb 22, 2026Qinfeng Xiao, Guofeng Mei, Bo Yang +33D Shape Matching Semantic Correspondence
Hong Kong Polytechnic University, HK SAR · Fondazione Bruno Kessler, Italy · University of Technology Sydney, Australia

Jul 6, 2026J/K move · Enter open · S save

Prompting Image Generators for Training-free Primitive Shape Abstraction

Gregor Kobsik, Tim Elsner, Leif Kobbelt

3D Part Segmentation

Open paper AskPDF arXiv

Compact primitive abstractions represent 3D shapes with a few geometric primitives while preserving recognizable components. Learned methods depend on their training classes, and optimization-based methods split shapes geometrically rather than into parts. We instead reuse the visual part knowledge of pretrained models without task-specific training or fine-tuning. A vision-language model names parts in multi-view renders, and an unmodified image generator paints color-coded part masks. Reprojection and spatial clustering recover 3D instances, and a classical optimizer fits one tapered and bent superquadric per part. With five to eight primitives per object, the abstractions match the Chamfer distance of the strongest learned baseline on HumanPrim, improve on it by 10% on Toys4K, and have the lowest overlap among compact methods, while chair legs, backrest bars and wheels remain separate primitives. Our accuracy also transfers better than theirs to objects outside the learned methods' ShapeNet training classes. Replacing the generated masks with part labels from the 3D segmentation methods P3-SAM or PartField lowers IoU by 7 to 17 points under the same fitter. Further studies relate the remaining volumetric error to part granularity and to parts that the rendered views observe from one side only.

Abstract

Explore similar work

Prompting Image Generators for Training-free Primitive Shape Abstraction

GLASS: Graph and Vision-Language Assisted Semantic Shape Correspondence

Universal 3D Shape Matching via Coarse-to-Fine Language Guidance