Organizations: Hong Kong Polytechnic University, HK SAR · Fondazione Bruno Kessler, Italy · Laboratory for Artificial Intelligence in Design, HK SAR · University of Technology Sydney, Australia
Establishing dense correspondence across 3D shapes is crucial for fundamental downstream tasks, including texture transfer, shape interpolation, and robotic manipulation. However, learning these mappings without manual supervision remains a formidable challenge, particularly under severe non-isometric deformations and in inter-class settings where geometric cues are ambiguous. Conventional functional map methods, while elegant, typically struggle in these regimes due to their reliance on isometry. To address this, we present GLASS, a framework that bridges the gap by integrating geometric spectral analysis with rich semantic priors from vision-language foundation models. GLASS introduces three key innovations: (i) a view-consistent strategy that enables robust multi-view visual feature extraction from powerful vision foundation models; (ii) the injection of language embeddings into vertex descriptors via zero-shot 3D segmentation, capturing high-level part semantics; and (iii) a graph-assisted contrastive loss that enforces structural consistency between regions (e.g., source's head'' ↔ target's head'') by leveraging geodesic and topological relationships between regions. This design allows GLASS to learn globally coherent and semantically consistent maps without ground-truth supervision. Extensive experiments demonstrate that GLASS achieves state-of-the-art performance across all regimes, maintaining high accuracy on standard near-isometric tasks while significantly advancing performance in challenging settings. Specifically, it achieves average geodesic errors of 0.21, 4.5, and 5.6 on the inter-class benchmark SNIS and non-isometric benchmarks SMAL and TOPKIDS, reducing errors from URSSM baselines of 0.49, 6.0, and 8.9 by 57%, 25%, and 37%, respectively.
Figures & tables
Figure 1: GLASS achieves robust dense semantic correspondence across diverse 3D matching scenarios by unifying the functional map framework with vision-language priors and semantic graph alignment. Compared with the functional map baseline URSSM [ 9 ] , our method resolves semantic ambiguities in: 1) Inter-class matching , where semantic part alignment ( e.g. , human arm to horse front leg) is required across distinct categories; 2) Non-isometric deformations , involving significant morphological variations across subjects; and 3) Topological noise and near-isometry , where GLASS maintains high precision despite severe topological artifacts.
Figure 2: The overall structure of our GLASS. It consists of three key stages: (1) View-consistent feature lifting , where we synthesize coherent textures and lift SD-DINO features [ 48 ] onto the 3D surface; (2) Language-guided semantic injection , which enriches visual descriptors with linguistic priors using sentence encoder ( e.g. , SigLip [ 44 ] ) embeddings derived from zero-shot region proposals; and (3) Region-aware map optimization , where we employ a functional map guided by a novel semantic graph-aware contrastive loss to ensure structural and semantic consistency.
Figure 3: Comparison of textured meshes and semantic features between Diff3F [ 13 ] and ours. By synthesizing highly realistic and view-coherent textures for 3D meshes, our view-consistent strategy ( Sec. 3.2 ) facilitates the extraction of high-fidelity, view-consistent SD-DINO features, thus boosting semantic matching. In contrast, Diff3F struggles with severe multi-view inconsistencies and visual artifacts during texturing, thereby degrading the quality of its resulting feature fields.
Figure 4: Example of a semantic region graph and region level contrasting. The graph represents high-level topological relationships between semantic regions, where nodes correspond to distinct parts and edges represent semantic relation priors. Guided by this structure, our graph-assisted contrastive loss pulls vertex features within the same region together while pushing distinct regions apart.
Method
Venue
Unsup.
Visual
Lang.
SNIS
▼ Axiomatic Methods
ZoomOut [ 30 ]
TOG 2019
✓
✗
✗
0.51
▼ Functional Map Methods
URSSM [ 9 ]
TOG 2023
✓
✗
✗
0.49
SimpFMap [ 26 ]
CVPR 2024
✓
✗
✗
0.51
▼ Semantic Methods
Table 1: Average geodesic errors ( ↓ ) of GLASS and baselines on the inter-class benchmark. The best and the second-best are shown in blue and orange respectively.
Method
Venue
Unsup.
Visual
Lang.
SMAL
TOPKIDS
▼ Axiomatic Methods
ZoomOut [ 30 ]
TOG 2019
✓
✗
✗
38.4
33.7
Smooth Shells [ 14 ]
CVPR 2020
✓
✗
✗
36.1
11.8
DiscreteOp [ 35 ]
CGF 2021
✓
✗
✗
38.1
35.5
▼ Functional Map Methods
UnsupFMNet [ 19 ]
CVPR 2019
✓
✗
✗
-
38.5
Table 2: Average geodesic errors ×100 ( ↓ ) of GLASS and baselines on non-isometric benchmarks. The best and the second-best are shown in blue and orange respectively.
Method
Venue
Unsup.
Visual
Lang.
FAUST
SCAPE
SHREC19
▼ Axiomatic Methods
BCICP [ 36 ]
TOG 2018
✓
✗
✗
6.4
11.0
8.0
ZoomOut [ 30 ]
TOG 2019
✓
✗
✗
6.1
7.5
7.8
Smooth Shells [ 14 ]
CVPR 2020
✓
✗
✗
2.5
4.7
7.6
▼ Functional Map Methods
UnsupFMNet [ 19 ]
CVPR 2019
✓
✗
✗
4.8
9.6
11.1
Table 3: Average geodesic errors ×100 ( ↓ ) of GLASS and baselines on near-isometric benchmarks. The best and the second-best are shown in blue and orange respectively.
Method
SMAL
TOPKIDS
Full (FM loss + GAC loss + XYZ + SD-DINO + SigLip)
4.5
5.6
Texturing strategy
FM loss + SD-DINO (no texturing)
7.8
10.5
FM loss + SD-DINO + SyncMVD (view-consistent)
4.9
5.9
Semantic Features
FM loss + WKS
27.3
11.2
Table 4: Ablation study on novel components. Average geodesic errors ( ×100 ) comparing different design choices on SMAL and TOPKIDS.
Time
Peak Memory
Colorization
≈30 s
6 GB
Segmentation
≈80 s
5 GB
SD-DINO Feature
≈5 s
19 GB
Inference (per-pair)
<1 s
3 GB
Table 5: Computational time and memory consumption of each component.
Establishing dense correspondences between shapes is a crucial task in computer vision and graphics, while prior approaches depend on near-isometric assumptions and homogeneous subject types (i.e., only operate for human shapes). However, building semantic correspondences for cross-category objects remains challenging and has received relatively little attention. To achieve this, we propose UniMatch, a semantic-aware, coarse-to-fine framework for constructing dense semantic correspondences between strongly non-isometric shapes without restricting object categories. The key insight is to lift "coarse" semantic cues into "fine" correspondence, which is achieved through two stages. In the "coarse" stage, we perform class-agnostic 3D segmentation to obtain non-overlapping semantic parts and prompt multimodal large language models (MLLMs) to identify part names. Then, we employ pretrained vision language models (VLMs) to extract text embeddings, enabling the construction of matched semantic parts. In the "fine" stage, we leverage these coarse correspondences to guide the learning of dense correspondences through a dedicated rank-based contrastive scheme. Thanks to class-agnostic segmentation, language guiding, and rank-based contrastive learning, our method is versatile for universal object categories and requires no predefined part proposals, enabling universal matching for inter-class and non-isometric shapes. Extensive experiments demonstrate UniMatch consistently outperforms competing methods in various challenging scenarios.
Qinfeng Xiao, Guofeng Mei, Bo Yang +3
Hong Kong Polytechnic University, HK SAR · Fondazione Bruno Kessler, Italy · University of Technology Sydney, Australia
Learning dense correspondences across deformable 3D shapes remains a long-standing challenge due to structural variability, non-isometric deformation, and inconsistent topology. Existing methods typically trade off generalization, geometric fidelity, and efficiency. We address this by proposing SGSoft, a unified intrinsic pipeline that (i) constructs a geodesic correspondence field on a canonical template, (ii) learns multimodal dense descriptors guided by pretrained semantic priors with this geodesic correspondence field supervision, (iii) retrieves dense correspondences in a single feed-forward pass via nearest-neighbor search in descriptor space. This formulation enables stable and topology-invariant supervision under large pose variation, structural differences, and remeshing. SGSoft achieves state-of-the-art inter-category generalization while offering the best accuracy-efficiency trade-off among prior methods. It also achieves near real-time inference without pre-alignment, pairwise optimization, or post-refinement. Learned descriptors can be transferred effectively to downstream tasks such as semantic segmentation and deformation transfer, establishing a scalable and deployment-ready paradigm for dense 3D correspondence.
Establishing accurate point-to-point correspondences between non-rigid 3D shapes remains a critical challenge, particularly under non-isometric deformations and topological noise. Existing functional map pipelines suffer from ambiguities that geometric descriptors alone cannot resolve, and spatial inconsistencies inherent in the projection of truncated spectral bases to dense pointwise correspondences. In this paper, we introduce SGMatch, a learning-based framework that couples 3D-lifted semantic cues with trajectory-level feature transport regularization. Specifically, we design a Semantic-Guided Local Cross-Attention module that integrates semantic features from vision foundation models into geometric descriptors while preserving local structural continuity. Furthermore, we adapt conditional flow matching as a time-conditioned feature transport regularizer that promotes spatially coherent point-wise recovery. Experimental results on multiple benchmarks demonstrate that SGMatch achieves competitive performance across near-isometric settings and consistent improvements under non-isometric deformations and topological noise.