Dense 3D shape correspondence remains a central challenge in computer vision and graphics as many deep learning approaches still rely on intermediate geometric features or handcrafted descriptors, limiting their effectiveness under non-isometric deformations, partial data, and non-manifold inputs. To overcome these issues, we introduce RINO, an unsupervised, rotation-invariant dense correspondence framework that effectively unifies rigid and non-rigid shape matching. The core of our method is the novel RINONet, a feature extractor that integrates vector-based SO(3)-invariant learning with orientation-aware complex functional maps to extract robust features directly from raw geometry. This allows for a fully end-to-end, data-driven approach that bypasses the need for shape pre-alignment or handcrafted features. Extensive experiments show unprecedented performance of RINO across challenging non-rigid matching tasks, including arbitrary poses, non-isometry, partiality, non-manifoldness, and noise.
Figures & tables
Figure 2: Our learned SO(3)-invariant features. We visualize the Euclidean distance to the blue surface point in the learned feature space (darker red means higher similarity). Semantic correspondences have a similar feature similarity pattern (left & mid.). The learned feature is invariant to rotations (mid. & right). Shape orientations are depicted by RGB frames.
Figure 3: SO(3)-invariant correspondences. Our method is unaffected by shape orientations. In contrast, baselines (represented by DUOFM) perform well only when train and test shapes are pre-aligned (I/I), failing dramatically on unseen rotations (I/SO(3)).
Figure 4: We propose RINONet , which learns SO(3)-invariant features F from input shape X . Our novel network inherits all nice properties from DiffusionNet [ 54 ] , and learns smooth, high-quality per-point features, while additionally retaining invariant to SO(3) actions applied to input shapes by employing vector neurons as the feature representation in our hidden layers. The RINONet has a simple structure and contains four consecutive RINONet Blocks at its core, which is combined with VN-EdgeConv and VN-linear layers to achieve the desired I/O dimensionality. The VN-invariant layer is employed to convert SO(3)-equivariant features to invariant ones.
Figure 5: Our RINONet block is the core of our RINONet, and it consists of three main modules: a VN-Diffusion layer, a VN-Gradient layer, and a VN-MLP. In contrast to the original DiffusionNet block [ 54 ] , ours is SO(3)-equivariant by design.
Figure 6: Intrinsic symmetry. Ours fully resolves symmetry, while baselines suffer. Top: partial left-right (legs) and front-back (belly-back) flips. Bottom: full left-right flips (arms and legs).
Figure 7: Overview of RINO. Our novel RINONet is used to extract SO(3)-invariant features FX and FY from input shapes X and Y . The features are further processed in three non-learnable but differentiable blocks: the FMaps solver to compute the functional map CXY , the CFMaps solver to compute the complex functional map QXY (based on feature gradients), and the pointwise map block to compute the soft pointwise map ΠXY based on feature similarity. Finally, our unsupervised loss prompts structural properties of CXY,QXY and consistency between different map representations.
Train/Test
I/I
I/SO(3)
SO(3)/SO(3)
Y/Y
CnsFM
5.4
58.7
9.1
5.4
DUOFM
11.7
41.4
25.2
16.5
URSSM
4.8
62.1
24.8
7.9
SMS
4.6
58.5
57.6
39.3
SmpFM
5.6
62.6
9.9
7.3
HbrFM
5.6
61.8
26.4
5.3
Table 1: SO(3)-invariance on SMAL. The rotation type is indicated by: I (aligned), SO(3) (fully random rotation), and Y (y-axis rotation). e.g. I/SO(3) means aligned shapes for training and fully randomly rotated shapes for testing. We report mGeoErr ( ↓ ) and ours consistently outperforms baselines, especially on unseen rotations. Best results are bold , and highlighted cells indicate the experiment settings used in baseline papers.
wks
xyz
SMAL
DT4D
FSCAN
SMAL
DT4D
FSCAN
CnsFM
23.4
9.3
3.6
9.1
7.4
50.5
DUOFM
6.7
15.8
-
25.2
55.8
-
URSSM
29.6
8.4
3.0
24.8
59.3
24.8
SMS
30.6
58.6
12.1
57.6
64.9
54.0
SmpFM
29.9
15.3
5.7
9.9
5.5
24.3
Table 2: Non-isometric and raw scan matching. We report mGeoErr ( ↓ ) and highlight the best and 2nd best results. Ours outperforms all baselines regardless of the input types. Highlighted cells indicate the experiment settings used in baseline papers.
Figure 8: Non-isometric matching on SMAL and DT4D. Our method outperforms all baselines; it is invariant to extrinsic rigid poses (no pre-alignment) and robust to intrinsic shape symmetry by directly processing raw shape geometry.
FAUST
SCAPE
E
ES
#Flips
E
ES
#Flips
CnsFM
4.4
2.8
25
27.2
7.4
249
DUOFM
28.0
7.2
204
28.3
8.9
232
URSSM
30.0
7.8
240
26.9
7.7
245
SmpFM
3.0
2.7
1
29.9
7.9
63
HbrFM
29.9
7.2
227
27.2
8.1
236
Table 3: Symmetry analysis . We compute mGeoErr ( ↓ ) by allowing only symmetry-free correspondences (col. E ) and additionally the symmetric ones (col. ES ), and report the number of symmetrically flipped estimations out of 400 test pairs (col. #Flips ).
Figure 9: Qualitative results on SHREC16-Partiality.
URSSM
EchoMatch
Wormhole
Ours
CUTS
42.1
12.7
48.2
6.7
HOLES
29.8
57.1
26.9
12.09
Table 4: mGeoErr ( ↓ ) on SHREC16-Partiality. Ours outperforms all baselines, including the supervised method EchoMatch.
Figure 10: Human Segmentation. We show the segmentation results of DiffusionNet and ours on the composite dataset [ 40 ] . Our RINONet produces sharper segments.
Figure 11: Robustness under noise perturbations.
Ours
URSSM
DUOFM
CnsFM
SMS
SmpFM
HbrFM
#params(k, ↓ )
327.7
510.3
477.8
477.8
1,266.2
1,848.4
494.3
runtime(s, ↓ )
0.065
0.023
0.071
0.055
15.311
0.171
0.043
mGeoErr ( ↓ )
4.6
24.8
25.2
9.1
57.6
9.9
26.4
Table 5: Comparison of runtime, #params and mGeoErr on SMAL.
Figure 12: MGeoErr( ↓ ) on SCAPE, FAUST, SCHREC19. (Left) trained on SCAPE; (Right) trained on FAUST. Y-axis dataset labels indicate the test dataset. MGeoErr is annotated on top of each bar. All baselines significantly deteriorate when changing from wks to xyz, whereas ours is more robust to these changes. To our best knowledge, RINO is the first method, which learns competitive features and correspondences directly from raw 3D geometry in a purely data-driven manner.
Figure 13: Statistics of wks vs. xyz. The box plot illustrates the statistical distribution of the mean feature channel values at semantically corresponding points across the entire shape collections of the SMAL and SCAPE datasets. The methodology involved aggregating the channel-wise mean of features across corresponding points for every shape, yielding a feature matrix (with dimensions #correspondences × #feature channels), where each column’s distribution is plotted as a box. The statistics shown for the 5th, 10th, 15th, and 20th wks channels demonstrate a pronounced difference in behavior between the datasets. Specifically, the wks statistics for SMAL display a substantially larger variance than those for SCAPE . This higher dispersion is a direct consequence of the non-isometries present in the SMAL dataset, which contrasts sharply with the stability observed in the xyz descriptor statistics.
Figure 14: Non-manifold shape matching. Methods employing the wks as input exhibit a significant performance deterioration, even for SoTA methods such as URSSM and HbrFM, an observation that also applies to our proposed method (Ours+wks). Fortunately, our SO(3)-invariance network allows for effective learning directly from the raw xyz coordinates (Ours). This approach offers robustness against the difficulties posed by non-manifoldness, bypassing the fragility observed when using wks.
FAUST
FAUST non-manifold
URSSM(wks)
1.6
17.6
HbrFM(wks)
1.4
16.6
Ours(wks)
1.6
15.6
Ours(xyz)
1.6
2.7
Table 6: MGeoErr( ↓ ) on FAUST non-manifold shapes.
Figure 15: Human Segmentation. We show the segmentation results of DiffusionNet and our RINONet on the composite dataset [ 40 ] . Our RINONet produces sharper and more semantically meaningful segments.
Method
Human dataset [ 40 ]
DiffusionNet(hks, w/o aug)
91.3%
Ours(hks, w/o aug)
91.0%
DiffusionNet(xyz, w/ aug)
90.3%
DiffusionNet(xyz, w/o aug)
83.6%
Ours(xyz, w/ aug)
90.6%
Ours(xyz, w/o aug)
90.1%
Table 7: Segmentation accuracy ( ↑ ) on human dataset [ 40 ] .
Figure 16: RNA segmentation
Figure 17: 3D rigid registration on Stanford dataset (bunny)
Non-rigid 3D shape matching is a fundamental task in computer vision and graphics. In this paper, we propose a hybrid self-supervised method based on a coarse-to-fine strategy, which ensures consistency between the coarse mapping and the refined correspondence produced by our refinement module. The architecture features a dual-branch design, consisting of two symmetric functional map learning streams: one based on the Laplacian basis and the other utilizing the elastic basis. Extensive experiments show that our approach not only maintains computational efficiency, but also achieves state-of-the-art performance across a variety of challenging scenarios, including non-isometric deformations and topological noise. Finally, we rigorously demonstrate that contrastive energies promote feature discrimination. Furthermore, integrating these energies with existing methods yields consistent improvements, validating the overall efficacy of our approach. Our code is available at https://github.com/LuoFeifan77/Coarse-to-Fine-Hybrid-Self-Supervised-Matching.
Feifan Luo, Ting Li, Zhao Li +1
Zhejiang University, China · Zhejiang Lab, China · Wenzhou University of Technology, China
Establishing accurate point-to-point correspondences between non-rigid 3D shapes remains a critical challenge, particularly under non-isometric deformations and topological noise. Existing functional map pipelines suffer from ambiguities that geometric descriptors alone cannot resolve, and spatial inconsistencies inherent in the projection of truncated spectral bases to dense pointwise correspondences. In this paper, we introduce SGMatch, a learning-based framework that couples 3D-lifted semantic cues with trajectory-level feature transport regularization. Specifically, we design a Semantic-Guided Local Cross-Attention module that integrates semantic features from vision foundation models into geometric descriptors while preserving local structural continuity. Furthermore, we adapt conditional flow matching as a time-conditioned feature transport regularizer that promotes spatially coherent point-wise recovery. Experimental results on multiple benchmarks demonstrate that SGMatch achieves competitive performance across near-isometric settings and consistent improvements under non-isometric deformations and topological noise.
Estimating correspondences between deformed shape instances is a long-standing problem in computer graphics; numerous applications, from texture transfer to statistical modelling, rely on recovering an accurate correspondence map. Many methods have thus been proposed to tackle this challenging problem from varying perspectives, depending on the downstream application. This state-of-the-art report is geared towards researchers, practitioners, and students seeking to understand recent trends and advances in the field. We categorise developments into three paradigms: spectral methods based on functional maps, combinatorial formulations that impose discrete constraints, and deformation-based methods that directly recover a global alignment. Each school of thought offers different advantages and disadvantages, which we discuss throughout the report. Meanwhile, we highlight the latest developments in each area and suggest new potential research directions. Finally, we provide an overview of emerging challenges and opportunities in this growing field, including the recent use of vision foundation models for zero-shot correspondence and the particularly challenging task of matching partial shapes.
Aleksei Zhuravlev, Lennart Bastian, Dongliang Cao +12
MPI for Informatics · Imperial College London · Technical University of Munich +5