cs.CVOct 23, 2026

SemGeo-Gen: Unsupervised Generation of Approximate Cross-Instance Semantic-Geometric Correspondences

Authors: Roy Amoyal, Shira Ifergane, Oren Freifeld

Organizations: The Stein Faculty of Computer and Information Science, Ben-Gurion University of the Negev, Israel

Abstract

We introduce SemGeo-Gen, an unsupervised method for generating approximate cross-instance semantic-geometric supervision from weakly aligned 3D object collections. Our goal is not to recover exact point-level correspondences, which are often ambiguous across different instances and would require expensive, impractical manual supervision at scale. Instead, we automatically produce approximate correspondences that are semantically meaningful, geometrically consistent, and diverse enough to train modern correspondence models. Given only coarse category-level rotation alignment, SemGeo-Gen lifts multi-view DINOv2 features to 3D, aligns object instances using continuous piecewise-affine registration, and prunes candidate matches using semantic consistency. The resulting 3D correspondences can be projected into rendered views, yielding scalable 3D-3D, 3D-2D, and 2D-2D supervision without manual annotation. We validate the generated correspondences against sparse human annotations from KeypointNet, obtaining 93% [email protected] and 97% [email protected]. More importantly, we show that using the generated correspondences for synthetic pretraining improves recent state-of-the-art semantic correspondence models on SPair-71k after fine-tuning on real data, with gains of up to 5 [email protected] points. Together, these results indicate that approximate semantic-geometric supervision generated from 3D assets can improve real-image correspondence learning and serve as a scalable alternative to costly manual annotation. Project page: https://bgu-cs-vil.github.io/SemGeo-Gen/.

Figures & tables

Appendix figures & tables33 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Geometry Matters: 3D Foundation Priors for Learning Semantic Correspondence

    May 28, 2026Artur Jesslen, Olaf Dünkel, Adam KortylewskiSemantic Correspondence

  2. Shape-of-You: Fused Gromov-Wasserstein Optimal Transport for Semantic Correspondence in-the-Wild

    Mar 12, 2026Jiin Im, Sisung Liu, Je Hyeong HongSemantic CorrespondenceOptimal Transport

  3. GeoWorld: Providing Full-frame Geometry Features to Facilitate 3D Scene Generation

    Nov 28, 2025Yuhao Wan, Lijuan Liu, Jingzhi Zhou +6Video Diffusion Models3D Scene Generation