Semantic Correspondence

Semantic correspondence aims to identify corresponding elements (pixels, points, regions) in different images or 3D shapes that share the same semantic meaning, despite variations in appearance, viewpoint, or modality. Current research heavily utilizes deep learning, particularly transformer networks and diffusion models, to learn robust feature representations and establish correspondences, often incorporating techniques like contrastive learning and self-supervised training to address data scarcity and annotation challenges. This field is crucial for advancing numerous computer vision and robotics applications, including image translation, object pose estimation, 3D shape reconstruction, and video editing, by enabling more accurate and robust understanding of visual scenes and object interactions.

Papers

June 6, 2023

Emergent Correspondence from Image Diffusion
Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, Bharath Hariharan
Image Diffusion Model Semantic Correspondence Joint Entity Diffusion Network Image Diffusion Temporal Correspondence Diffusion Model Feature

May 24, 2023

Unsupervised Semantic Correspondence Using Stable Diffusion
Eric Hedlin, Gopal Sharma, Shweta Mahajan, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, Kwang Moo Yi
Text to Image Diffusion Model Jina Embeddings Stable Diffusion Semantic Information Semantic Correspondence Prompt Embeddings

May 23, 2023

Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondence
Grace Luo, Lisa Dunlap, Dong Huk Park, Aleksander Holynski, Trevor Darrell
Diffusion Model Time Matter Deep Space Image Pair Semantic Correspondence Diffusion Model Feature

May 13, 2023

Temporal Consistent Automatic Video Colorization via Semantic Correspondence
Yu Zhang, Siqi Chen, Mingdao Wang, Xianlin Zhang, Chuang Zhu, Yue Zhang, Xueming Li
Semantic Correspondence Video Colorization Line Art Colorization

May 3, 2023

SimSC: A Simple Framework for Semantic Correspondence with Temperature Learning
Xinghui Li, Kai Han, Xingchen Wan, Victor Adrian Prisacariu
Supervised ImageNet Semantic Matching Feature Matching Semantic Correspondence Feature Backbone Fine Tuning

March 1, 2023

Bootstrapping Parallel Anchors for Relative Representations
Irene Cannistraci, Luca Moschella, Valentino Maiorca, Marco Fumero, Antonio Norelli, Emanuele Rodolà
Semantic Correspondence Latent Embeddings Adaptive Sparse Anchor Relative Representation Embedded Space

December 29, 2022

Learning Implicit Functions for Dense 3D Shape Correspondence of Generic Objects
Feng Liu, Xiaoming Liu
Dense Correspondence Semantic Correspondence Generic Model Shape Latent Deep Implicit Function Dense Shape Correspondence

October 12, 2022

Multi-Granularity Cross-modal Alignment for Generalized Medical Visual Representation Learning
Fuying Wang, Yuyin Zhou, Shujun Wang, Varut Vardhanabhuti, Lequan Yu
Representation Learning Cross Modal Alignment Semantic Correspondence Cross Modality Alignment

October 6, 2022

Neural Matching Fields: Implicit Representation of Matching Fields for Visual Correspondence
Sunghwan Hong, Jisu Nam, Seokju Cho, Susung Hong, Sangryul Jeon, Dongbo Min, Seungryong Kim
Implicit Representation Matching Accuracy Semantic Correspondence Visual Correspondence Pixel Correspondence Matching Field

August 15, 2022

Learning Semantic Correspondence with Sparse Annotations
Shuaiyi Huang, Luyu Yang, Bo He, Songyang Zhang, Xuming He, Abhinav Shrivastava
Noisy Pseudo Label Semantic Correspondence Sparse Label Sparse Annotation Label Denoising Dense Pseudo Label

August 7, 2022

CVLNet: Cross-View Semantic Correspondence Learning for Video-based Camera Localization
Yujiao Shi, Xin Yu, Shan Wang, Hongdong Li
Semantic Correspondence Video Localization

July 21, 2022

Semantic-Aware Fine-Grained Correspondence
Yingdong Hu, Renhao Wang, Kaifeng Zhang, Yang Gao
Fine Grained Dense Correspondence Semantic Correspondence Image Correspondence Complementary Representation

July 11, 2022

Demystifying Unsupervised Semantic Correspondence Estimation
Mehmet Aygün, Oisin Mac Aodha
Unsupervised Learning Unsupervised Method Semantic Matching Semantic Correspondence Correspondence Prediction

June 28, 2022

Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsing
Lihan Wang, Bowen Qin, Binyuan Hui, Bowen Li, Min Yang, Bailin Wang, Binhua Li, Fei Huang, Luo Si, Yongbin Li
Pre Trained Language Model Text to SQL Full Information Graph Based Semantic Correspondence Text to SQL Parser Relational Structure Schema Inference

June 27, 2022

ContraReg: Contrastive Learning of Multi-modality Unsupervised Deformable Image Registration
Neel Dey, Jo Schlemper, Seyed Sadegh Mohseni Salehi, Bo Zhou, Guido Gerig, Michal Sofka
Contrastive Learning Semantic Correspondence Multi Modal Alignment Modality Registration Brain MRI Registration

May 23, 2022

TransforMatcher: Match-to-Match Attention for Semantic Correspondence
Seungwook Kim, Juhong Min, Minsu Cho
Human Attention Semantic Correspondence Attention Scheme

April 5, 2022

Joint Learning of Feature Extraction and Cost Aggregation for Semantic Correspondence
Jiwon Kim, Youngjo Min, Mira Kim, Seungryong Kim
Feature Extraction Weakly Supervised Joint Learning Dense Correspondence Semantic Correspondence Cost Aggregation Confidence Aware Contrastive

March 30, 2022

Semi-Supervised Learning of Semantic Correspondence with Pseudo-Labels
Jiwon Kim, Kwangrok Ryoo, Junyoung Seo, Gyuseong Lee, Daehwan Kim, Hansang Cho, Seungryong Kim
Semi Supervised Learning Semi Supervised Ground Truth Weakly Supervised Learning Dense Correspondence Semantic Correspondence

March 8, 2022