SCD4VPR: Multi-modal Scene Change Detection for Long-term Visual Place Recognition Database Update

Diwei Sheng; Vijayraj Gohil; Satyam Gaba; Zihan Liu; Giles Hamilton-Fletcher; John-Ross Rizzo; Yongqing Liang; Chen Feng

cs.CVApr 13, 2026

SCD4VPR: Multi-modal Scene Change Detection for Long-term Visual Place Recognition Database Update

Authors: Diwei Sheng, Vijayraj Gohil, Satyam Gaba, Zihan Liu, Giles Hamilton-Fletcher, John-Ross Rizzo, Yongqing Liang, Chen Feng

Organizations: New York University

Abstract

Long-term autonomy in mobile robotics requires maps that remain accurate as environments change over time. Visual Place Recognition (VPR), a core localization capability, degrades sharply as the temporal gap between query and database images grows, particularly across seasonal transitions. Scene Change Detection (SCD) offers a principled mechanism for database maintenance, but existing methods rely on binary, uni-modal visual features that cannot distinguish structural changes from viewpoint-induced differences - a distinction essential for correct update decisions. We propose SCD4VPR, a scene change detection that jointly reasons about what has changed and distinguishes genuine change from viewpoint-induced difference in a unified vision-language framework. SCD4VPR fuses VLM-generated semantic descriptions with visual features via cross-modal attention and refines predictions with geometric-semantic matching, producing multi-class change masks that separately identify object changes, appearance changes, and viewpoint-induced changes. We introduce NYC-CD, the first real-world street-view SCD benchmark with pixel-level multi-class annotations across 8,122 image pairs. Experiments across four SCD benchmarks show that SCD4VPR consistently improves three architecturally distinct backbones. In a controlled VPR database maintenance experiment on NYU-VPR spanning summer through late winter, we confirm that retrieval performance deteriorates substantially when the database is left unchanged, and show that SCD4VPR-guided updates recover most of this loss (+30.1 R@1 at the largest time gap) while keeping the database far more compact than naive append.

Figures & tables

Dataset	Real/Sim	Pairs	Obj.	App.	VP.
CARLA-OBJCD [ 28 ]	Sim	15000	✓	✗	✓
Changesim [ 26 ]	Sim	$\sim$ 130k	✓	✓	✗
Standardsim [ 27 ]	Sim	12718	✓	✗	✗
CD2014 [ 29 ]	Real	7000	✓	✗	✗
PCD [ 23 ]	Real	200	✓	✗	✗
VL-CMU-CD [ 8 ]	Real	1362	✓	✗	✗

TABLE I : Comparison of major SCD datasets with our NYC-CD dataset. Obj. : object change, App. : appearance change, VP. : viewpoint difference.

Fig. 2 : Four image pairs annotated by our pipeline. Each pair shows $T_{0}$ (top) and $T_{1}$ (bottom) with multi-class change masks: new objects , appearance changes , and viewpoint-induced changes .

Fig. 3 : Overview of the proposed SCD4VPR. Two images ( $T_{0}$ , $T_{1}$ ) are processed by a shared-weight image encoder. The Cross-Modal Feature Enhancer injects language-derived priors about new objects in $T_{1}$ into visual features, producing an initial change mask. The Geometric–Semantic Matching Module refines this mask by enforcing spatial completeness and semantic consistency.

Model	ChangeVPR		VL-CMU-CD		PSCD		NYC-CD (ours)
	F1	IoU	F1	IoU	F1	IoU	F1	IoU
GeSCF	0.47	0.36	0.77	0.65	0.40	0.28	0.16	0.10
GeSCF+SCD4VPR	0.64 (+17%)	0.55 (+19%)	0.82 (+5%)	0.71 (+6%)	0.49 (+9%)	0.37 (+9%)	0.57 (+41%)	0.49 (+38%)
RSCD	0.24	0.16	0.83	0.73	0.54	0.40	0.13	0.17
RSCD+SCD4VPR	0.63 (+39%)	0.54 (+38%)	0.85 (+2%)	0.75 (+2%)	0.59 (+5%)	0.46 (+6%)	0.70 (+57%)	0.58 (+51%)
C-3PO	0.12	0.08	0.50	0.39	0.66	0.54	0.09	0.10

TABLE II : F1 and IoU comparison of GeSCF, RSCD, and C-3PO with and without SCD4VPR across four datasets. Bold: highest per dataset. Green : improvement over baseline.

Class	F1		IoU
Class	RSCD	+SCD4VPR ( $\Delta$ )	RSCD	+SCD4VPR ( $\Delta$ )
Non-change	0.899	0.945 (+4.6%)	0.816	0.895 (+7.9%)
New/Missing Obj.	0.438	0.472 (+3.4%)	0.281	0.309 (+2.8%)
Appearance	0.655	0.703 (+4.8%)	0.487	0.542 (+5.5%)
Viewpoint	0.459	0.563 (+10.4%)	0.298	0.391 (+9.3%)
Mean	0.613	0.671 (+5.8%)	0.471	0.534 (+6.3%)

TABLE III : Multi-class change detection on NYC-CD (RSCD backbone).

Method	F1	IoU
RSCD	0.127	0.168
RSCD + Text (InternVL)	0.513	0.345
RSCD + Text (QwenVL)	0.537	0.401
RSCD + Text (GPT-4o)	0.563	0.424
RSCD + Text + SM	0.584	0.487
RSCD + Text + SM + GM	0.696	0.582

TABLE IV : Ablation on NYC-CD using RSCD as the base architecture.

Fig. 4 : Qualitative comparison on NYC-CD. SCD4VPR produces semantically accurate multi-class masks distinguishing object changes , appearance changes , and viewpoint-induced changes , while RSCD (SOTA baseline) produces fragmented binary detections that miss or misclassify meaningful scene changes.

Strategy	Obj.	App.	VP.	No-chg.	DB size
NU (naïve update)	replace	replace	replace	replace	constant
R (replace)	replace	replace	skip	skip	constant
R+V (replace+viewpoint-add)	replace	replace	add	skip	grows
A (add-all)	add	add	add	skip	grows
NA (naïve append)	add	add	add	add	grows

TABLE V : Database update strategies. Each column shows the action taken for a given SCD4VPR per-pair output category—object change ( Obj. ), appearance change ( App. ), viewpoint-induced change ( VP. ), or no change ( No-chg. ).

Fig. 5 : ( Top ) Recall@1 over four temporal windows; all strategies evaluate against the current database before updating. ( Bottom ) Database size over time. R+V achieves the highest recall at W3 and W4 while maintaining a 27% smaller database than NA. NU, R, and

D_{0}

maintain constant.

Fig. 6 : Recall@1 vs. database size across four update windows. Marker fill shows progression from Window 1 (hollow) to Window 4 (solid, outlined); Windows 2–3 are faded. Static (D 0 ), R, and NU maintain a constant database size of 20,687 images across all windows; their markers are offset horizontally by $\pm$ 500 images purely for visual separation and do not reflect actual database size differences.

Window	Strategy	R@1	R@5	R@10	R@20
W2	Static ( $D_{0}$ )	79.7	86.6	88.0	89.5
	NU	84.1	90.9	92.0	93.0
	R	82.5	89.5	90.9	92.4
	A	86.0	92.0	93.2	94.1
	NA	86.7	92.4	93.6	94.4
	R+V	85.5	91.8	93.0	94.1

TABLE VI : Extended recall metrics (R@1/5/10/20) by strategy and window.

Semantic Change Detection

Semantic change detection identifies how meanings, concepts, or scene contents differ across time in text, images, and other data.

See all 63 papers

VSCD: Video-based Scene Change Detection in Unaligned Scenes
May 20, 2026Jiae Yoon, Ue-Hwan KimTemporal Video Understanding
Department of AI Convergence, Gwangju Institute of Science and Technology, Gwangju, Republic of Korea · GIST InnoCORE AI-Nano Convergence Institute for Early Detection of Neurodegenerative Diseases, Gwangju Institute of Science and Technology, 61005 Gwangju, Republic of Korea.
CogVis: Must Open-Vocabulary Change Detection Perceive the Scene Anew for Every Query?
Aug 6, 2026Zijie Wang, Chen Zhong, Wei HeRemote Sensing Change Detection Semantic Change Detection
Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University Wuhan, China
Environmental Change Detection for Real-World Change Analysis
Jun 13, 2025Kyusik Cho, Suhan Woo, Hongje Seong +1
Yonsei University · University of Seoul

May 20, 2026J/K move · Enter open · S save

VSCD: Video-based Scene Change Detection in Unaligned Scenes

Jiae Yoon, Ue-Hwan Kim

Department of AI Convergence, Gwangju Institute of Science and Technology, Gwangju, Republic of Korea · GIST InnoCORE AI-Nano Convergence Institute for Early Detection of Neurodegenerative Diseases, Gwangju Institute of Science and Technology, 61005 Gwangju, Republic of Korea.

Temporal Video Understanding

Open paper AskPDF arXiv

Detecting what has changed in an environment is essential for long-term autonomy, yet most change detection settings assume fixed viewpoints, mild misalignment, or only a few changed objects. We introduce Video-based Scene Change Detection (VSCD), which predicts a pixel-wise change mask for each query frame, given a reference and a query RGB video of the same indoor space recorded at different times under unconstrained camera motion. The two videos are not temporally synchronized, and many object instances may appear or disappear. To study this setting, we build a large-scale benchmark with over 1.1 million frames annotated with pixel-accurate change masks, together with a real-world test set for evaluating transfer beyond simulation. We propose a query-centric multi-reference model that learns temporal matching implicitly from change-mask supervision, aligns candidate reference features to the query via local patch correspondence, and fuses per-candidate change features using frame-level and patch-level confidence before decoding a high-resolution mask once per frame. Our approach achieves state-of-the-art performance against strong image- and video-based baselines, and we validate its real-world impact by deploying it on a mobile robot for two downstream applications -- visual surveillance and object incremental learning.

Abstract

Semantic Change Detection

Explore similar work

VSCD: Video-based Scene Change Detection in Unaligned Scenes

CogVis: Must Open-Vocabulary Change Detection Perceive the Scene Anew for Every Query?

Environmental Change Detection for Real-World Change Analysis