VidTAG: Temporally Aligned Video to GPS Geolocalization with Denoising Sequence Prediction at a Global Scale

Parth Parag Kulkarni; Rohit Gupta; Prakash Chandra Chhipa; Mubarak Shah

cs.CVApr 14, 2026

VidTAG: Temporally Aligned Video to GPS Geolocalization with Denoising Sequence Prediction at a Global Scale

Authors: Parth Parag Kulkarni, Rohit Gupta, Prakash Chandra Chhipa, Mubarak Shah

Organizations: Institute of Artificial Intelligence, University of Central Florida, USA · Amazon Prime Video Science, USA · Lule˚a Tekniska Universitet, Sweden

Abstract

The task of video geolocalization aims to determine the precise GPS coordinates of a video's origin and map its trajectory; with applications in forensics, social media, and exploration. Existing classification-based approaches operate at a coarse city-level granularity and fail to capture fine-grained details, while image retrieval methods are impractical on a global scale due to the need for extensive image galleries which are infeasible to compile. Comparatively, constructing a gallery of GPS coordinates is straightforward and inexpensive. We propose VidTAG, a dual-encoder framework that performs frame-to-GPS retrieval using both self-supervised and language-aligned features. To address temporal inconsistencies in video predictions, we introduce the TempGeo module, which aligns frame embeddings, and the GeoRefiner module, an encoder-decoder architecture that refines GPS features using the aligned frame embeddings. Evaluations on Mapillary (MSLS) and GAMa datasets demonstrate our model's ability to generate temporally consistent trajectories and outperform baselines, achieving a 20% improvement at the 1 km threshold over GeoCLIP. We also beat current State-of-the-Art by 25% on global coarse grained video geolocalization (CityGuessr68k). Our approach enables fine-grained video geolocalization and lays a strong foundation for future research. More details on the project webpage: https://parthpk.github.io/vidtag_webpage/

Figures & tables

Figure 1: Frame-wise Video to GPS Sequence Prediction. Our model geolocalizes every frame in a video on a global scale to give a temporally consistent trajectory

Figure 2: Temporal Inconsistency Issue. Geolocalizing each frame separately leads to incoherent trajectories. The problem becomes worse as the scale increases. Yellow: GT, Red: Predicted.

Figure 3: Schematic Illustration of the proposed Approach. The training of the model is divided in two phases. (a) Phase I training. Each frame is simultaneously passed through DINOv2 and CLIP, and concatenated to generate an embedding. TempGeo module specifically instills positional order in frames of a video and aligns them with other frames, inculcating temporal consistency within frame embeddings. The Video Frame Encoder and TempGeo module are contrastively trained with the Location Encoder. (b) Phase II training. The GeoRefiner module encodes the temporally aligned frame embeddings of a video. Noisy GPS embeddings are input into the decoder along with the encoded frame embeddings. GPS embeddings are cross-attended with the frame embeddings, which is trained to denoise.

Figure 4: Phase I failure modes Most commonly observed noise patterns in Phase I predictions. Phase II training refines GPS predictions to address these issues. Yellow: GT, Red: Predicted.

Figure 5: Inference Procedure. Frames are passed through the dual encoders followed by TempGeo to generate frame-wise embeddings, while GPS embeddings are obtained from a gallery of coordinates via the location encoder. The frame-wise embeddings are compared with GPS embeddings to make initial predictions. These predictions, along with corresponding frame embeddings, are refined by the GeoRefiner model, which outputs refined GPS embeddings. A second retrieval with refined embeddings produces final predictions.

Model	Frame-wise Metrics					Video Metrics
	Distance ( $a_{r}$ [%] @ km) $\uparrow$				Median Distance Error(km) $\downarrow$	Distance ( $a_{r}$ [%] @ km) $\uparrow$				Median Distance Error(km) $\downarrow$	DFD $\downarrow$	MRD $\downarrow$
	Sub-street	Street	Locality	City		Sub-street	Street	Locality	City
	0.5 km (500 m)	1 km	5 km	25 km		0.5 km (500 m)	1 km	5 km	25 km
GeoCLIP 54 -ZeroShot	0.9	2.7	22.9	84.6	11.54	1.3	3.8	27.6	89.9	9.85	24.94	2.83
PlaNet 59	4.4	10.5	38.4	81.0	7.38	3.9	8.6	35.6	78.9	7.66	67.42	9.86
ISNs 32	4.9	11.2	38.6	84.3	7.22	4.3	10.1	36.0	80.4	7.51	62.55	8.95

Table 1: Comparison of our model with baseline methods on Mapillary (MSLS) dataset

Model	Frame-wise Metrics					Video Metrics
	Distance ( $a_{r}$ [%] @ km) $\uparrow$				Median Distance Error(km) $\downarrow$	Distance ( $a_{r}$ [%] @ km) $\uparrow$				Median Distance Error(km) $\downarrow$	DFD $\downarrow$	MRD $\downarrow$
	Sub-street	Street	Locality	City		Sub-street	Street	Locality	City
	0.5 km	1 km	5 km	25 km		0.5 km	1 km	5 km	25 km
GeoCLIP 54 -ZeroShot	4.1	10.1	33.5	74.9	11.15	4.0	9.6	32.2	75.3	11.08	14.29	1.51
PlaNet 59	7.7	15.4	44.1	84.5	6.26	6.3	14.3	43.5	86.4	6.21	43.19	6.76
ISNs 32	8.2	16.3	44.8	86.1	6.11	7.0	14.9	45.1	87.2	6.06	40.07	6.14

Table 2: Comparison of our model with baseline methods on GAMa split of the BDD100k dataset

Figure 6: Qualitative Results. Mapped near-perfect geolocalized trajectories predicted by VidTAG. Yellow: GT, Red: Predicted.

Model	City $\uparrow$	State $\uparrow$	Country $\uparrow$	Continent $\uparrow$
PlaNet 59	55.8	56.3	60.8	74.1
ISNs 32	59.5	59.9	64.1	75.9
GeoDecoder 5	64.2	64.5	69.5	79.9
GeoReasoner 26	38.5	42.8	64.4	81.9
Qwen2.5-VL 2	55.1	59.9	78.2	91.3
OSV 1	23.1	33.5	55.6	78.1

Table 3: City-level prediction results on CityGuessr68k

Ablation Component				Frame-wise Metrics			Video Metrics
Ablation Component				Distance ( $a_{r}$ [%] @ km) $\uparrow$		Median Distance Error(km) $\downarrow$	Distance ( $a_{r}$ [%] @ km) $\uparrow$		Median Distance Error(km) $\downarrow$	DFD $\downarrow$	MRD $\downarrow$
Backbone		TempGeo	GeoRefiner	Street	Locality		Street	Locality
CLIP	DINOv2	TempGeo	GeoRefiner	1 km	5 km		1 km	5 km
✓	✗	✗	✗	22.5	63.0	2.97	18.6	58.2	3.69	22.52	2.82
✗	✓	✗	✗	26.4	70.1	2.13	24.7	69.4	2.32	9.15	1.03
✗	✓	✓	✗	30.2	72.5	1.91	30.5	72.3	1.99	3.01	0.41

Table 4: Component-wise Ablation Study

Figure 7: Performance vs Throughput Tradeoff. Our model is significantly better with only a minimal throughput decrease.

Resolution	Distance (a_r [%] @ km) $\uparrow$			Median Distance Error(km) $\downarrow$	DFD $\downarrow$	MRD $\downarrow$
Resolution	Sub-street	Street	Locality
(km)	0.5 km (500 m)	1 km	5 km
1	16.7	38.6	75.1	1.43	4.21	1.18
0.5	20.1	40.3	75.5	1.35	3.98	1.11
0.1	21.5	41.0	76.7	1.35	3.87	1.07

Table 5: Impact of Gallery Resolution

Encoders	Distance ( $a_{r}$ [%] @ km) $\uparrow$				Median Distance Error(km) $\downarrow$
	Sub-street	Street	Locality	City
	0.5 km (500 m)	1 km	5 km	25 km
CLIP + SigLIP	11.4	27.8	71.0	97.1	2.15
CLIP + DINOv2	22.9	40.1	75.4	97.4	1.38

Table 6: Effectiveness of our dual encoder

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

$\alpha$ , $\beta$	Distance ( $a_{r}$ [%] @ km) $\uparrow$				Median Distance Error(km) $\downarrow$	DFD $\downarrow$	MRD $\downarrow$
	Sub-street	Street	Locality	City
	0.5 km (500 m)	1 km	5 km	25 km
1,1	21.5	41.3	75.5	97.4	1.36	5.21	1.41
10,1	21.5	41.0	76.7	97.9	1.35	3.87	1.07
100,1	21.4	40.9	75.4	97.2	1.38	7.63	1.57

Appendix

Table 7: Ablation for values of

\alpha

and

\beta

No. of MLP Layers	Distance ( $a_{r}$ [%] @ km) $\uparrow$				Median Distance Error(km) $\downarrow$	Trainable Parameters
No. of MLP Layers	Sub-street 0.5 km (500 m)	Street 1 km	Locality 5 km	City 25 km	Median Distance Error(km) $\downarrow$	Trainable Parameters
2	12.5	32.6	75.4	98.5	1.71	55.7M
3	13.5	33.1	77.4	98.7	1.66	56.3M
4	13.8	31.9	75.6	98.1	1.72	57.4M
5	12.8	30.6	76.2	98.8	1.82	58M

Appendix

Table 8: Ablation on Number of layers in MLP in Image Encoder

$\sigma$ values			Distance ( $a_{r}$ [%] @ km) $\uparrow$				Median Distance Error(km) $\downarrow$
$\sigma_{0}$	$\sigma_{1}$	$\sigma_{2}$	Sub-street 0.5 km (500 m)	Street 1 km	Locality 5 km	City 25 km	Median Distance Error(km) $\downarrow$
$2^{0}$	$2^{3}$	$2^{8}$	13.5	33.1	77.4	98.7	1.66
$2^{0}$	$2^{4}$	$2^{7}$	13.5	32.5	77.4	98.5	1.71
$2^{0}$	$2^{4}$	$2^{8}$	13.0	32.7	75.4	98.7	1.68
$2^{0}$	$2^{4}$	$2^{9}$	13.3	32.3	77.3	98.8	1.71
$2^{0}$	$2^{5}$	$2^{8}$	13.1	31.5	76.6	98.8	1.78

Appendix

Table 9: Ablation on

\sigma

values for RFF in Location Encoder

Training Sequence Length (No. of frames)	Distance ( $a_{r}$ [%] @ km) $\uparrow$				Median Distance Error(km) $\downarrow$
Training Sequence Length (No. of frames)	Sub-street 0.5 km (500 m)	Street 1 km	Locality 5 km	City 25 km	Median Distance Error(km) $\downarrow$
4	11.0	26.6	73.2	99.1	2.13
8	12.8	31.3	74.9	98.7	1.80
16	13.5	33.1	77.4	98.7	1.66

Appendix

Table 10: Ablation on Training Sequence Length

Figure 8: Distance Error vs Number of Frames. Median error is consistent, showing robustness of our model.

Model	Uniform Grid Gallery Evaluation					Validation-set Gallery Evaluation					$\Delta a_{r}$ [%] @ 1 km	$\Delta$ Median Distance
	Distance ( $a_{r}$ [%] @ km)				Median Distance Error(km)	Distance ( $a_{r}$ [%] @ km)				Median Distance Error(km)
	Sub-street	Street	Locality	City		Sub-street	Street	Locality	City
	0.5 km (500 m)	1 km	5 km	25 km		0.5 km (500 m)	1 km	5 km	25 km
GeoCLIP-FineTuned	8.3	22.5	63.0	93.9	2.97	19.9	31.1	66.3	94.1	2.30	8.6	0.67
VidTAG (Ours)	21.5	41.0	76.7	97.9	1.35	30.8	46.1	77.8	96.3	1.17	5.1	0.18

Appendix

Table 11: Comparison of Uniform Grid Gallery evaluation and Validation-set Gallery evaluation on Mapillary(MSLS) dataset

Panel of Figure 9 — Figure 9: Feature Space Visualization comparison for output of GeoCLIP 54 and VidTAG. Figure shows the feature analysis in the form of PCA plots for GeoCLIP and our model. First row shows The GPS spread of sequences in Melbourne and Helsinki respectively. Second row displays corresponding PCA plots for sequence-wise outputs of GeoCLIP, while row 3 displays the corresponding PCA plots for our model VidTAG. We see that the features of VidTAG are a lot more distinct as compared to GeoCLIP, whose features are more scattered and unorganized.

Model	Mapillary(MSLS)							GAMa
	Distance ( $a_{r}$ [%] @ km) $\uparrow$				Median Distance Error(km) $\downarrow$	DFD $\downarrow$	MRD $\downarrow$	Distance ( $a_{r}$ [%] @ km) $\uparrow$				Median Distance Error(km) $\downarrow$	DFD $\downarrow$	MRD $\downarrow$
	Sub-street	Street	Locality	City				Sub-street	Street	Locality	City
	0.5 km (500 m)	1 km	5 km	25 km				0.5 km (500 m)	1 km	5 km	25 km
Dataset-specific Models
GeoCLIP-FineTuned	8.3	22.5	63.0	93.9	2.97	22.52	2.82	16.3	28.3	57.8	89.1	3.28	6.50	0.50
VidTAG	21.5	41.0	76.7	97.9	1.35	3.87	1.07	35.4	53.1	77.8	94.4	0.88	0.39	0.17

Appendix

Table 12: Performance of Unified Model on Mapillary(MSLS) and GAMa datasets

Model	City $\uparrow$	State $\uparrow$	Country $\uparrow$	Continent $\uparrow$
CityGuessr	69.6	70.2	74.8	83.8
VidTAG	94.9	95.5	96.8	98.5
VidTAG (Unified)	79.0	80.2	89.1	95.0

Appendix

Table 13: Performance of Unified Model on CityGuessr68k

Model	Test GMACs	Train Time	Acc@1km
GeoCLIP-ZS	81.91	272 hrs	2.7
GeoCLIP (FT)		169 hrs	22.5
GeoCLIP (total)		441 hrs	22.5
Ours	83.47	68 hrs	41.0
with pre-computed backbone features
GeoCLIP-ZS	-	2.5 hours	2.7

Appendix

Table 14: Efficiency Analysis of proposed approach

Panel of Figure 10 — Figure 10: Qualitative Examples for trajectories of sequences in GaMa 56 dataset. Row 1 shows almost perfectly predicted trajectories, while row 2 shows coherent trajectories predicted close to the actual trajectory.

Panel of Figure 11 — Figure 11: Figure shows confusion matrix heatmaps for 24 most difficult cities in CityGuesr68k for models VidTAG and CityGuessr respectively. It shows that our model can distinguish cities better, in cases where CityGuessr struggles. Also we see that both models mistake certain pairs of cities for each other, but CityGuessr in some cases confuses them with other cities, which does not happen in case of our model.

Figure 12: Supplemental error analysis of VidTAG . (a) through (c) Breakdown of common model error types, highlighting dominant failure modes. (d) and (e) different scales of error: predictions are within the locality in vast majority of the cases, is in a different part of the city approximately 20% of the time. Region level errors happen in $<$ 2% of cases.

Figure 13: Visualization of samples for which our model correctly predicts location within 500 m (0.5 km).

Figure 14: Visualization of samples for which our model correctly predicts location within 1 km but not within 500m (0.5 km).

Figure 15: Visualization of samples for which our model correctly predicts location within 5 km but not within 1 km.

Figure 16: Visualization of samples for which our model correctly predicts location within 25km but not within 5km.

Figure 17: Visualization of samples for which our model cannot predict the location within 25 km.

Geospatial Representation Learning

Geospatial representation learning builds reusable embeddings of places, imagery, maps, mobility, and Earth data for spatial prediction and reasoning.

See all 133 papers

GeoGAT: Bidirectional Temporal Sampling Meets Hierarchical Graph Attention for Global Video Geo-localization
Sep 30, 2026Junchao Cui, Xuanzi Ma, Wenqi Shi +4Hierarchical Classification
Information Engineering University, Zhengzhou, China · Zhengzhou University, Zhengzhou, China
DualGeo: A Dual-View Framework for Worldwide Image Geo-localization
Apr 28, 2026Junchao Cui, Wenqi Shi, Shaoyong Du +4Cross-View Geo-Localization Geospatial Representation Learning
Henan Key Laboratory of Cyberspace Situation Awareness, Zhengzhou, China · Information Engineering University, Zhengzhou, China
GeoMetric: Injecting Metric Geographic Structure into Worldwide Image Geo-Localization
Jun 8, 2026Junchao Cui, Xuanzi Ma, Wenqi Shi +3Geospatial Representation Learning
Information Engineering University Zhengzhou, China

Sep 30, 2026J/K move · Enter open · S save

GeoGAT: Bidirectional Temporal Sampling Meets Hierarchical Graph Attention for Global Video Geo-localization

Junchao Cui, Xuanzi Ma, Wenqi Shi, Hangyu Li+3

Information Engineering University, Zhengzhou, China · Zhengzhou University, Zhengzhou, China

Hierarchical Classification

Open paper AskPDF arXiv

Global video geo-localization aims to infer the geographic location of a video worldwide, evaluating performance across four geographic hierarchies: city, state/province, country, and continent. Existing methods typically employ one-way uniform sampling to process video frames and train independent classifiers for each hierarchy, which leads to the loss of key geographic cues and prediction conflicts between hierarchies, especially for complex multi-shot edited videos. To address these limitations, we propose GeoGAT, which integrates bidirectional temporal sampling with graph attention networks (GATs). Specifically, GeoGAT extracts forward and offset-reversed frame sequences to construct complementary spatiotemporal features. These fused features are then fed into a predefined geographical hierarchy graph, where GATs perform structure-aware message passing, while a dual-constraint mechanism prunes predictions to eliminate cross-hierarchy conflicts. We construct GeoGAT10k, comprising 9,720 multi-shot edited videos from 166 cities worldwide, specifically to benchmark generalization ability on complex video structures. Experimental results on CityGuessr68k and GeoGAT10k demonstrate that GeoGAT eliminates hierarchical conflicts entirely and achieves state-of-the-art performance across all four geographic hierarchies. On CityGuessr68k, GeoGAT outperforms the strongest baseline, evaluated under both classification and retrieval protocols, by 2.6 percentage points at the city level. On the more challenging GeoGAT10k with multi-shot edited videos, the accuracy improvement exceeds 24 percentage points, validating strong generalization to complex real-world scenarios.

Abstract

Geospatial Representation Learning

Explore similar work

GeoGAT: Bidirectional Temporal Sampling Meets Hierarchical Graph Attention for Global Video Geo-localization

DualGeo: A Dual-View Framework for Worldwide Image Geo-localization

GeoMetric: Injecting Metric Geographic Structure into Worldwide Image Geo-Localization