PIVOT: A Multi-Trajectory Dataset and Testbed for Pose, Intrinsics, and Novel Viewpoint Evaluation in Real-World 3D Reconstruction

Mary Raymond

cs.CVAug 26, 2026

PIVOT: A Multi-Trajectory Dataset and Testbed for Pose, Intrinsics, and Novel Viewpoint Evaluation in Real-World 3D Reconstruction

Authors: Mary Raymond

Organizations: Independent Researcher

Abstract

Neural radiance fields (NeRFs), 3D Gaussian Splatting (3DGS), and related novel-view synthesis methods are commonly evaluated under capture and reconstruction conditions cleaner than those encountered by robots, drones, and autonomous systems. Benchmarks often rely on reconstruction-friendly trajectories, optimized camera poses and intrinsics, and held-out views sampled from trajectories represented during training. These assumptions can obscure performance with measured poses, reusable camera calibration, and structurally different camera paths. We introduce PIVOT (Pose, Intrinsics and Viewpoint Oriented Testbed), a multi-trajectory dataset, processing pipeline, and evaluation framework for independently studying these factors. PIVOT captures each scene using diverse camera trajectories and retains, where available, both sensor-derived measured poses and COLMAP-optimized poses, together with calibrated and optimized camera intrinsics. It defines three benchmark families: (1) seen versus unseen trajectory novel-view generalization, (2) measured versus optimized pose sensitivity, and (3) calibrated versus optimized intrinsics sensitivity. We also introduce a directed pose-space Chamfer distance to quantify how well training poses cover an evaluation trajectory. PIVOT v1 contains five real-world scenes captured with a DJI Mini 4 Pro and provides an open processing and Nerfstudio-based evaluation toolchain. Benchmark results show a consistent quality gap between held-out views on represented trajectories and unseen trajectories, as well as substantial sensitivity to pose source and camera intrinsics.

Figures & tables

Fig. 1: Overview of the PIVOT testbed and reconstruction pipeline. (a) PIVOT ’s interactive trajectory-aware viewer jointly visualizes the COLMAP sparse reconstruction, camera trajectories, trajectory-level pose errors, and scene statistics. (b) Example Splatfacto reconstruction of the same scene, trained using 80% of the frames from all trajectories with COLMAP-optimized poses and intrinsics. PIVOT preserves trajectory identity together with measured and optimized camera parameters to support controlled reconstruction evaluation.

Fig. 2: PIVOT raw-data processing pipeline.

Scene	Frames	Registered	Reg. (%)	Sparse pts.	AABB (m)	Reproj. (px)
Church	1,612	1,538	95.4	882,387	28.68	1.06
Village Street	1,733	1,726	99.5	1,430,217	55.12	0.89
Victorian Garden	1,547	1,536	99.2	767,781	44.62	0.92
Frontyard	920	913	99.2	719,123	13.16	1.09
Backyard	1,536	1,527	99.4	110,1253	25.3	0.97

TABLE I: PIVOT v1 scene summary. Frontyard and Backyard statistics are pending completion of the currently running processing/benchmark jobs and will be populated before release.

Trajectory	Motion	Altitude	Camera direction
orbit_inward_low	orbit	low	scene inward
orbit_inward_mid	orbit	mid	scene inward
orbit_inward_high	orbit	high	scene inward
orbit_outward_low	orbit	low	scene outward
traversal_forward_low	traversal	low	along track
traversal_backward_low	traversal	low	along track

TABLE II: Representative core trajectory types in PIVOT.

Scene	Model	Eval. Type	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$
church	Nerfacto	Seen avg.	0.44	0.53	19.53
	Nerfacto	Unseen avg.	0.31	0.70	16.47
	Splatfacto	Seen avg.	0.65	0.30	22.16
	Splatfacto	Unseen avg.	0.40	0.57	16.86
victorian_garden	Nerfacto	Seen avg.	0.33	0.62	17.00
	Nerfacto	Unseen avg.	0.29	0.71	15.15

TABLE III: BM1 quantitative results comparing seen and unseen evaluation trajectories. Higher SSIM and PSNR are better (

\uparrow

), while lower LPIPS is better (

\downarrow

Panel of Figure 6 — Fig. 3: Relationship between directed normalized pose Chamfer distance to the training trajectories and reconstruction quality (LPIPS) across the PIVOT scenes. Left: Nerfacto. Right: Splatfacto. Seen trajectories (blue) correspond to held-out views from trajectories represented during training, while unseen (orange) trajectories correspond to independently captured evaluation trajectories. Lower LPIPS indicates better reconstruction quality.

Scene	Model	Pose Type (TR)	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	$\Delta$ PSNR vs. OO
church	Nerfacto	OO	0.44	0.46	20.25	0.00
		OM	0.20	0.79	13.42	-6.82
		MO	0.24	0.73	15.82	-4.43
		MM	0.20	0.80	13.55	-6.69
	Splatfacto	OO	0.63	0.29	22.13	0.00
		OM	0.25	0.60	14.64	-7.48

TABLE IV: BM2 quantitative results for optimized and measured camera poses.

Scene	Model	Camera Calibration Type	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	$\Delta$ PSNR vs. Optimized
church	Nerfacto	COLMAP optimized	0.44	0.45	20.26	0.00
	Nerfacto	OpenCV calibrated	0.24	0.74	15.78	-4.47
	Splatfacto	COLMAP optimized	0.63	0.29	22.13	0.00
	Splatfacto	OpenCV calibrated	0.29	0.54	16.42	-5.71
victorian_garden	Nerfacto	COLMAP optimized	0.22	0.60	16.46	0.00
	Nerfacto	OpenCV calibrated	0.11	0.84	14.71	-1.74

TABLE V: BM3 quantitative results comparing COLMAP-optimized and OpenCV-calibrated camera intrinsics.

Panel of Figure 9 — Fig. 4: Qualitative BM1: Seen vs unseen reconstruction results for representative seen and unseen evaluation trajectories. Seen examples are sampled from a training trajectory, while unseen examples are sampled from an independently captured trajectory. Ground-truth images are shown alongside Nerfacto and Splatfacto reconstructions.

Panel of Figure 10 — Fig. 5: Qualitative BM2:Measured vs. Optimized Poses reconstruction results for two representative scenes under different combinations of optimized and measured camera pose components. OO denotes optimized translation and optimized rotation, MO denotes measured translation and optimized rotation, OM denotes optimized translation and measured rotation, and MM denotes measured translation and measured rotation. Ground-truth images are shown for reference.

Panel of Figure 11 — Fig. 6: Qualitative BM3 reconstruction results for two representative scenes using COLMAP-optimized and independently calibrated camera intrinsics. Ground-truth (GT) images are shown for reference. The COLMAP condition uses scene-optimized camera intrinsics, while the calibrated condition uses the fixed OpenCV camera calibration.

Explore similar work

May 18, 2026

Can These Views Be One Scene? Evaluating Multiview 3D Consistency when 3D Foundation Models Hallucinate

Multiview 3D evaluation assumes that the images being scored are observations of one static 3D scene. This assumption can fail in NVS and sparse-view reconstruction: inputs or generated outputs may contain artifacts, outlier frames, repeated views, or noise, yet still receive high 3D consistency scores. Existing reference-based metrics require ground truth, while ground-truth-free metrics such as MEt3R depend on learned reconstruction backbones whose failure modes are poorly characterized. We study this reliability problem by comparing neural reconstruction priors with classical geometric verification. We introduce \benchmark, a controlled robustness benchmark for multiview 3D consistency, and a parametric family that decomposes neural metrics into backbone, residual, and aggregation components. This family recovers MEt3R and yields variants up to

3\times

more robust. Our analysis shows that VGGT, MASt3R, DUSt3R, and Fast3R can hallucinate dense geometry and cross-view support for unrelated scenes, repeated images, and random noise. We introduce COLMAP-based metrics that use matches, registration, dense support, and reconstruction failure as failure-aware consistency signals. On real NVS outputs and a structured human study, these metrics achieve up to

4\times

higher correlation with human judgments than MEt3R.

Soumava Paul, Prakhar Kaushik, Alan Yuille

CCVL, Johns Hopkins University

3D Foundation Models Multi-View Consistency Multi-View Geometry 3D Reconstruction 3D Representation Learning

Ask PDF

Sep 1, 2026

TAPVid-MV: A Benchmark for Tracking Any Point in 3D Across Multiple Views

Multi-camera systems are increasingly practical for robotics, AR/VR, and autonomous driving because complementary views reduce depth ambiguity and preserve visibility under occlusion. Existing point-tracking benchmarks, however, focus on a single video or static multi-camera rigs. None test long-term 3D point tracking across several synchronized views under camera motion. We introduce TAPVid-MV (Tracking Any Point in Video across Multiple Views), the first benchmark for this setting. It contains a curated set of 284 sequences, 1,142 calibrated camera streams, and 109,769 point tracks across seven subsets spanning indoor and outdoor domains, from robotics and human activity to driving and synthetic procedural scenes. We obtain these trajectories using dataset-specific auxiliary modalities: sensor depth, LiDAR, SLAM and SfM points, human meshes, posed object meshes, and simulation. Every sequence and trajectory is visually verified by human annotators. Across more than 30 baselines, no method comes close to solving the task. Surprisingly, existing multi-view point trackers do not consistently outperform monocular point trackers. By evaluating reconstruction and point tracking on the same datasets, TAPVid-MV helps distinguish errors in recovered geometry from errors in point correspondence. Through this joint analysis, we identify geometry recovery as a major bottleneck for accurate 3D point tracking. Beyond multi-view 3D point tracking, our released annotations support monocular 2D and 3D point tracking, future-trajectory prediction, and 4D reconstruction.

Skanda Koppula, Frano Rajic, Abdullah Faiz Ur Rahman +9

1Google DeepMind · University College London · 3ETH Zürich +1

Point Tracking 3D Point Tracking Multi-View 3D Reconstruction

Ask PDF

Apr 15, 2026

DF3DV-1K: A Large-Scale Dataset and Benchmark for Distractor-Free Novel View Synthesis

Advances in radiance fields have enabled photorealistic novel view synthesis. In several domains, large-scale real-world datasets have been developed to support comprehensive benchmarking and to facilitate progress beyond scene-specific reconstruction. However, for distractor-free radiance fields, a large-scale dataset with clean and cluttered images per scene remains lacking, limiting the development. To address this gap, we introduce DF3DV-1K, a large-scale real-world dataset comprising 1,048 scenes, each providing clean and cluttered image sets for benchmarking. In total, the dataset contains 89,924 images captured using consumer cameras to mimic casual capture, spanning 128 distractor types and 161 scene themes across indoor and outdoor environments. A curated subset of 41 scenes, DF3DV-41, is systematically designed to evaluate the robustness of distractor-free radiance field methods under challenging scenarios. Using DF3DV-1K, we benchmark nine recent distractor-free radiance field methods and 3D Gaussian Splatting, identifying the most robust methods and the most challenging scenarios. Beyond benchmarking, we demonstrate an application of DF3DV-1K by fine-tuning a diffusion-based 2D enhancer to improve radiance field methods, achieving average improvements of 0.96 dB PSNR and 0.057 LPIPS on the held-out set (e.g., DF3DV-41) and the On-the-go dataset. We hope DF3DV-1K facilitates the development of distractor-free vision and promotes progress beyond scene-specific approaches. The dataset and leaderboard are available at https://johnnylu305.github.io/df3dv1k_web/.

Cheng-You Lu, Yi-Shan Hung, Wei-Ling Chi +6

University of Technology Sydney · University of Sydney · National Yang Ming Chiao Tung University

Novel View Synthesis Neural Radiance Fields

Ask PDF