Triangulation-Free Bundle Adjustment with Graduated Non-Convexity for Camera Pose Refinement from Coarse Priors

Nikolaos Kyriazis

cs.CVAug 21, 2026

Triangulation-Free Bundle Adjustment with Graduated Non-Convexity for Camera Pose Refinement from Coarse Priors

Authors: Nikolaos Kyriazis

Abstract

Mobile AR frameworks attach a metric pose prior to every casual phone capture, and turning it into reconstruction-grade poses cheaply on CPU is the step before novel-view synthesis. The least a refiner owes an accurate prior is not to make it worse. The workhorse refiner does. On 15 ScanNet++ iPhone room captures, COLMAP triangulation plus prior-seeded bundle adjustment degrades an accurate ARKit prior in all 15, 0.55 degrees to 0.74 degrees by scene-mean. The cause is the seeding. Structure is triangulated from the prior before anything is optimized, so the prior's error is baked into the structure the optimizer trusts. We remove the triangulation. Every keypoint owns a scalar depth along its own back-projected ray and each match contributes two symmetric cross-projection residuals, so structure is re-expressed at every iterate. The same solve holds the room prior at 0.57 degrees and never fails in 330 perturbed room runs, and at object scale reaches 0.265 degrees/1.80 mm from a prior at 0.456 degrees in a median of 10 s per scene on one CPU, against 2.5 GPU-hours for a learned refiner. Because no structure is committed, the objective also admits graduated non-convexity, which measures how deep the defect goes. Classical refinement collapses past 1-2 degrees of prior error, barely beyond a real ARKit prior, and no classical refinement arm survives 32 degrees. Ours recovers 425 of 425 runs through 16 degrees/80 mm and 85% at 32 degrees/160 mm, and perturbed rooms through 32 degrees. Nominal object-scale accuracy is on par rather than better, on a benchmark at its own noise floor, where classical bundle adjustment is a strong baseline absent from the literature. One scene fails for every solver already at zero perturbation. Re-mapping from position priors matches us in the prior's frame but discards it, so it cannot exploit a prior worth keeping or be warm-started.

Figures & tables

Figure 1: The claim at a glance. (a) Classical refinement decides matches once, at the prior. Triangulation stamps its verdict where the prior’s rays cross and culls the matches that disagree, among them the ones pointing at the ghosted truth, so iterating returns the same wrong state. Ours never hard-commits. Every match keeps a soft vote along its own ray, and annealing the loss scale makes the consensus more pronounced, discarding nothing, until poses and associations settle on the truth together. (b) Success rate against prior perturbation on the 17 MobileBrick scenes whose matches admit the true solution, five seeds per scene (castle is the failure case, Section 4.11 ; protocol in Section 4.8 ). Classical prior-seeded BA collapses beyond $1$ – $2^{\circ}$ ; ours with GNC recovers every run through $16^{\circ}$ and 85% at $32^{\circ}$ . The dotted line is the mean real ARKit prior error, $0.46^{\circ}$ . (c) 3D Gaussian Splatting trained on each solver’s result from the identical $16^{\circ}$ perturbed prior (aston, seed 0), rendered at the same held-out view.

	All 18 scenes		Excl. castle
Method	Rot.	Tr.	Rot.	Tr.
ARKit prior [ 2 ]	0.46	1.90	—	—
BARF [ 39 ]	0.57	2.50	—	—
L2G-NeRF [ 12 ]	0.58	2.84	—	—
SPARF [ 58 ]	0.61	3.39	—	—
PoRF [ 4 ]	0.22	1.23	—	—

Table 1: Pose accuracy on the MobileBrick evaluation set, mean over scenes, lower is better. Baseline numbers from PoRF [ 4 ] , Table 3 (the BARF, L2G-NeRF, and SPARF rows are PoRF’s re-runs under its protocol). The right-hand block drops castle, the one scene whose matches prefer a wrong optimum for every filter-free solver tested (Section 4.11 ), and it is where the trust region stops mattering. Differences at or below the

{\sim}0.5^{\circ}

/2 mm ground-truth accuracy are not rankable. Rotation in degrees, translation in millimetres.

	Classical tri.+BA				Ours, staged GNC
Pert.	med. ∘	mm	${<}1^{\circ}$	$\times\tfrac{1}{2}$	med. ∘	mm	${<}1^{\circ}$	$\times\tfrac{1}{2}$
$1^{\circ}$	1.96	56	6	2	0.52	16	24	18
$2^{\circ}$	2.47	66	0	0	0.52	16	24	25
$4^{\circ}$	3.87	70	0	0	0.52	16	24	30
$8^{\circ}$	7.88	93	0	0	0.52	16	59	75
$16^{\circ}$	15.86	142	0	0	0.54	16	57	75

Table 2: Room-scale basin on 15 ScanNet++ scenes, two seeds per level (30 runs per level and solver), i.i.d. perturbations, protocol of Section 4.8 with self-calibration for our arm (staged after a poses-only annealed pass) and the OPENCV model for the classical control. “med.” columns are the median final rotation (deg) and translation (mm), “

{<}1^{\circ}

” counts runs below

1^{\circ}

absolute error (the misses are the kitchen, lab and apartment whose nominal solves already sit at or above

1^{\circ}

at every seed, joined marginally by a copy room from

8^{\circ}

), “

\times\tfrac{1}{2}

” counts runs that at least halve the prior’s rotation error. Our arm carries two seeds below

8^{\circ}

and five at

8^{\circ}

and above, so its counts are out of 30 and out of 75 respectively; the classical control carries two seeds throughout, 30 runs per level. Each perturbation level pairs the stated rotation with 5 mm per degree.

Method	Hardware	Min	Med.	Mean	Max
PoRF [ 4 ]	A40 GPU	2.5 h §
SPARF [ 58 ] ‡	A100 GPU	${\sim}$ 10 h
COLMAP BA from prior	1 CPU	9	53	55	139
Ours (full)	1 CPU	9	101	141	580
Ours (cap 100)	1 CPU	1	10	11	36
GLOMAP, prior-free	1 CPU	19	90	94	275

Table 3: Per-scene runtime on the MobileBrick evaluation scenes, in seconds unless marked. Baseline times are the published figures [ 4 , 58 ] ; CPU times cover the solve only (matching excluded) on a single CPU (32 threads), GPU times the full inference or alignment.

Pose source	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
ARKit prior	16.55	0.554	0.389
Ours (full)	17.48	0.659	0.337
COLMAP BA from prior	17.71	0.667	0.331
Ground truth	18.29	0.702	0.314

Table 4: Downstream 3D Gaussian splatting on held-out views, mean over the 18 evaluation scenes. Identical training configuration per pose source; poses held fixed.

Variant	Rot. ( ∘ ) $\downarrow$	Trans. (mm) $\downarrow$	Time
Per-obs. (full, $\lambda_{R}{=}10^{3}$ )	0.246	1.61	101 s
Per-obs. (full, $\lambda_{R}{=}0$ )	0.245	1.66	98 s
Anchored tracks ( $\lambda_{R}{=}0$ )	0.277	1.82	25 s
Per-obs., cap 100 ( $\lambda_{R}{=}0$ )	0.265	1.80	10 s

Table 5: Ablation on the MobileBrick evaluation set (18 scenes), mean pose error and median solve time.

Figure 2: Why triangulation narrows the basin. (a) Measured cost along the pose interpolation from the accurate prior (

\alpha{=}0

) to a

16^{\circ}

/80 mm perturbation (

\alpha{=}1

) on aston, each curve min–max normalized (only shape and argmin are comparable). Classical BA with structure triangulated at the perturbed prior and held fixed has its minimum at the perturbation. The same objective with oracle structure, and our structure-marginalized objective at every arctan scale, have theirs at the accurate prior, with larger scales retaining gradient along the whole path (the annealed landscapes GNC descends). (b) The causal control. Identical COLMAP BA from identical perturbed poses, swapping only the triangulation input (seed 0). Oracle structure recovers through

16^{\circ}

where prior structure collapses from

2^{\circ}

; both fail at

32^{\circ}

, where GNC still recovers (Figure 3 ).

Perturbation	Succ.	Med.	Succ.	Med.	Succ.	Med.
	COLMAP BA		Ours, plain		Ours, GNC
$1^{\circ}$	76	0.27	85	0.23	85	0.22
$2^{\circ}$	57	0.87	84	0.23	85	0.22
$4^{\circ}$	0	3.20	85	0.22	85	0.22
$8^{\circ}$	0	7.61	83	0.24	85	0.22
$16^{\circ}$	0	15.85	77	0.24	85	0.22

Table 6: Basin of convergence on all 18 evaluation scenes, five noise seeds per level, i.i.d. per-frame perturbations, trust region disabled, cap 100. A run succeeds when its final mean rotation error falls below

1^{\circ}

. “Succ.” counts successes out of the 90 runs at that level, “med.” is the median final mean rotation error in degrees, where a value tracking the perturbation level means the solver returned the prior it was given and

{\sim}0.22^{\circ}

is nominal accuracy. Castle fails at every level for both of our arms for reasons unrelated to the basin (Section 4.11 ), so 85 is their ceiling.

Prior terms (6 scenes, $n=30$ )
Arm	$1^{\circ}$	$2^{\circ}$	$4^{\circ}$	$8^{\circ}$	$16^{\circ}$	$32^{\circ}$	$64^{\circ}$
GNC, default	30	30	30	30	30	30	4
GNC, $\lambda_{p}=0$	30	30	30	30	30	30	3
Noise structure (6 scenes, $n=30$ )
GNC, i.i.d.	30	30	30	30	30	30	4
GNC, walk	30	30	30	30	30	30	19

Table 7: Controls on the basin result, successes per perturbation level out of

n

runs per level (scenes

\times

five seeds). The prior block asks whether the prior terms drive the recovery, the noise block replaces i.i.d. per-frame noise with a correlated random walk of equal mean magnitude, the schedule block varies the GNC annealing, and the parameterization block swaps per-observation depths for anchored-track depths. Each block is matched, so its arms run on the same scenes and seeds.

Figure 3: Basin of convergence for three solvers started from identical perturbed priors, 18 scenes with 5 noise seeds per level (630 runs per solver; trust region disabled). (a) Fraction of runs recovering below

1^{\circ}

final rotation error, with Wilson 95% bands. The dotted ceiling (85/90) is castle, whose repeated-texture failure is nominal, not basin-induced (Section 4.11 ). Classical COLMAP BA (red) collapses to zero from

4^{\circ}

/20 mm, since triangulation from a coarse prior leaves nothing to optimize; the triangulation-free solve (orange) degrades stochastically from

2^{\circ}

and materially from

16^{\circ}

; GNC (teal) recovers every non-castle run through

16^{\circ}

/80 mm and 72/90 at

32^{\circ}

/160 mm (85% of the non-castle runs). (b) Median final rotation error with interquartile band (log–log); the dashed line means returning the perturbed prior unchanged. Where recovery happens, it is to nominal accuracy (

{\sim}0.22^{\circ}

	$1^{\circ}$	$8^{\circ}$	$16^{\circ}$	$32^{\circ}$
Absolute frame (no alignment)
Prior, unrefined	1.09/5	8.01/40	16.00/80	32.00/160
Ann. re-tri.	0.99/32	7.95/253	16.04/297	28.97/324
Prior-pos. map.	0.20 / 1	1.10 / 6	2.20/ 11	4.45/22
Ours (GNC)	0.28/3	1.10 /6	2.17 /12	4.42 / 22
Sim(3)-aligned, same runs (rotation only)

Table 8: Pose error in the prior’s own metric frame, with no alignment of any kind, over the six control scenes and five noise seeds, 120 cells per arm (median over runs; rotation in degrees, translation in millimetres; the figures quoted in the text are these same medians). Sim(3)-aligned rotation for the same runs is given for contrast. Perturbation levels pair each rotation with 5 mm per degree. Classical refinement recovers shape and loses the frame; re-mapping is anchored but its shape ignores the prior entirely; in-place refinement and re-mapping are indistinguishable in the absolute frame at every coarse level. Bold marks the lower unrounded median, so two entries that print equal can still be bolded apart.

Drift over 178 chunks
	Rot. ( ∘ )	Trans. (cm)
10 s chunks, median	0.31	2.8
60 s chunks, median	0.53	17
60 s chunks, 90th pct.	1.1	38
Plain solve on the seven 60 s CAB chunks
	Prior	Refined	$\Delta$

Table 9: Real ARKit drift on LaMAR validation captures (CAB, HGE, LIN) and what refinement does with it. The upper block is the per-chunk mean error after anchoring each chunk on its first ten frames of ground truth. The lower block is the plain solve on the seven 60 s CAB chunks re-sampled from the raw 10 Hz streams, reported as the median over frames and, where a group is given, the median over its chunks.

(a) Solutions on castle
System	Rot. ( ∘ )	Tr. (mm)	Coh.	Axis dot
ARKit prior	0.555	2.42	—	—
COLMAP BA from prior	0.682	3.58	—	—
Ours, plain	4.905	20.69	0.955	1
Ours, GNC	4.893	20.73	—	—
GLOMAP, prior-free	4.848	20.83	0.954	0.99997

Table 10: The castle failure. (a) Where each system lands on castle. Axis coherence is the norm of the mean unit rotation-error axis across frames, where 1 means a perfectly shared axis, and the last column is the dot product of that mean axis with ours under identical Sim(3) alignment. Everything below the rule converges to the same

5^{\circ}

twist, with or without matching, with or without the prior. (b) Interventions that leave the twist in place.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

	Ours (full)			COLMAP BA from prior
Scene	Rot.	Trans.	Time	Rot.	Trans.	Time
aston	0.130	1.03	171	0.118	0.97	71
audi	0.326	1.53	9	0.279	1.39	10
beetles	0.174	1.28	34	0.127	0.98	24
big_ben	0.236	2.57	580	0.230	2.64	139
boat	0.191	1.23	425	0.176	1.03	106

Appendix

Table 11: Per-scene results on the 18 MobileBrick evaluation scenes. Mean rotation error ( ∘ ), mean translation error (mm), and solve time (s). “Ours (full)” scores castle as the ARKit prior via the trust-region fallback (raw solve 4.905 ∘ /20.69 mm); the COLMAP control uses no fallback.

Scene	ARKit	Ours	COLMAP BA	GT
aston	18.05	20.27	20.38	21.42
audi	14.55	14.16	14.23	14.13
beetles	14.09	14.79	14.90	14.75
big_ben	13.87	16.18	16.49	17.02
boat	14.58	16.47	16.51	16.92
bridge	18.12	18.11	17.77	18.24

Appendix

Table 12: Per-scene held-out PSNR (dB) of 3D Gaussian splatting per pose source, identical training configuration throughout. Higher is better. Castle’s “Ours” entry scores the returned prior (trust-region fallback).

Bundle Adjustment

Bundle adjustment jointly refines camera poses and 3D structure by minimizing image reprojection errors in multi-view vision.

See all 21 papers

Aug 18, 2026

Initialization-Free Bundle Adjustment Revisited: A Controlled Experimental Study

Initialization-free bundle adjustment (InitFree BA) aims to recover camera poses and scene structure directly from image observations, avoiding the geometric initialization stages of conventional structure-from-motion pipelines. Recent methods based on Object-Space Error (OSE) formulations and Variable Projection (VarPro) show encouraging optimization behavior from random camera configurations. However, existing evaluations primarily measure optimization success, leaving unclear whether a low OSE objective yields a valid metric 3D reconstruction. We revisit InitFree BA experimentally through a unified evaluation framework combining a C++ implementation of existing OSE formulations with a Blender-based dataset generator providing exact ground truth and controlled camera configurations and observation densities. Our experiments reveal a previously overlooked optimization--reconstruction gap: projective solutions with similarly low OSE values can lead to substantially different Euclidean reconstructions after metric upgrade. We identify initialization priors, landmark observation density, and metric-upgrade stability as key factors governing reconstruction success. Overall, our results suggest that the main challenge of InitFree BA is not merely minimizing OSE objectives, but obtaining projective reconstructions that admit reliable metric upgrade. We believe that the proposed benchmark, implementation, and analysis establish stronger experimental foundations for future research on initialization-free bundle adjustment, a problem largely unexplored within the computer vision community. Project page is available at https://github.com/simonwebertum/InitFreeBA.git.

Simon Weber, Mateo de Mayo, Je Hyeong Hong +3

department of Computer Science at the University of Oxford, United Kingdom · Technical University of Munich and the Munich Center for Machine Learning, Germany · department of Electronic Engineering at Hanyang University, South Korea +1

Bundle Adjustment 3D Reconstruction Multi-View Geometry Camera Pose Estimation

Ask PDF

Jul 17, 2026

CSS-BA: Gate-Guided Column Space Search for Bundle Adjustment

Bundle adjustment (BA) remains a critical refinement module for image-based 3D reconstruction and continues to improve geometric accuracy even in learning-based pipelines. However, in low-parallax and near-rotational regimes, classical Schur-based Levenberg--Marquardt (LM) often becomes ill-conditioned and yields unreliable pose and calibration estimates. We propose Gate-Guided CSS-BA, a solver-side modification of Schur-LM that preserves the classical BA objective and trust-region framework while constraining each update to a geometrically informed low-dimensional subspace. By integrating Column Space Search (CSS) with geometry-aware gating, the method stabilizes the Schur-LM update without altering the estimation problem. In contrast to keyframe or state-selection approaches, all camera and point parameters remain in the optimization problem; only the update direction is restricted. The method serves as a drop-in replacement for existing BA pipelines. Experiments on both generic and challenging weak-geometry scenarios show more stable optimization, improved relative pose accuracy, and competitive calibration behavior while maintaining reprojection quality.

Ayano Kaneda, Takafumi Taketomi, Shugo Yamaguchi +1

Waseda University, Tokyo, Japan · CyberAgent, Inc., Tokyo, Japan

Multi-View Geometry Bundle Adjustment Multi-View 3D Reconstruction Camera Pose Estimation

Ask PDF

Sep 3, 2026

Stable and Scalable Bundle Adjustment of Holistic 3D Structures

Bundle Adjustment (BA) is a cornerstone of 3D computer vision and has benefited from decades of advances in sparse optimization and numerical methods. It was originally developed for jointly optimizing camera intrinsics, poses and sparse 3D points. While extensions incorporate lines and other primitives, integrating richer geometric structures such as parallelism, coplanarity, or wireframes often introduces significantly increased computational cost and reduced numerical stability. In this paper, we propose a unified framework that extends bundle adjustment to jointly optimize geometric features and higher-order relations. We first introduce a taxonomy that distinguishes scalable geometric features with direct 2D measurements (e.g., points and lines), from groups encoding higher-order relations (e.g., coplanarity, parallelism, etc.), where we show that groups can be modeled as camera-like entities within the bundle adjustment framework. Building on this formulation, we propose that both group constraints and cross-feature relations (i.e., point-line associations) can be expressed through 2D reprojection measurements. By formulating group-induced and cross-feature reprojection errors, we preserve the sparsity structure of classical point-based BA under Schur elimination, while avoiding direct 3D regularization that degrades the conditioning and stability. Experiments on both real-world and synthetic datasets demonstrate runtime performance comparable to classical point-only bundle adjustment, while producing significantly richer 3D structures and improved geometric accuracy.

Shaohui Liu, Rémi Pautrat, Daniel Barath +3

ETH Zurich · Microsoft Spatial AI Lab · Australian National University +1

Multi-View Geometry Bundle Adjustment 3D Reconstruction

Ask PDF

Abstract

Bundle Adjustment

Explore similar work

Initialization-Free Bundle Adjustment Revisited: A Controlled Experimental Study

CSS-BA: Gate-Guided Column Space Search for Bundle Adjustment

Stable and Scalable Bundle Adjustment of Holistic 3D Structures