Initialization-Free Bundle Adjustment Revisited: A Controlled Experimental Study

Simon Weber; Mateo de Mayo; Je Hyeong Hong; Carl Olsson; Daniel Cremers; Ronald Clark

cs.CVAug 18, 2026

Initialization-Free Bundle Adjustment Revisited: A Controlled Experimental Study

Authors: Simon Weber, Mateo de Mayo, Je Hyeong Hong, Carl Olsson, Daniel Cremers, Ronald Clark

Organizations: department of Computer Science at the University of Oxford, United Kingdom · Technical University of Munich and the Munich Center for Machine Learning, Germany · department of Electronic Engineering at Hanyang University, South Korea · Centre for Mathematical Sciences at Lund University, Sweden

Abstract

Initialization-free bundle adjustment (InitFree BA) aims to recover camera poses and scene structure directly from image observations, avoiding the geometric initialization stages of conventional structure-from-motion pipelines. Recent methods based on Object-Space Error (OSE) formulations and Variable Projection (VarPro) show encouraging optimization behavior from random camera configurations. However, existing evaluations primarily measure optimization success, leaving unclear whether a low OSE objective yields a valid metric 3D reconstruction. We revisit InitFree BA experimentally through a unified evaluation framework combining a C++ implementation of existing OSE formulations with a Blender-based dataset generator providing exact ground truth and controlled camera configurations and observation densities. Our experiments reveal a previously overlooked optimization--reconstruction gap: projective solutions with similarly low OSE values can lead to substantially different Euclidean reconstructions after metric upgrade. We identify initialization priors, landmark observation density, and metric-upgrade stability as key factors governing reconstruction success. Overall, our results suggest that the main challenge of InitFree BA is not merely minimizing OSE objectives, but obtaining projective reconstructions that admit reliable metric upgrade. We believe that the proposed benchmark, implementation, and analysis establish stronger experimental foundations for future research on initialization-free bundle adjustment, a problem largely unexplored within the computer vision community. Project page is available at https://github.com/simonwebertum/InitFreeBA.git.

Figures & tables

Panel of Figure 1 — Fig. 1 : Traditional BA and initialization-free BA solve different input problems. (a) Traditional BA minimizes reprojection error from image observations and an initial estimate of cameras and landmarks. It is a well-studied problem [ 1 ] . (b) In contrast, initialization-free BA starts only from image observations and must recover cameras and structure without a geometric initialization. This problem is recent [ 2 , 3 ] and still largely uncharted.

Panel of Figure 2 — Fig. 2 : Initialization-free BA can recover structure where randomly initialized traditional BA fails. (a) We apply a recent traditional BA solver [ 4 ] randomly initialized on our simple Set3 problem. It collapses to an invalid solution. InitFree BA, here with pOSE [ 3 ] , instead estimates (b) a projective reconstruction, and can then be upgraded (c) to a metric reconstruction.

Panel of Figure 3 — Fig. 3 : Initialization-free BA is highly sensitive to formulation and initialization, even on a controlled scene. Panels show (a), (b) Set2 and (c), (d) Set5 after optimization and linear metric upgrade, with the corresponding normalized OSE objective shown below each reconstruction. Although the OSE objective decreases rapidly and reaches close final errors for each corresponding pair, the resulting Euclidean reconstructions can differ dramatically. This illustrates that a decreasing OSE objective is not by itself a certificate of successful 3D reconstruction.

Fig. 4 : Metric upgrade failure rate as a function of the final OSE residual. A run is counted as failed when Rot. AUC@20 $<20$ , Trans. AUC@20 $<20$ , or landmark RMSE $>30$ . In contrast to an underlying assumption of previous works, a low OSE error does not necessarily ensure a valid reconstruction.

Fig. 5 : Overview of the dataset generator. Each row shows a rendered image, 2D observations (blue) and projected 3D landmarks (red), depth map, covisibility matrix, and reconstructed points and cameras. From top to bottom: Set 6; an artist-made realistic scene 4 4 4 Model courtesy of Aurélien Martel, licensed CC BY-NC 4.0, Sketchfab. ; and a photogrammetry scan of Skokloster Castle 5 5 5 Scan courtesy of Skoklosters slott, licensed CC BY 4.0, Sketchfab. . The examples illustrate that the generator supports both controlled and realistic scene geometries. Our generator is not restricted to InitFree BA problem, and is a reusable tool for controlled evaluation in SfM, SLAM, and related 3D reconstruction problems

Fig. 6 : The eight sequences used in our experiments. Sets 1–2 and 3–4 test the effect of observation sparsity on convergence, Set 5 is a real-world COLMAP reconstruction used to validate our controlled findings in a realistic setting, and Sets 6–8 test the effect of camera distribution and lookat targets on convergence. Bottom row: covisibility matrix for each sequence, giving an indication of the density of the Schur complement and of how well connected the cameras are (cyan: weakly covisible, magenta: highly covisible; normalized per matrix).

Formulation	Rot. A5	Rot. A20	Trans. A5	Trans. A20
pOSE	25.43	34.25	26.43	34.10
rOSE	24.33	31.99	28.82	37.56
RpOSE	9.28	15.24	12.29	19.85
expOSE	25.74	35.89	26.39	35.94
pOSE+rot	42.14	53.87	45.80	57.08

TABLE I : Average pose accuracy by formulation. Results are averaged over all datasets, initialization distributions, and non-robust/Cauchy variants after the linear metric upgrade. We report AUC 5° and AUC 20°, for rotation and translation. Higher is better.

Formulation	Lmk. Med. $\downarrow$	Lmk. RMSE $\downarrow$	Lmk. P90 $\downarrow$
pOSE	22.68	24.19	33.28
rOSE	20.52	21.82	29.92
RpOSE	27.93	29.58	40.66
expOSE	19.71	21.40	29.81
pOSE+rot	15.27	16.63	22.46

TABLE II : Average structure accuracy by formulation. Results are averaged over all datasets, initialization distributions, and non-robust/Cauchy variants after the linear metric upgrade. Lower is better.

Panel of Figure 9 — Fig. 7 : Effect of initialization. On Set8 with pOSE+rot, normal randomization collapses, whereas unit-circle initialization recovers a coherent reconstruction.

Initialization	Rot. A5	Rot. A20	Trans. A5	Trans. A20	Lmk. RMSE $\downarrow$
Normal	24.23	29.60	26.39	32.21	24.63
Uniform	23.47	30.54	25.71	33.04	24.27
Unit circle	28.45	42.60	31.74	45.47	19.27

TABLE III : Average effect of initialization. Results are averaged over all datasets, formulations, and non-robust/Cauchy variants after the linear metric upgrade. Higher is better for AUC; lower is better for landmark RMSE.

Initialization	Rot. A5	Rot. A20	Trans. A5	Trans. A20
Normal	25.01 / 23.46	30.18 / 29.01	26.29 / 26.49	31.64 / 32.79
Uniform	25.72 / 21.22	32.59 / 28.49	27.35 / 24.08	34.27 / 31.80
Unit circle	27.95 / 28.94	42.24 / 42.96	31.58 / 31.89	45.80 / 45.13

TABLE IV : Effect of initialization and Cauchy robustification on pose recovery. Results are averaged over all datasets and formulations after the linear metric upgrade. Each entry reports without Cauchy/with Cauchy, and the best value for each initialization is bolded.

Initialization	Lmk. Med. $\downarrow$	Lmk. RMSE $\downarrow$	Lmk. P90 $\downarrow$
Normal	22.66 / 23.19	24.50 / 24.76	33.96 / 34.17
Uniform	22.50 / 22.86	23.98 / 24.56	33.18 / 33.98
Unit circle	17.81 / 18.31	18.99 / 19.55	25.60 / 26.48

TABLE V : Effect of initialization and Cauchy robustification on structure accuracy. Results are averaged over all datasets and formulations after the linear metric upgrade. Each entry reports without Cauchy/with Cauchy, and the best value for each initialization is bolded. Lower is better.

Panel of Figure 13 — Fig. 8 : Effect of robustification. On Set2 with pOSE and normal initialization, the non-robust solver collapses, while the Cauchy loss recovers a valid metric reconstruction.

Dataset	Form.	Init.	Loss	Rot. A20	Trans. A20	Lmk. RMSE $\downarrow$
Set2	pOSE	Normal	None	0.4	3.9	40.66
			Cauchy	97.8	99.0	0.10
Set2	pOSE+rot	Uniform	None	2.5	9.9	40.66
			Cauchy	99.3	99.7	0.03
Set2	rOSE	Normal	None	3.7	7.0	40.66
			Cauchy	96.4	98.3	0.17

TABLE VI : Robustification can prevent catastrophic failures with linear metric upgrade. Each row compares the non-robust formulation with the Cauchy variant for the same dataset, formulation, and initialization.

Panel of Figure 15 — Fig. 9 : Effect of observation density. On Set2 with rOSE, filtering weakly observed landmarks stabilizes the metric upgrade and produces a more coherent reconstruction.

Loss	$\Delta$ Rot. A5	$\Delta$ Rot. A20	$\Delta$ Trans. A5	$\Delta$ Trans. A20	$\Delta$ Lmk. RMSE $\downarrow$
None	+6.31	+6.21	+6.46	+5.10	-2.63
Cauchy	+8.55	+10.08	+9.38	+9.54	-4.67

TABLE VII : Effect of densification. We report the change obtained by increasing the landmark visibility threshold from three to six observations, computed on the six complete common datasets: Set1–Set5 and Set8. Positive values are better for AUC metrics; negative values are better for landmark RMSE. Results use linear metric upgrade.

Form.	$\Delta$ Rot. A20	$\Delta$ Trans. A20	$\Delta$ Lmk. RMSE $\downarrow$
pOSE	+1.04	+2.35	-2.53
rOSE	+12.60	+12.01	-5.68
RpOSE	+18.34	+14.17	-5.15
expOSE	+1.99	+3.61	-1.37
pOSE+rot	+6.75	+4.46	-3.53

TABLE VIII : Model-wise effect of densification. We report the change induced by increasing the minimum landmark visibility from three to six observations, using the six complete common datasets. Results are averaged over initialization distributions and non-robust/Cauchy variants after linear metric upgrade.

Dataset	Form.	Init.	Loss	Rot. A20	Trans. A20	Lmk. RMSE $\downarrow$
Set4	expOSE	Normal	None	0.6	0.3	41.52
			Cauchy	100.0	100.0	0.00
Set2	RpOSE	Normal	None	1.3	2.2	41.00
			Cauchy	99.9	99.9	0.01
Set3	pOSE	Normal	None	0.9	3.8	41.19
			Cauchy	100.0	100.0	0.00

TABLE IX : Robustification can prevent catastrophic failures for minimum 6 obs. per landmark, with linear metric upgrade. Each row compares the non-robust formulation with the best robust variant for the same dataset, formulation, and initialization. Metrics are reported after the linear metric upgrade.

Stage	Rot. A20	Trans. A20	Lmk. RMSE $\downarrow$
Before metric upgrade	16.34	6.23	25.86
Linear metric upgrade	36.30	39.06	22.14
Mean change	+19.96	+32.83	-3.73

TABLE X : Effect of linear metric upgrade. We compare the projective solution before metric upgrade with the reconstruction obtained after the linear metric upgrade. Results are averaged over all datasets, formulations, initializations, and losses.

Fig. 10 : Metric upgrade failure rate as a function of the final refinement residual. We use the final mean residual after the refinement optimization and evaluate failure after linear metric upgrade with the same criterion as in Fig. 4 .

Panel of Figure 21 — Fig. 11 : Failure and recovery modes on BAL after linear metric upgrade. Each row varies one factor while keeping the scene and formulation fixed, for Trafalgar-150 (first row), and Venice-52 (other rows). The OSE residual decreases in both successful and failed cases, whereas the refinement residual more clearly separates successful metric reconstructions from collapsed ones.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Formulation	Pre-stage-2	Stage 2	Solver total	Wall-clock
pOSE	4.64	4.15	8.87	9.38
rOSE	4.79	7.35	10.36	10.93
RpOSE	15.39	3.79	19.43	19.98
expOSE	4.44	3.48	7.69	8.03
pOSE+rot	4.85	4.93	11.74	12.39

Appendix

TABLE XI : Median runtime decomposition. Values are seconds and are computed over all datasets, initialization distributions, and the non-robust/Cauchy variants. The pre-stage-2 column includes formulation-specific preprocessing and intermediate steps before reprojection refinement, including the RpOSE local radial refinement and the pOSE+rot pair/matrix construction.

Variant	Rot. A5	Rot. A20	Trans. A5	Trans. A20	Lmk. RMSE	Wall time
pOSE+rot	42.14	53.87	45.81	57.08	16.63	38.39
pOSE+rot est.	41.80	52.90	45.30	54.83	17.04	69.03

Appendix

TABLE XII : pOSE+rot with ground-truth and estimated relative rotations. Results use the linear metric upgrade and are averaged over all datasets, initialization distributions, and the non-robust/Cauchy variants. Higher is better for AUC; lower is better for landmark RMSE.

Fig. 12 : Metric upgrade failure rate as a function of the final OSE residual. We consider non-robust and Cauchy losses, followed by selected metric upgrade. A run is counted as failed when Rot. AUC@20 $<20$ , Trans. AUC@20 $<20$ , or landmark RMSE $>30$ .

Fig. 13 : Metric upgrade failure rate as a function of the final refinement residual. We use the final mean residual after the refinement optimization and evaluate failure after selected metric upgrade with the same criterion as in Fig. 12 .

Formulation	Rot. A5	Rot. A20	Trans. A5	Trans. A20
pOSE	24.89	34.15	26.73	35.64
rOSE	24.95	33.48	31.24	41.45
RpOSE	9.18	15.01	12.49	20.36
expOSE	25.08	35.04	26.68	37.28
pOSE+rot	41.99	54.16	46.08	57.63

Appendix

TABLE XIII : Average pose accuracy by formulation. Results are averaged over all datasets, initialization distributions, and non-robust/Cauchy variants after the selected linear/refined metric upgrade. Higher is better.

Formulation	Lmk. Med. $\downarrow$	Lmk. RMSE $\downarrow$	Lmk. P90 $\downarrow$
pOSE	21.06	22.80	31.03
rOSE	18.67	20.06	27.30
RpOSE	28.01	29.62	40.75
expOSE	18.39	20.37	28.14
pOSE+rot	15.34	16.67	22.44

Appendix

TABLE XIV : Average structure accuracy by formulation. Results are averaged over all datasets, initialization distributions, and non-robust/Cauchy variants after the selected linear/refined metric upgrade. Lower is better.

Initialization	Rot. A5	Rot. A20	Trans. A5	Trans. A20	Lmk. RMSE $\downarrow$
Normal	24.30	30.29	27.48	34.62	22.81
Uniform	23.30	30.38	26.57	34.68	23.81
Unit circle	28.06	42.43	31.89	46.10	19.09

Appendix

TABLE XV : Average effect of initialization. Results are averaged over all datasets, formulations, and non-robust/Cauchy variants after the selected linear/refined metric upgrade. Higher is better for AUC; lower is better for landmark RMSE.

Loss	Rot. A20	Trans. A20	Lmk. Med. $\downarrow$	Lmk. RMSE $\downarrow$	Lmk. P90 $\downarrow$
None	34.87	37.69	20.53	22.06	30.22
Cauchy	33.87	39.24	20.07	21.74	29.65

Appendix

TABLE XVI : Average effect of robustification. Results are averaged over all datasets, formulations, and initialization distributions after the selected linear/refined metric upgrade. Higher is better for AUC; lower is better for landmark errors.

		pOSE			rOSE			RpOSE			expOSE			pOSE+rot
Dataset	Initialization	A5	A10	A20	A5	A10	A20	A5	A10	A20	A5	A10	A20	A5	A10	A20
Set1	Normal	0.2/86.6/86.6%	1.7/93.3/93.3%	5.5/96.6/96.6%	0.0/76.8/76.8%	1.4/88.4/88.4%	4.5/94.2/94.2%	1.3/0.1/0.0%	4.3/1.9/1.2%	10.5/5.4/3.6%	0.2/71.2/71.2%	1.8/85.6/85.6%	5.2/92.8/92.8%	0.4/56.3/56.3%	2.5/78.1/78.1%	6.7/89.1/89.1%
	Uniform	0.9/83.1/83.1%	3.4/91.6/91.6%	8.7/95.8/95.8%	2.6/70.8/70.8%	6.7/85.4/85.4%	15.0/92.7/92.7%	0.8/1.7/1.7%	3.6/5.2/5.2%	10.3/12.6/12.6%	1.8/88.5/88.5%	5.1/94.2/94.2%	12.1/97.1/97.1%	2.0/90.8/90.8%	5.8/95.4/95.4%	14.9/97.7/97.7%
	Unit circle	81.8/6.2/6.2%	90.9/14.9/14.9%	95.5/34.4/34.4%	17.6/5.6/5.6%	50.1/13.5/13.5%	75.0/31.6/31.6%	86.3/0.0/0.0%	93.1/1.3/1.4%	96.6/3.9/4.1%	83.5/8.0/8.0%	91.8/19.4/19.4%	95.9/41.4/41.4%	32.2/18.5/18.5%	63.0/37.8/37.8%	81.5/64.6/64.6%
Set2	Normal	0.0/0.0/0.0%	0.1/0.0/0.0%	0.2/0.4/0.4%	0.4/0.0/0.0%	2.7/1.2/1.2%	9.0/3.7/3.7%	0.0/0.0/0.0%	0.4/0.4/0.4%	1.9/1.6/1.6%	0.1/0.0/0.0%	0.8/0.4/0.4%	2.7/1.4/1.4%	3.0/89.5/89.5%	7.2/94.7/94.7%	15.4/97.4/97.4%
	Uniform	0.0/0.0/0.0%	0.5/1.1/1.0%	2.5/3.3/3.2%	0.0/0.0/0.0%	0.4/0.7/0.7%	2.2/2.6/2.6%	0.0/0.0/0.0%	0.3/0.5/0.5%	1.8/1.5/1.5%	0.0/0.0/0.0%	0.2/1.2/1.2%	1.8/3.6/3.6%	0.1/0.0/0.0%	0.6/0.7/0.7%	2.3/2.5/2.3%

Appendix

TABLE XVII : Rotation AUC results without robust loss. Each entry reports before/linear/selected metric upgrade.

		pOSE			rOSE			RpOSE			expOSE			pOSE+rot
Dataset	Initialization	A5	A10	A20	A5	A10	A20	A5	A10	A20	A5	A10	A20	A5	A10	A20
Set1	Normal	0.7/0.6/0.0%	3.0/2.8/1.5%	10.1/7.0/4.5%	0.5/42.7/42.7%	2.5/69.0/69.0%	7.4/84.5/84.5%	0.8/0.0/0.0%	3.4/1.4/1.4%	11.2/4.2/4.2%	0.2/26.3/26.3%	1.7/48.9/48.9%	5.6/73.8/73.8%	0.1/79.3/79.3%	1.5/89.7/89.7%	4.6/94.8/94.8%
	Uniform	0.6/19.6/19.6%	3.2/39.3/39.3%	10.5/66.0/66.0%	3.0/64.4/44.7%	7.6/82.2/71.5%	16.4/91.1/85.7%	0.1/0.1/0.1%	0.8/0.9/0.9%	3.3/3.5/3.5%	0.6/35.7/2.8%	3.1/61.2/7.2%	10.1/80.6/16.7%	0.1/19.8/19.8%	1.9/37.8/37.8%	7.5/63.6/63.6%
	Unit circle	82.2/5.7/5.7%	91.1/13.7/13.7%	95.6/32.0/32.0%	17.6/5.9/5.9%	49.9/14.1/14.1%	75.0/32.8/32.8%	86.3/0.0/0.0%	93.1/1.3/1.3%	96.6/3.9/4.0%	83.5/7.2/7.2%	91.8/17.4/17.4%	95.9/38.5/38.5%	31.4/18.3/18.3%	62.3/37.5/37.5%	81.2/64.3/64.3%
Set2	Normal	0.1/91.4/91.4%	1.1/95.7/95.7%	3.9/97.8/97.8%	3.1/85.7/85.7%	9.7/92.9/92.9%	35.5/96.4/96.4%	0.0/0.0/0.0%	0.5/0.8/0.8%	2.3/3.0/3.0%	0.5/0.0/0.0%	2.7/1.2/1.2%	7.2/3.6/3.6%	0.1/0.0/0.0%	0.6/0.7/0.4%	2.4/2.4/1.9%
	Uniform	0.0/0.0/0.0%	0.5/1.1/1.2%	2.7/3.5/3.7%	0.0/0.0/0.0%	0.3/1.1/1.0%	2.3/3.3/3.2%	0.1/0.0/0.0%	0.8/0.8/0.8%	3.3/3.0/3.0%	0.0/0.1/0.0%	0.6/0.4/0.5%	2.6/1.9/2.2%	0.8/97.2/97.2%	3.3/98.6/98.6%	8.5/99.3/99.3%

Appendix

TABLE XVIII : Rotation AUC results with Cauchy loss. Each entry reports before/linear/selected metric upgrade.

		pOSE			rOSE			RpOSE			expOSE			pOSE+rot
Dataset	Initialization	T5	T10	T20	T5	T10	T20	T5	T10	T20	T5	T10	T20	T5	T10	T20
Set1	Normal	0.2/93.8/93.8%	0.5/96.9/96.9%	1.9/98.5/98.5%	0.2/89.3/89.3%	1.0/94.6/94.6%	4.1/97.3/97.3%	0.2/1.8/1.9%	1.1/3.3/3.6%	3.8/6.4/7.2%	1.5/86.6/86.6%	6.7/93.3/93.3%	19.5/96.7/96.7%	0.9/79.9/79.9%	4.0/89.9/89.9%	11.2/95.0/95.0%
	Uniform	0.4/92.2/92.2%	1.2/96.1/96.1%	3.3/98.1/98.1%	0.2/86.5/86.5%	0.5/93.3/93.3%	2.1/96.6/96.6%	0.6/12.1/12.1%	1.6/21.4/21.4%	3.6/36.6/36.6%	0.2/94.7/94.7%	0.8/97.3/97.3%	3.1/98.7/98.7%	0.3/95.7/95.7%	1.3/97.9/97.9%	4.1/98.9/98.9%
	Unit circle	1.0/26.9/26.9%	4.2/43.5/43.5%	10.7/66.8/66.8%	0.0/25.6/25.6%	0.2/41.5/41.5%	1.5/64.3/64.3%	2.0/2.0/1.8%	4.2/3.8/3.7%	8.0/7.5/7.3%	0.3/30.2/30.2%	1.5/48.6/48.6%	4.5/72.1/72.1%	1.4/43.8/43.8%	3.1/68.1/68.1%	6.6/84.0/84.0%
Set2	Normal	0.1/0.7/0.7%	0.2/1.7/1.7%	1.1/3.9/3.9%	0.1/1.6/1.6%	0.3/3.5/3.5%	0.7/7.0/7.0%	0.2/0.6/0.6%	0.4/2.2/2.2%	1.0/5.5/5.5%	0.2/0.2/0.2%	0.3/0.8/0.8%	1.1/2.8/2.8%	0.2/95.1/95.1%	0.8/97.6/97.6%	2.9/98.8/98.8%
	Uniform	0.2/2.2/2.0%	0.7/4.3/3.5%	1.7/8.3/6.6%	0.1/0.5/0.5%	0.1/2.6/2.6%	0.5/7.4/7.4%	0.1/0.6/0.6%	0.3/1.8/1.8%	1.0/4.9/4.9%	0.1/1.6/1.6%	0.3/3.5/3.5%	2.4/7.2/7.2%	0.6/2.1/1.5%	1.8/4.5/3.9%	5.9/9.9/8.9%

Appendix

TABLE XIX : Translation-direction AUC results without robust loss. Each entry reports before/linear/selected metric upgrade.

		pOSE			rOSE			RpOSE			expOSE			pOSE+rot
Dataset	Initialization	T5	T10	T20	T5	T10	T20	T5	T10	T20	T5	T10	T20	T5	T10	T20
Set1	Normal	0.1/2.7/2.2%	0.2/4.8/4.2%	1.3/9.3/8.4%	0.3/71.5/71.5%	1.0/85.8/85.8%	4.0/92.9/92.9%	0.1/1.9/1.9%	0.9/3.8/3.8%	3.4/7.6/7.6%	0.2/53.4/53.4%	0.7/76.2/76.2%	3.6/88.1/88.1%	0.1/90.5/90.5%	0.2/95.2/95.2%	1.0/97.6/97.6%
	Uniform	0.1/44.8/44.8%	0.5/69.2/69.2%	4.0/84.6/84.6%	3.9/83.5/76.3%	8.1/91.7/88.2%	15.3/95.9/94.1%	0.2/0.1/0.1%	0.3/0.5/0.5%	1.5/3.8/3.8%	0.2/63.9/33.2%	0.7/81.9/54.7%	3.0/91.0/77.1%	0.9/41.9/41.9%	3.5/66.1/66.1%	8.7/83.0/83.0%
	Unit circle	1.0/25.8/25.8%	4.1/41.8/41.8%	10.5/64.6/64.6%	0.0/26.2/26.2%	0.2/42.4/42.4%	1.8/65.4/65.4%	2.0/1.9/1.8%	4.2/3.7/3.7%	8.0/7.4/7.3%	0.3/28.8/28.8%	1.6/46.4/46.4%	4.5/70.0/70.0%	1.4/43.5/43.5%	3.0/67.8/67.8%	6.4/83.9/83.9%
Set2	Normal	0.3/96.0/96.0%	1.0/98.0/98.0%	3.4/99.0/99.0%	0.1/93.4/93.4%	0.3/96.7/96.7%	1.7/98.3/98.3%	1.0/1.7/1.7%	2.1/4.0/4.0%	4.7/7.7/7.7%	0.1/1.8/1.8%	0.4/3.6/3.6%	1.4/7.2/7.2%	0.1/1.5/0.5%	0.1/3.1/2.1%	1.1/5.8/6.6%
	Uniform	0.1/1.9/1.7%	0.3/3.7/3.4%	0.5/7.4/6.9%	0.1/1.4/1.7%	0.2/3.6/3.6%	0.6/7.5/7.4%	0.1/0.3/0.3%	0.5/1.2/1.2%	2.3/4.0/4.0%	0.4/1.2/0.5%	1.3/3.1/2.2%	3.5/7.0/6.0%	0.6/98.7/98.7%	1.9/99.3/99.3%	5.6/99.7/99.7%

Appendix

TABLE XX : Translation-direction AUC results with Cauchy loss. Each entry reports before/linear/selected metric upgrade.

		pOSE			rOSE			RpOSE			expOSE			pOSE+rot
Dataset	Initialization	Med	RMSE	P90	Med	RMSE	P90	Med	RMSE	P90	Med	RMSE	P90	Med	RMSE	P90
Set1	Normal	27.89/0.13/0.13	32.26/0.17/0.17	45.94/0.24/0.24	21.73/0.23/0.23	26.31/0.29/0.29	38.77/0.41/0.41	28.22/39.68/39.68	32.39/41.16/41.16	44.84/58.11/58.11	32.51/0.29/0.29	35.84/0.36/0.36	51.25/0.52/0.52	24.64/0.44/0.44	26.34/0.54/0.54	34.17/0.78/0.78
	Uniform	30.02/0.17/0.17	33.92/0.21/0.21	45.62/0.30/0.30	22.07/0.29/0.29	27.38/0.36/0.36	38.55/0.52/0.52	27.49/7.19/7.19	31.32/8.85/8.85	43.70/13.31/13.31	27.39/0.12/0.12	31.82/0.14/0.14	42.62/0.20/0.20	22.26/0.09/0.09	26.60/0.11/0.11	35.64/0.16/0.16
	Unit circle	0.94/3.87/3.87	1.13/4.32/4.32	1.56/6.24/6.24	2.99/4.20/4.20	3.98/4.70/4.70	5.89/6.78/6.78	3.01/39.68/39.68	5.26/41.16/41.16	8.38/58.11/58.11	2.10/3.17/3.17	2.43/3.54/3.54	3.55/5.11/5.11	2.49/1.60/1.60	2.72/1.82/1.82	3.96/2.65/2.65
Set2	Normal	39.10/39.12/39.12	40.65/40.66/40.66	58.19/58.19/58.19	19.03/39.12/39.12	21.59/40.66/40.66	31.34/58.19/58.19	38.23/39.12/39.12	39.25/40.66/40.66	54.92/58.19/58.19	39.12/39.12/39.12	40.66/40.66/40.66	58.20/58.19/58.19	14.85/0.10/0.10	17.19/0.13/0.13	23.09/0.18/0.18
	Uniform	38.76/39.12/39.12	40.03/40.66/40.66	58.12/58.19/58.20	38.42/39.12/39.12	39.77/40.66/40.66	57.30/58.19/58.19	39.03/39.12/39.12	40.50/40.66/40.66	57.94/58.19/58.19	39.12/39.12/39.12	40.66/40.66/40.66	58.19/58.19/58.19	38.84/39.12/39.12	40.28/40.66/40.66	57.99/58.19/58.19

Appendix

TABLE XXI : Landmark accuracy without robust loss. Each entry reports before/linear/selected metric upgrade; lower is better.

		pOSE			rOSE			RpOSE			expOSE			pOSE+rot
Dataset	Initialization	Med	RMSE	P90	Med	RMSE	P90	Med	RMSE	P90	Med	RMSE	P90	Med	RMSE	P90
Set1	Normal	19.52/39.68/39.67	19.52/41.15/41.16	26.30/58.11/58.11	25.98/0.64/0.64	29.38/0.77/0.77	39.68/1.12/1.12	20.65/39.68/39.68	25.58/41.16/41.16	37.85/58.11/58.11	29.72/1.14/1.14	32.48/1.32/1.32	44.53/1.93/1.93	32.04/0.21/0.21	33.38/0.26/0.26	43.63/0.37/0.37
	Uniform	20.88/1.51/1.51	20.57/1.73/1.73	28.12/2.52/2.52	14.07/0.36/0.52	16.27/0.44/0.65	22.23/0.64/0.92	33.31/39.68/39.68	36.94/41.16/41.16	53.32/58.11/58.11	17.66/0.81/2.24	19.39/0.98/2.73	25.88/1.42/3.94	18.99/1.62/1.62	21.18/1.90/1.90	27.47/2.78/2.78
	Unit circle	0.94/4.14/4.14	1.13/4.63/4.63	1.55/6.69/6.69	3.00/4.05/4.05	3.99/4.53/4.53	5.90/6.54/6.54	3.01/39.68/39.68	5.26/41.16/41.16	8.38/58.11/58.11	2.12/3.43/3.43	2.44/3.84/3.84	3.57/5.53/5.53	2.53/1.61/1.61	2.77/1.83/1.83	4.04/2.68/2.68
Set2	Normal	27.78/0.09/0.09	31.38/0.10/0.10	44.39/0.15/0.15	6.38/0.14/0.14	8.07/0.17/0.17	12.05/0.25/0.25	36.04/39.12/39.12	37.71/40.66/40.66	52.40/58.18/58.18	26.61/39.12/39.12	27.55/40.66/40.66	34.47/58.19/58.19	38.66/39.12/39.12	39.99/40.66/40.66	57.89/58.19/58.19
	Uniform	38.61/39.12/39.12	39.59/40.66/40.66	54.64/58.19/58.20	38.62/39.12/39.12	39.90/40.66/40.66	57.95/58.19/58.19	36.73/39.12/39.12	38.51/40.66/40.66	54.20/58.19/58.19	38.94/39.12/39.12	40.46/40.66/40.66	57.60/58.19/58.19	30.97/0.03/0.03	33.76/0.03/0.03	44.19/0.05/0.05

Appendix

TABLE XXII : Landmark accuracy with Cauchy loss. Each entry reports before/linear/selected metric upgrade; lower is better.

		pOSE			rOSE			RpOSE			expOSE			pOSE+rot
Dataset	Initialization	A5	A10	A20	A5	A10	A20	A5	A10	A20	A5	A10	A20	A5	A10	A20
Set1	Normal	0.0/0.4%	1.5/2.4%	5.6/6.4%	0.3/43.4%	2.2/69.4%	6.9/84.7%	2.5/95.0%	7.0/97.5%	21.0/98.7%	0.7/33.0%	3.3/57.5%	10.3/78.7%	1.5/63.8%	5.7/81.9%	13.7/90.9%
	Uniform	2.1/14.9%	8.7/30.2%	25.4/53.4%	3.0/74.2%	7.9/87.1%	18.2/93.6%	0.4/92.1%	2.5/96.1%	7.2/98.0%	1.1/58.8%	3.8/79.4%	9.2/89.7%	0.9/22.2%	3.6/41.6%	9.1/67.6%
	Unit circle	82.2/8.4%	91.1/20.4%	95.6/42.9%	17.4/7.7%	49.7/18.6%	74.8/40.4%	86.2/0.0%	93.1/1.5%	96.5/4.3%	83.1/8.7%	91.5/21.2%	95.8/44.0%	30.9/21.3%	61.9/42.1%	80.9/68.6%
Set2	Normal	0.0/0.0%	0.7/1.2%	3.0/3.5%	5.7/98.2%	14.8/99.1%	39.6/99.6%	0.8/99.4%	3.5/99.7%	9.0/99.9%	1.5/0.0%	4.7/1.5%	11.4/4.4%	1.7/84.9%	5.1/92.5%	12.3/96.2%
	Uniform	1.4/85.7%	4.7/92.9%	14.0/96.4%	0.7/76.5%	3.0/88.2%	8.1/94.1%	0.0/0.0%	0.4/0.2%	2.6/1.3%	1.6/78.9%	5.0/89.5%	11.7/94.7%	1.4/85.5%	4.5/92.8%	11.1/96.4%

Appendix

TABLE XXIII : Rotation AUC with minimum 6 observations per landmark and Cauchy loss. Each entry reports before/linear metric upgrade.

		pOSE			rOSE			RpOSE			expOSE			pOSE+rot
Dataset	Initialization	A5	A10	A20	A5	A10	A20	A5	A10	A20	A5	A10	A20	A5	A10	A20
Set1	Normal	0.6/67.4%	3.0/83.7%	7.8/91.8%	1.0/78.7%	3.6/89.3%	9.0/94.7%	1.4/95.2%	4.4/97.6%	10.6/98.8%	1.1/81.8%	3.7/90.9%	9.3/95.5%	1.4/57.9%	4.5/78.9%	14.1/89.5%
	Uniform	1.3/74.9%	4.1/87.5%	10.0/93.7%	2.9/65.9%	7.6/83.0%	17.8/91.5%	1.3/41.0%	4.4/67.2%	10.7/83.6%	2.6/70.0%	7.1/85.0%	16.7/92.5%	1.1/6.9%	3.9/16.9%	9.7/33.8%
	Unit circle	81.8/8.9%	90.9/21.7%	95.5/44.7%	17.4/8.0%	49.8/19.5%	74.9/41.7%	86.2/0.0%	93.1/1.3%	96.6/3.9%	83.1/8.7%	91.5/21.3%	95.8/44.1%	31.7/17.7%	62.6/36.6%	81.3/63.4%
Set2	Normal	2.1/92.7%	6.0/96.4%	14.1/98.2%	4.5/98.5%	11.8/99.3%	31.7/99.6%	0.1/0.0%	0.7/0.2%	2.8/1.3%	4.0/96.1%	9.9/98.1%	21.8/99.0%	2.2/54.3%	6.2/77.0%	14.7/88.5%
	Uniform	0.0/0.0%	1.1/1.1%	3.7/3.4%	1.5/95.1%	4.9/97.6%	11.5/98.8%	0.0/0.0%	0.9/0.7%	3.0/2.6%	2.7/88.1%	6.9/94.0%	16.0/97.0%	1.7/94.1%	5.1/97.1%	12.2/98.5%

Appendix

TABLE XXIV : Rotation AUC with minimum 6 observations per landmark without robust loss. Each entry reports before/linear metric upgrade. Note that Set6 and Set7 are too sparse for supporting this densification.

		pOSE			rOSE			RpOSE			expOSE			pOSE+rot
Dataset	Initialization	A5	A10	A20	A5	A10	A20	A5	A10	A20	A5	A10	A20	A5	A10	A20
Set1	Normal	0.1/6.9%	0.1/11.4%	0.6/18.3%	0.3/71.5%	1.6/85.8%	6.8/92.9%	0.1/97.7%	0.5/98.8%	3.7/99.4%	0.1/60.4%	0.5/80.2%	1.7/90.1%	0.6/83.2%	2.4/91.6%	6.9/95.8%
	Uniform	0.1/34.9%	0.3/56.4%	0.9/78.0%	0.1/88.0%	0.4/94.0%	1.6/97.0%	0.1/96.4%	0.4/98.2%	1.8/99.1%	0.2/80.9%	1.0/90.4%	5.4/95.2%	5.5/45.4%	11.0/69.8%	19.7/84.9%
	Unit circle	1.0/31.1%	4.1/49.8%	10.7/73.2%	0.0/29.8%	0.2/47.9%	1.6/71.4%	2.0/1.9%	4.2/3.8%	8.1/7.5%	0.3/31.6%	1.6/50.6%	4.6/73.8%	1.4/47.4%	2.9/71.6%	6.4/85.8%
Set2	Normal	0.1/2.0%	0.3/3.8%	1.2/7.5%	0.0/99.2%	0.0/99.6%	0.5/99.8%	0.2/99.7%	0.5/99.9%	1.5/99.9%	0.2/1.5%	0.3/3.4%	0.5/7.3%	0.6/93.0%	1.6/96.5%	3.7/98.3%
	Uniform	0.1/93.4%	0.4/96.7%	1.3/98.4%	0.3/89.1%	0.9/94.6%	3.3/97.3%	0.2/0.3%	0.5/1.0%	1.8/3.0%	1.1/90.2%	3.9/95.1%	11.2/97.6%	0.3/93.3%	1.1/96.6%	3.4/98.3%

Appendix

TABLE XXV : Translation-direction AUC with minimum 6 observations per landmark with Cauchy loss. Each entry reports before/linear metric upgrade.

		pOSE			rOSE			RpOSE			expOSE			pOSE+rot
Dataset	Initialization	A5	A10	A20	A5	A10	A20	A5	A10	A20	A5	A10	A20	A5	A10	A20
Set1	Normal	0.3/84.9%	1.3/92.4%	4.1/96.2%	2.2/90.1%	5.3/95.1%	13.2/97.5%	0.1/97.8%	1.6/98.9%	5.1/99.4%	0.3/91.6%	1.3/95.8%	3.7/97.9%	0.1/80.5%	0.3/90.2%	1.2/95.1%
	Uniform	0.1/88.4%	0.3/94.2%	0.9/97.1%	0.2/84.2%	0.6/92.1%	2.1/96.0%	0.4/69.5%	1.5/84.8%	4.3/92.4%	0.3/86.1%	0.6/93.1%	1.8/96.5%	4.3/24.4%	11.2/39.7%	22.6/61.7%
	Unit circle	1.0/32.0%	4.3/51.2%	10.8/74.3%	0.0/30.4%	0.1/48.9%	1.3/72.3%	2.0/2.0%	4.3/3.8%	8.1/7.4%	0.4/31.6%	1.6/50.7%	4.6/73.9%	1.4/42.7%	3.0/66.9%	6.6/83.5%
Set2	Normal	0.8/96.7%	2.3/98.3%	5.8/99.2%	0.4/99.3%	1.4/99.7%	6.6/99.8%	0.2/0.2%	0.3/0.7%	0.9/2.2%	0.3/98.2%	1.1/99.1%	4.7/99.6%	0.7/78.7%	1.9/89.3%	4.5/94.7%
	Uniform	0.6/1.6%	1.7/3.4%	4.0/6.7%	0.1/97.7%	0.5/98.9%	3.0/99.4%	0.2/1.6%	0.8/4.2%	2.0/9.6%	0.1/94.5%	0.5/97.3%	1.7/98.6%	0.7/97.3%	1.8/98.6%	4.0/99.3%

Appendix

TABLE XXVI : Translation-direction AUC with minimum 6 observations per landmark without robust loss. Each entry reports before/linear metric upgrade.

Bundle Adjustment

Bundle adjustment jointly refines camera poses and 3D structure by minimizing image reprojection errors in multi-view vision.

See all 21 papers

Stable and Scalable Bundle Adjustment of Holistic 3D Structures
Sep 3, 2026Shaohui Liu, Rémi Pautrat, Daniel Barath +3Multi-View Geometry Bundle Adjustment
ETH Zurich · Microsoft Spatial AI Lab · Australian National University +1
BLASt3R: Bundle Adjustment of Any Image Set with Multi-View Matching and Monocular Priors
Sep 4, 2026Vincent Leroy, Philippe Weinzaepfel, Lojze Zust +2Structure from Motion Bundle Adjustment
NAVER LABS Europe
CSS-BA: Gate-Guided Column Space Search for Bundle Adjustment
Jul 17, 2026Ayano Kaneda, Takafumi Taketomi, Shugo Yamaguchi +1Multi-View Geometry Bundle Adjustment
Waseda University, Tokyo, Japan · CyberAgent, Inc., Tokyo, Japan

Sep 3, 2026J/K move · Enter open · S save

Stable and Scalable Bundle Adjustment of Holistic 3D Structures

Shaohui Liu, Rémi Pautrat, Daniel Barath, Richard Hartley+2

ETH Zurich · Microsoft Spatial AI Lab · Australian National University+1

Multi-View Geometry Bundle Adjustment 3D Reconstruction

Open paper AskPDF arXiv

Bundle Adjustment (BA) is a cornerstone of 3D computer vision and has benefited from decades of advances in sparse optimization and numerical methods. It was originally developed for jointly optimizing camera intrinsics, poses and sparse 3D points. While extensions incorporate lines and other primitives, integrating richer geometric structures such as parallelism, coplanarity, or wireframes often introduces significantly increased computational cost and reduced numerical stability. In this paper, we propose a unified framework that extends bundle adjustment to jointly optimize geometric features and higher-order relations. We first introduce a taxonomy that distinguishes scalable geometric features with direct 2D measurements (e.g., points and lines), from groups encoding higher-order relations (e.g., coplanarity, parallelism, etc.), where we show that groups can be modeled as camera-like entities within the bundle adjustment framework. Building on this formulation, we propose that both group constraints and cross-feature relations (i.e., point-line associations) can be expressed through 2D reprojection measurements. By formulating group-induced and cross-feature reprojection errors, we preserve the sparsity structure of classical point-based BA under Schur elimination, while avoiding direct 3D regularization that degrades the conditioning and stability. Experiments on both real-world and synthetic datasets demonstrate runtime performance comparable to classical point-only bundle adjustment, while producing significantly richer 3D structures and improved geometric accuracy.

Abstract

Bundle Adjustment

Explore similar work

Stable and Scalable Bundle Adjustment of Holistic 3D Structures

BLASt3R: Bundle Adjustment of Any Image Set with Multi-View Matching and Monocular Priors

CSS-BA: Gate-Guided Column Space Search for Bundle Adjustment