Dark3R: Learning Structure from Motion in the Dark

Andrew Y Guo; Anagh Malik; SaiKiran Tedla; Yutong Dai; Yiqian Qin; Zach Salehe; Benjamin Attal; Sotiris Nousias

Figure 2: Existing hand-crafted and data-driven feature-matching pipelines such as SuperGlue [ 49 ] and MASt3R [ 33 ] perform reliably under well-illuminated conditions (top row) but performance significantly worsens when the image signal-to-noise ratio (SNR) drops to below –3 dB (bottom row). In contrast, Dark3R robustly identifies corresponding points in both imaging regimes. Green and red lines denote correspondences whose symmetric epipolar distance (SED) is below or above two pixels, respectively, for a randomly selected set of 20 putative matches. The average SED over all matches is also reported. We compute SED using calibrated camera intrinsics and the essential matrix predicted from MASt3R correspondences on the high-SNR image pair.

Figure 3: Method overview. (a) Dark3R is trained using paired clean and noisy raw images,

(\mathbf{I}_{\text{clean}}^{({1})},\mathbf{I}_{\text{clean}}^{({2})})

and

(\mathbf{I}_{\text{noisy}}^{({1})},\mathbf{I}_{\text{noisy}}^{({2})})

. The model is initialized from the weights of a pretrained MASt3R [ 33 ] network and adapted to low-light conditions using low-rank adaptation [ 25 ] . We fine-tune the encoder

\mathcal{\tilde{E}}

, decoder

\mathcal{\tilde{D}}

, and output head

\mathcal{\tilde{H}}

. We supervise training by minimizing the difference between MASt3R’s encoder features

\mathbf{F}_{\mathcal{E}}

, decoder features

\mathbf{F}_{\mathcal{D}}

, and correspondence map

\mathbf{C}

from the clean pair and Dark3R ’s predictions

\mathbf{\tilde{F}}_{\mathcal{E}}

,

\mathbf{\tilde{F}}_{\mathcal{D}}

, and

\mathbf{\tilde{C}}

on the noisy pair. (b) After training, the predicted poses and depth maps from Dark3R enable view synthesis in the dark via a coarse-to-fine optimization process. The rendered novel views are passed through an image signal processor (ISP) to produce the final sRGB outputs.

Figure 4: Pose prediction assessment. We compare Dark3R (blue) against MASt3R-SfM [ 16 ] (gray) across four pose and depth metrics: relative pose error in translation (RPE tran.), relative pose error in rotation (RPE rot.), absolute relative depth error (AbsRel), and the accuracy threshold

\delta<1.25

. Each curve shows mean performance with shaded regions indicating standard deviation across scenes. As image SNR decreases, Dark3R maintains lower pose and depth errors and higher reconstruction accuracy compared to MASt3R-SfM.

Figure 5: We compare point clouds and camera poses recovered by Dark3R with those estimated by MASt3R-SfM [ 16 ] on two low-light scenes. Dark3R produces more accurate geometry and camera trajectories that better align with a reference solution obtained by running COLMAP [ 51 ] on well-exposed images captured from the same viewpoints as the noisy inputs.

Figure 6: We compare novel view synthesis results using two pose estimation methods—Dark3R (D) and MASt3R-SfM (M-SfM) [ 16 ] —and two radiance field reconstruction methods—RawNeRF [ 41 ] (RN) and our proposed Dark3R-NeRF (DN). An oracle configuration consisting of Dark3R-NeRF using COLMAP [ 51 ] poses from well-exposed images serves as an upper bound. Our method produces more consistent novel views (insets) and maintains higher quality under increasing noise, as shown in the plots of peak signal-to-noise ratio (PSNR) and LPIPS versus SNR. Please see the project webpage for video results.

		Camera pose error			3D reconstruction
Method	Input	ATE $\downarrow$	RPE T $\downarrow$	RPE R $\downarrow$	AbsRel $\downarrow$	$\delta<1.25$ $\uparrow$
COLMAP [ 51 ]	sRGB	0.669	0.155	1.644	0.638	54.38
MASt3R [ 33 ]	raw	0.787	0.472	2.802	0.318	39.66
VGGT [ 61 ]	sRGB	0.252	0.216	1.047	0.232	63.28
MASt3R - SfM [ 16 ]	raw	0.088	0.038	0.201	0.196	79.39
Dark3R	raw	0.050	0.020	0.121	0.091	93.14

Table 1: Camera pose estimation results. using 120 sequential images (top rows) or the full set (330 to 500 images) from each scene (bottom rows). The test scenes have an average SNR of -3.87 dB. We report pose estimation errors after Sim(3) alignment with COLMAP poses computed on the longest exposure in the exposure bracket. We also provide 3D reconstruction metrics by running depth-supervised Nerfacto [ 57 ] with estimated camera poses paired with clean sRGB images to offer better intuition on how the pose errors relate to reconstruction quality. We test each pose estimation method on both sRGB and raw inputs and report the best result.

Method	Camera poses	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Dark3R-NeRF	MASt3R-SfM [ 16 ]	34.60	0.835	0.308
RawNeRF [ 41 ]	Dark3R	34.24	0.848	0.291
LE3D [ 28 ]	Dark3R	35.77	0.878	0.339
Dark3R-NeRF	Dark3R	36.17	0.866	0.257
Dark3R-NeRF	Oracle	37.16	0.882	0.228

Table 2: View synthesis results. We assess photometric quality resulting from use of different methods for view synthesis on low-SNR images and camera pose estimation. We also compare to an “oracle” that takes camera poses estimated from clean image data. Our approach for view synthesis Dark3R-NeRF outperforms other techniques when paired with Dark3R for pose estimation.

Condition	ATE $\downarrow$	RPE T $\downarrow$	RPE R $\downarrow$
1. w/o LoRA	0.476	0.074	0.512
2. fine-tune & test w/ sRGB	0.087	0.024	0.140
3. supervise w/ clean sRGB	0.064	0.020	0.122
4. only simulated data	0.078	0.032	0.181
5. only captured data	0.063	0.020	0.128
6. fine-tune $\mathcal{\tilde{E}}$ only	0.030	0.024	0.149

Table 3: Camera pose estimation ablation study . We ablate a number of design choices, including use of (1) LoRA vs. full fine-tuning, (2) using sRGB instead of raw images as input, or (3) supervising only with the former, (4–5) using only simulated or captured data, and (6–8) using different loss functions.

Condition	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
1. w/o depth supervision	34.91	0.858	0.307
2. w/ black level clipping	34.98	0.856	0.267
3. w/o stochastic preconditioning	36.05	0.857	0.262
proposed	36.17	0.866	0.257

Table 4: View synthesis ablation study. We assess the impact of depth supervision, stochastic preconditioning, and un-normalized raw inputs on view synthesis. The proposed approach (Dark3R-NeRF) performs the best in terms of photometric quality.

Scene	Num. Images	Scene	Num. Images	Scene	Num. Images	Scene	Num. Images
Bookstore 1	249	Furniture Store 9	248	Kitchen 1	172	Old College 5	492
Bookstore 2	196	Furniture Store 10	336	Lab 1	191	Old College 6	212
Bookstore 3	231	Furniture Store 11	225	Lecture Hall 1	270	Old College 7	251
Catering 1	184	Furniture Store 12	173	Library 1	409	Parking Lot 1	216
Classroom 1	242	Furniture Store 13	154	Library 2	230	Parking Lot 2	245
Classroom 2	302	Furniture Store 14	284	Library 3	235	Parking Lot 3	309

Table S1 : Dataset Statistics. Summary of the 92 scenes from our handheld-captured dataset used for simulated low-light finetuning. We break down the number of images along with a description per scene.

Figure S1 : Scenes included in the tripod-captured dataset. We capture 12 different scenes and use seven for training and five for evaluation.

Figure S2 : We visualize the different exposures included per scene in our dataset with shutter speeds indicated above each image. We capture nine different exposures for each scene across a wide range of SNRs. Images are contrast-stretched to facilitate visualization.

Figure S3 : We illustrate the result of applying the 2D denoiser LED [ 29 ] to captured images in our dataset. Inspecting the output of the denoiser shows that the appearance of image features is not multi-view consistent (see insets, arrows), which complicates feature matching used by conventional structure-from-motion pipelines.

Figure S4 : Pose prediction assessment for all evaluated scenes. We compare Dark3R (blue) against MASt3R-SfM [ 16 ] (gray) across four pose and depth metrics: relative pose error in translation (RPE tran.), relative pose error in rotation (RPE rot.), absolute relative depth error (AbsRel), and the accuracy threshold

\delta

<

1.25. Each curve shows mean performance with shaded regions indicating standard deviation across scenes. As image SNR decreases, Dark3R generally maintains lower pose and depth errors and higher reconstruction accuracy compared to MASt3R-SfM.

Figure S5 : Visualization of naive averaging on a captured scene. We average an increasing number of sequentially captured images together to perform denoising, but this comes at the cost of blurring the image. Hence, the dataset contains a non-trivial amount of disparity between the captured images, and the amount of noise makes it challenging to apply conventional burst denoising techniques that require image alignment.

Figure S6 : iPhone dataset. We test the generalization capabilities of our model by testing it on a multi-view raw image dataset captured by an iPhone 16. We capture this scene at four different exposure settings (including a reference, well-exposed capture), resulting in different signal-to-noise ratios. As shown in Table S6 , the Dark3R outperforms the baseline even when applied to this dataset.

Scene	Num Images	Environment	Split
Lounge	430	Indoors	Train
Chapel Table	420	Indoors	Train
Traincar	400	Outdoors	Train
Quad	400	Outdoors	Train
Bookshelf	361	Indoors	Train
Table	332	Indoors	Train

Table S2 : Dataset scenes. A summary of the scenes used in our dataset, detailing the number of images, environment type, and train/test splits. Each scene contains nine different exposures with ground truth camera poses.

Method	Input	ATE $\downarrow$	RPE T $\downarrow$	RPE R $\downarrow$
MASt3R - SfM [ 16 ]	raw	0.080	0.061	0.289
MASt3R - SfM [ 16 ]	denoised raw	0.048	0.048	0.238
Dark3R	raw	0.038	0.028	0.146

Table S3 : Camera pose estimation using a 2D denoiser. We first process raw noisy images using the 2D denoiser LED [ 29 ] before using those images as input for MASt3R-SfM [ 16 ] . While using a 2D denoiser improves performance slightly compared to the baseline of providing raw input images to MASt3R-SfM, it performs much worse than Dark3R, which is trained to operate directly on noisy raw images. These results were computed on the chapel scene using 120 images.

Method/Input			Camera pose error			3D reconstruction
Scene	Method	Input	ATE $\downarrow$	RPE T $\downarrow$	RPE R $\downarrow$	AbsRel $\downarrow$	$\delta<1.25$ $\uparrow$
Chapel	COLMAP [ 51 ]	sRGB	0.187	0.250	1.229	0.187	84.21
	MASt3R [ 33 ]	raw	0.832	0.614	2.877	0.234	60.56
	VGGT [ 61 ]	sRGB	0.405	0.451	2.197	0.212	59.24
	MASt3R - SfM [ 16 ]	raw	0.080	0.061	0.289	0.210	70.74
	Dark3R	raw	0.038	0.027	0.144	0.050	98.22

Table S4 : Camera pose estimation results by scene. We report results for 120 sequential images (top rows) or the full set (bottom rows) from each scene, grouped by scene name. Metrics include pose estimation errors after Sim(3) alignment and 3D reconstruction quality.

Scene	Method	Camera poses	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Chapel	LE3D [ 28 ]	Dark3R	38.01	0.868	0.398
	RawNeRF [ 41 ]	Dark3R	35.59	0.871	0.347
	Dark3R-NeRF	MASt3R-SfM [ 16 ]	36.48	0.826	0.356
	Dark3R-NeRF	Dark3R	37.44	0.840	0.310
	Dark3R-NeRF	Oracle	37.87	0.873	0.270
Statue	LE3D [ 28 ]	Dark3R	25.20	0.839	0.283

Table S5 : View synthesis results by scene. We assess photometric quality on low-SNR images. We compare to an “oracle” obtained by performing pose estimation on clean images.

Method	Input	Exposure (s)	ATE $\downarrow$	RPE T $\downarrow$	RPE R $\downarrow$
MASt3R - SfM [ 16 ]	raw	1/1000	0.105	0.062	0.238
Dark3R	raw	1/1000	0.043	0.045	0.172
MASt3R - SfM [ 16 ]	raw	1/2000	0.124	0.127	0.380
Dark3R	raw	1/2000	0.089	0.068	0.244
MASt3R - SfM [ 16 ]	raw	1/4000	2.484	1.174	6.278
Dark3R	raw	1/4000	0.227	0.178	0.575

Table S6 : Generalization to other cameras. We assess the ability of Dark3R to generalize to other types of cameras using images captured on an iPhone 16. We captured a series of low-light multi-view images on an iPhone 16 and tested Dark3R against MASt3R-SfM [ 16 ] . Despite not being finetuned with iPhone images, Dark3R still outperforms MASt3R-SfM for camera pose estimation accuracy.

Method	ATE $\downarrow$	RPE T $\downarrow$	RPE R $\downarrow$
M-SfM [16]	0.088	0.006	0.031
Dark3R	0.104	0.006	0.030

Table S7 : Clean image performance. We compare the camera pose error of Dark3R to MASt3R - SfM [ 16 ] . We evaluate on all 330–500 sRGB images from each scene at the longest exposure in our captured dataset. We omit the depth metrics shown in Table 1 because they used the M - SfM output on clean images as the reference.

		Camera pose error			3D reconstruction
Method	Intrinsics	ATE $\downarrow$	RPE T $\downarrow$	RPE R $\downarrow$	AbsRel $\downarrow$	$\delta<1.25$ $\uparrow$
Dark3R	unknown	0.043	0.019	0.121	0.090	93.63
Dark3R	known	0.050	0.020	0.121	0.091	93.14

Table S8 : Known vs. unknown intrinsics. Average camera pose error and 3D reconstruction metrics with unknown and known intrinsics evaluated on the captured dataset with 120 images per scene.

Dark3R: Learning Structure from Motion in the Dark

Abstract

Figures & tables

Explore similar work

Toward Robust and 3D-Aware RGB-NIR Imaging in the Dark

Noctif3R: Feed-Forward Monocular Real-Time SLAM for Photon-Limited Scenes on Embedded Hardware

Fast Low-light Enhancement and Deblurring for 3D Dark Scenes