cs.CVNov 8, 2026
SaveMartine: Benchmarking Multi-View 3D Surface Reconstruction Across Viewpoint Coverage, Resolution, and Lighting
Organizations: University of Zurich, Switzerland · IRIT, Université de Toulouse, CNRS, Toulouse, France · Balgrist University Hospital, University of Zurich, Switzerland · Independent researcher
Abstract
Controlled acquisition setups are widely used for high-accuracy multi-view 3D surface reconstruction, as they allow sensor configuration, image resolution, viewpoint distribution, and illumination to be specified. Yet benchmarking remains difficult, since reconstruction accuracy depends jointly on the reconstruction paradigm, object properties, and acquisition regime. Existing benchmarks often restrict viewpoint coverage, resolution, or illumination, which can bias comparisons towards methods suited to the chosen setting. Others lack accurate 3D ground truth or quantified image-to-geometry alignment, limiting their use for metric surface evaluation. We introduce Martine, a benchmark for controlled 3D object surface reconstruction. Martine comprises 28 objects, including transparent materials and fine structures, captured at 9568 x 6376 pixels from 84 views over a spherical cap under both single-light and multi-light regimes. It provides object-independent camera calibration, accurate 3D ground-truth scans, calibration and image-to-geometry accuracy quantified on test images, and 3D property masks for analysing errors by material properties, visibility, curvature and fine structures. Experiments across photogrammetry, classical multi-view stereo, neural implicit surfaces, Gaussian-splatting-based reconstruction, and multi-view photometric stereo show that Martine enables regime-aware evaluation with explicit control over reference-data accuracy. The benchmark resolves sub-millimetric reconstruction differences, with a best observed Chamfer distance of 0.058 mm over the evaluated visible surface.
Figures & tables
Figure 1: Examples of image–GT misalignment in OpenSubstance [ 20 ] . Inaccurate COLMAP reconstructions of an opaque flower (left) and a transparent glass (right) in red. Registered GT scan meshes in green.
Figure 2: Dataset overview. (a) All objects in the proposed dataset, showing high geometric and optical diversity, including low-texture regions, fine structures, transparent and specular materials, and high-frequency geometric details. (b) Dense viewpoint sampling with wide angular coverage, obtained by coupling a robotic arm with a turntable. (c) Two examples of ground-truth meshes with manually curated challenge-level 3D masks highlighting specular regions (top, in blue) and transparent regions (bottom, in red) for challenge-level evaluation.
Figure 3: Acquisition setup. (a) Single-light acquisition regime, using an ambient illumination setup designed to be rotation-invariant around the turntable axis. (b) Multi-light acquisition regime, using a rig of 30 LEDs rigidly mounted around the camera. At each viewpoint, the LEDs are activated sequentially while the camera pose remains fixed.
| rock | vase | hammer head | candle | can | glass | petanque ball | jam pot |
| 1.06 | 1.08 | 1.02 | 0.78 | 1.21 | 1.14 | 0.84 | 1.26 |
Table 2: Leave-one-out registration residuals (mean reprojection error in pixels, over 8 control views per object.
Figure 4: Global precision, recall, and F-score curves pooled across all evaluated objects at quarter resolution (d4). Each curve aggregates all point-to-surface distances before computing the metric, so larger objects contribute proportionally more.
Figure 5: F-score at 1 mm on the vase object versus number of input views on a log scale (top) and image resolution (bottom). Crosses plotted at 0% F-score mark failed reconstructions in the 5-view and one-eighth-resolution settings.
Figure 6: Qualitative comparison of accuracy error heatmaps (viridis colour map, 0–4 mm, larger errors saturated) at quarter resolution (d4).
Figure 7: Radar chart of F-score at 0.5 mm per reconstruction challenge, at quarter resolution, for the five representative methods and Neuralangelo. Higher is better.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| Calibration | #corners | Mean | Median | RMS | IQR | P95 | Max |
|---|---|---|---|---|---|---|---|
| Textured cloth (ours) | 6 480 | 0.542 | 0.524 | 0.584 | 0.298 | 0.931 | 1.489 |
| ArUco cylinder [ 20 ] | 6 480 | 0.666 | 0.542 | 0.824 | 0.555 | 1.621 | 4.010 |
| COLMAP | 6 480 | 0.809 | 0.799 | 0.913 | 0.612 | 1.559 | 2.917 |
| ChArUco board [ 21 ] | 6 480 | 1.086 | 0.924 | 1.312 | 0.865 | 2.541 | 5.199 |
Table 3: Distribution of the per-corner reprojection residual on the test checkerboard on the 20 frontal views, in pixels at the full resolution.
Figure 8: Calibration setup comparison. All configurations include a test checkerboard used only for evaluation. (a) Our approach uses SfM on the textured black cloth. The inset shows a brightness-enhanced crop of the cloth micro-texture, which provides point correspondences at the ultra-high image resolution. (b) ChArUco board [ 39 ] . (c) ArUco cylinder on the turntable base.
Figure 9: IoU limitations in assessing boundary precision. Visualisation of the discrepancy between SAM (magenta) and ground-truth (cyan) masks, with their intersection in orange. The boundary analysis reveals systematic contour shifts (4.80 and 4.51 px on average) that are critical for 2D–3D registration accuracy.
| SAM | SAM + Canny (Ours) | ||||
|---|---|---|---|---|---|
| Object | IoU | Dist. (px) | IoU | Dist. (px) | |
| rock | 4.80 | 1.28 | |||
| warrior head | 4.51 | 1.37 | |||
Table 4: Contour refinement. The proposed Canny-based refinement significantly reduces the mean contour distance (px).
Figure 10: Sample of OpenSubstance objects that are unsuitable for rigorous 3D reconstruction analysis.
Figure 11: (Left) A visualisation of a single isophote (green) on the calibration sphere. (Right) Distribution of pixel intensity variations for each isophote across the sphere’s height. The standard deviation ranges from 0.0054 to 0.0267, with an average of 0.0141, confirming consistent illumination across viewpoints.
Figure 12: Light-calibration spheres. (Left) Matte sphere. (Right) Glossy sphere.
Figure 13: Colour calibration board seen in the single-light regime (left), and in the multi-light regime with one LED turned on (right).
Figure 14: LED rig. Two views showing the spatial distribution of the LEDs around the camera.
| Method | Family |
|---|---|
| Meshroom [ 3 ] | Photogrammetry |
| RealityScan [ 2 ] | Photogrammetry |
| COLMAP [ 4 , 22 ] | Classical MVS |
| PGSR [ 36 ] | 3D Gaussian Splatting |
| NeuS2 [ 37 ] | Neural implicit |
| Neuralangelo [ 7 ] | Neural implicit |
Table 5: Evaluated methods.
| RealityScan Meshroom COLMAP PGSR Object d1 d2 d4 d8 d4 d2 d4 d8 d2 d4 d8 rock 0.108 / 99.1 0.106 / 99.1 0.148 / 98.4 0.261 / 90.4 0.108 / 98.2 0.102 / 98.2 0.115 / 97.5 0.156 / 94.8 0.340 / 87.4 0.508 / 68.7 0.774 / 12.7 vase 0.135 / 95.9 0.256 / 77.4 0.638 / 41.3 1.135 / 19.1 0.309 / 82.5 0.243 / 79.3 0.267 / 73.7 – 0.577 / 50.0 0.625 / 49.0 – hammer head 0.088 / 98.9 0.104 / 99.0 0.174 / 97.8 0.459 / 65.3 0.102 / 98.4 0.065 / 99.2 0.110 / 97.5 0.865 / 8.0 0.161 / 98.1 0.282 / 95.6 0.566 / 29.9 candle 0.821 / 43.6 1.106 / 33.3 1.362 / 21.8 2.004 / 7.2 1.546 / 16.5 0.755 / 39.2 0.895 / 25.5 – 0.486 / 87.8 0.587 / 62.7 – can 0.149 / 96.9 0.184 / 96.2 0.276 / 90.1 0.672 / 44.1 0.364 / 83.0 0.250 / 85.7 0.254 / 89.1 0.639 / 43.9 0.500 / 67.0 0.585 / 57.9 0.726 / 42.9 glass 1.525 / 20.7 1.456 / 17.7 1.759 / 7.9 1.999 / 4.7 2.535 / 0.3 1.224 / 24.8 1.345 / 21.8 1.415 / 12.9 1.289 / 26.6 1.287 / 25.6 1.356 / 20.5 petanque ball 0.158 / 99.0 0.210 / 97.2 0.378 / 79.2 0.993 / 9.1 0.232 / 94.4 0.145 / 97.5 0.361 / 63.9 1.422 / 0.2 0.294 / 97.1 0.375 / 81.3 0.599 / 24.5 jam pot 0.964 / 39.8 1.018 / 36.8 1.141 / 33.1 1.373 / 25.1 1.551 / 19.2 0.973 / 33.9 0.910 / 40.0 1.021 / 30.9 0.973 / 21.4 1.254 / 12.8 1.669 / 4.7 colander 0.408 / 69.8 0.415 / 70.0 0.731 / 38.0 0.929 / 30.5 0.460 / 65.1 0.622 / 54.5 0.551 / 61.5 0.620 / 53.2 0.394 / 73.2 0.368 / 75.8 0.387 / 72.5 wood 0.121 / 97.4 0.166 / 94.9 0.277 / 85.6 0.463 / 67.7 0.227 / 89.4 0.171 / 92.8 0.243 / 88.7 0.359 / 79.6 0.302 / 83.7 0.318 / 83.2 0.401 / 73.4 plate 0.080 / 99.2 0.077 / 99.4 0.069 / 99.3 0.149 / 95.6 0.088 / 99.4 0.104 / 98.3 0.114 / 98.0 0.147 / 92.5 0.148 / 92.2 0.340 / 80.2 0.648 / 67.8 paw pad 0.316 / 81.5 0.514 / 68.7 1.384 / 19.2 – 1.928 / 3.9 0.991 / 3.8 0.886 / 12.0 – 0.824 / 48.8 0.905 / 42.9 1.231 / 16.3 moka pot 0.127 / 95.8 0.158 / 93.2 0.258 / 79.0 0.504 / 58.4 0.231 / 88.3 0.272 / 84.7 0.327 / 80.5 0.590 / 60.4 0.749 / 47.4 0.779 / 44.5 0.953 / 17.4 lego 0.463 / 67.1 0.428 / 67.9 0.407 / 68.1 0.496 / 60.3 0.479 / 64.1 0.499 / 67.0 0.452 / 69.4 0.674 / 42.4 0.334 / 78.0 0.406 / 75.7 0.539 / 58.1 surgery scissors 0.318 / 84.3 0.473 / 60.2 1.000 / 22.6 – – 0.270 / 88.9 0.502 / 55.1 – 0.163 / 92.5 0.209 / 92.6 – knife 0.131 / 99.5 0.165 / 97.9 0.262 / 92.2 0.562 / 50.6 0.142 / 97.2 0.089 / 99.2 0.112 / 98.5 0.508 / 64.1 0.204 / 97.4 0.313 / 90.4 0.603 / 25.5 spikes ball 0.385 / 70.2 0.495 / 59.8 0.725 / 45.4 0.919 / 33.6 0.719 / 46.9 0.464 / 66.0 0.543 / 54.4 1.029 / 7.4 0.553 / 57.5 0.483 / 61.4 0.577 / 42.5 straw hat 0.178 / 99.4 0.199 / 98.0 0.245 / 91.9 0.330 / 76.9 0.201 / 96.9 0.169 / 98.5 0.200 / 96.4 0.253 / 91.0 0.283 / 87.1 0.307 / 84.5 0.404 / 65.5 sneakers 0.162 / 96.2 0.160 / 97.3 0.223 / 92.9 0.527 / 59.7 0.276 / 89.4 0.173 / 94.6 0.195 / 93.6 0.286 / 86.1 0.312 / 81.2 0.315 / 82.1 0.460 / 57.5 surgery saw 0.204 / 91.4 0.245 / 88.4 0.632 / 61.1 1.221 / 24.6 0.792 / 53.1 0.398 / 75.9 0.510 / 69.2 1.024 / 39.6 0.361 / 80.5 0.422 / 74.0 0.624 / 26.8 whisk 0.301 / 88.9 0.468 / 50.9 1.433 / 12.0 – 1.488 / 14.7 0.397 / 66.1 0.598 / 33.0 – 0.211 / 91.3 0.255 / 89.8 – warrior head 0.188 / 94.3 0.246 / 88.9 0.386 / 73.9 0.628 / 48.7 0.328 / 83.0 0.338 / 78.6 0.308 / 83.5 0.382 / 76.1 0.488 / 69.6 0.515 / 68.3 0.670 / 34.3 plant 0.403 / 78.7 0.579 / 59.5 0.690 / 47.2 0.982 / 29.3 0.664 / 56.9 0.657 / 52.3 0.736 / 44.8 0.970 / 27.6 0.619 / 50.9 0.622 / 52.2 0.712 / 31.8 grid 0.216 / 93.4 0.363 / 81.9 0.428 / 60.9 0.838 / 28.1 0.408 / 64.1 0.434 / 74.2 0.601 / 63.2 0.875 / 27.1 0.347 / 78.2 0.346 / 75.6 0.568 / 49.6 bowl 0.276 / 90.5 0.484 / 73.4 1.024 / 34.2 1.791 / 6.6 1.022 / 37.9 0.364 / 79.1 0.729 / 45.7 0.927 / 15.8 1.134 / 24.5 1.175 / 20.3 1.190 / 14.8 spine 0.465 / 70.6 0.528 / 65.1 0.732 / 43.6 1.022 / 26.7 0.742 / 46.1 0.508 / 65.5 0.579 / 58.4 0.843 / 33.4 0.569 / 61.7 0.552 / 65.3 0.632 / 43.3 Mean 0.333 / 82.4 0.381 / 77.4 0.536 / 63.6 0.731 / 45.4 0.477 / 68.1 0.380 / 74.5 0.432 / 70.6 0.557 / 54.7 0.503 / 65.9 0.559 / 62.1 0.702 / 40.1 Reconstructed 26/26 26/26 26/26 23/26 25/26 26/26 26/26 21/26 26/26 26/26 22/26 |
| NeuS2 Neuralangelo RNb-NeuS2 SuperNormal Object d2 d4 d8 d4 d2 d4 d8 d4 rock 0.103 / 99.6 0.085 / 99.8 0.098 / 99.8 0.099 / 98.4 0.068 / 100.0 0.108 / 99.9 0.370 / 82.1 0.063 / 99.8 vase 0.271 / 67.0 0.268 / 66.9 0.302 / 65.4 0.276 / 81.8 0.610 / 60.7 0.495 / 67.7 0.529 / 68.4 0.947 / 42.9 hammer head 0.105 / 99.1 0.096 / 99.8 0.105 / 99.7 0.058 / 99.8 0.078 / 99.2 0.081 / 99.2 0.160 / 97.2 0.147 / 97.5 candle 0.253 / 86.6 0.257 / 86.0 0.263 / 86.3 0.247 / 89.9 0.304 / 81.0 0.292 / 81.8 0.365 / 75.1 1.482 / 0.8 can 0.420 / 78.8 0.388 / 79.9 0.412 / 79.7 0.307 / 83.6 0.515 / 68.0 0.528 / 67.1 0.708 / 53.1 0.563 / 51.7 glass 1.051 / 31.1 0.880 / 46.0 0.834 / 46.6 0.837 / 47.8 1.256 / 19.5 1.247 / 19.3 1.258 / 20.1 2.006 / 5.1 petanque ball 0.218 / 92.8 0.162 / 93.9 0.185 / 90.6 0.178 / 94.3 0.113 / 99.3 0.074 / 99.6 0.088 / 99.7 0.126 / 96.7 jam pot 0.383 / 77.4 0.386 / 76.3 0.438 / 70.6 0.550 / 63.9 1.203 / 9.9 1.254 / 10.2 1.213 / 10.4 1.255 / 10.9 colander 0.805 / 42.1 0.782 / 44.0 0.796 / 41.2 0.599 / 51.8 0.574 / 51.7 0.569 / 52.8 0.776 / 39.7 0.488 / 62.6 wood 0.269 / 87.3 0.272 / 87.0 0.308 / 84.9 0.225 / 89.7 0.185 / 97.5 0.161 / 96.8 0.172 / 96.5 0.145 / 94.1 plate 0.106 / 98.0 0.133 / 96.5 0.154 / 96.1 0.138 / 95.5 1.793 / 9.7 1.650 / 9.7 1.635 / 10.9 1.783 / 10.0 paw pad 0.461 / 71.9 0.461 / 71.2 0.460 / 69.6 0.257 / 85.9 0.874 / 33.0 0.881 / 33.4 0.906 / 31.5 1.327 / 18.4 moka pot 0.485 / 68.0 0.415 / 73.7 0.357 / 79.0 0.354 / 74.1 0.451 / 71.6 0.300 / 81.8 0.268 / 84.0 0.486 / 63.5 lego 0.326 / 80.1 0.332 / 79.8 0.334 / 79.5 0.294 / 81.8 0.604 / 59.2 0.539 / 61.1 0.613 / 58.2 0.723 / 56.9 surgery scissors 0.080 / 99.3 0.089 / 99.0 0.083 / 99.1 1.691 / 8.5 0.066 / 100.0 0.073 / 100.0 0.086 / 99.8 0.084 / 99.3 knife 0.134 / 98.4 0.141 / 97.7 0.141 / 98.4 0.278 / 89.3 0.231 / 87.5 0.198 / 91.2 0.216 / 89.7 0.380 / 77.1 spikes ball 0.304 / 80.8 0.249 / 86.9 0.247 / 87.7 0.232 / 87.9 0.193 / 97.0 0.166 / 98.1 0.137 / 98.3 0.256 / 93.3 straw hat 0.229 / 91.0 0.227 / 91.4 0.224 / 91.6 0.167 / 97.9 0.283 / 88.6 0.234 / 91.3 0.292 / 85.5 0.283 / 88.2 sneakers 0.265 / 82.8 0.288 / 81.4 0.282 / 81.2 0.151 / 91.4 0.279 / 82.8 0.262 / 84.4 0.269 / 84.3 0.285 / 82.8 surgery saw 0.256 / 87.6 0.252 / 87.6 0.241 / 88.9 0.214 / 92.3 0.313 / 80.9 0.300 / 83.2 0.338 / 79.1 0.444 / 67.3 whisk 0.126 / 96.2 0.124 / 97.8 0.134 / 95.6 – 0.084 / 98.9 0.065 / 99.2 0.109 / 98.5 0.097 / 98.1 warrior head 0.472 / 72.3 0.485 / 71.4 0.471 / 72.8 0.175 / 94.1 0.277 / 86.5 0.283 / 86.4 0.284 / 86.6 0.373 / 78.4 plant 0.517 / 68.8 0.533 / 67.4 0.545 / 66.9 0.418 / 74.9 0.531 / 64.5 0.395 / 76.0 0.422 / 73.8 0.462 / 68.5 grid 0.219 / 93.5 0.222 / 93.1 0.235 / 91.6 1.011 / 25.1 0.248 / 86.6 0.224 / 89.2 0.260 / 86.1 0.291 / 83.8 bowl 0.546 / 50.5 0.591 / 48.6 0.560 / 51.3 0.824 / 44.3 0.864 / 34.0 0.927 / 35.7 0.999 / 31.9 1.165 / 24.5 spine 0.437 / 71.0 0.434 / 71.7 0.447 / 70.8 0.368 / 80.8 0.202 / 91.2 0.228 / 90.1 0.296 / 85.2 0.404 / 77.4 Mean 0.393 / 75.0 0.388 / 75.5 0.391 / 75.3 0.353 / 77.9 0.501 / 64.0 0.469 / 66.7 0.540 / 62.4 0.571 / 60.8 Reconstructed 26/26 26/26 26/26 25/26 26/26 26/26 26/26 26/26 |
Table 6: Chamfer distance (mm, Eq. 7 ) / F-score at 0.5 mm (%) per object, method and image resolution (d1 to d8: full to one-eighth resolution), with all 84 views; Meshroom at d4 only; methods in two sub-tables, one under the other. Bold : best per row over both sub-tables, underline : second best, for each metric separately (lowest Chamfer distance, highest F-score). –: failure of the method (no reconstruction, Table 8 ). Mean : point distances of all objects pooled, as if they formed a single mesh, so that larger objects weigh more; a failed reconstruction keeps its ground-truth points in the pool, all missed: they lower the F-score, and lying beyond the cutoff of Eq. 7 they do not enter the Chamfer distance, so Chamfer means over different numbers of reconstructed objects ( Reconstructed row) are not directly comparable.
| Views (d4) | Resolution (84 views) | ||||||||
| Method | 84 | 40 | 20 | 10 | 5 | d1 | d2 | d4 | d8 |
| RealityScan | 26 | 26 | 25 | 24 | 13 | 26 | 26 | 26 | 23 |
| COLMAP | 26 | 24 | 21 | 14 | 4 | – | 26 | 26 | 21 |
| PGSR | 26 | 24 | 21 | 13 | 3 | – | 26 | 26 | 22 |
| NeuS2 | 26 | 26 | 26 | 26 | 26 | – | 26 | 26 | 26 |
| RNb-NeuS2 | 26 | 26 | 26 | 26 | 26 | – | 26 | 26 | 26 |
Table 7: Number of objects reconstructed (out of 26) per method, for each subset of input views at quarter resolution (d4) and for each image resolution with all 84 views. A missing reconstruction is a failure of the method (Table 8 ). –: configuration not run; at full resolution, the methods other than RealityScan fail for lack of GPU memory (Section 3.2 ).
| Configuration | Failed | Objects |
|---|---|---|
| Meshroom d4 | 1/26 | surgery scissors |
| RealityScan d4, 20 views | 1/26 | surgery scissors |
| RealityScan d4, 10 views | 2/26 | surgery scissors , whisk |
| RealityScan d4, 5 views | 13/26 | candle , glass , petanque ball , moka pot , lego , surgery scissors , spikes ball , surgery saw , whisk , plant , grid , bowl , spine |
| RealityScan d8 | 3/26 | paw pad , surgery scissors , whisk |
| COLMAP d4, 40 views | 2/26 | paw pad , surgery scissors |
Table 8: Failed reconstructions per configuration. COLMAP and PGSR fail when structure from motion on the images of the configuration alone yields no usable sparse model (single attempt, default settings, no fallback); the other failures are runs that produced no mesh. Failed objects are excluded from all aggregated scores and declared as in the captions.