RACE-FPP: A Robust AI-assisted Characterisation Enhancement for Fringe Projection Profilometry

Osman Ali; Xiangjun Kong; Tibebe Yalew; Waiel Elmadih; Samanta Piano

cs.CVOct 6, 2026

RACE-FPP: A Robust AI-assisted Characterisation Enhancement for Fringe Projection Profilometry

Authors: Osman Ali, Xiangjun Kong, Tibebe Yalew, Waiel Elmadih, Samanta Piano

Organizations: Manufacturing Metrology Team, University of Nottingham, Nottingham, United Kingdom · Taraz Metrology Ltd., Nottingham, United Kingdom

Abstract

Fringe Projection Profilometry (FPP) requires precise system characterisation to achieve reliable three-dimensional (3D) reconstructions; however, characterisation accuracy strongly depends on robust checkerboard feature localisation, which can deteriorate under challenging imaging conditions such as lens blur and characterisation target orientations. Existing deep learning-based corner detectors are typically assessed using detection metrics and camera reprojection error alone, without considering their wider impact on projector characterisation, camera-projector stereo characterisation consistency, or overall measurement accuracy. In this work, we introduce a complete FPP characterisation pipeline that incorporates deep learning-based corner detection into the standard camera characterisation workflow. We also characterise the projector by sampling phase values at the centres of the white squares in the characterisation target. Rather than treating corner detection as an isolated task, the proposed framework explicitly analyses how localisation errors propagate throughout the entire FPP characterisation chain. Performance is evaluated using detection metrics (e.g., precision and recall), camera and projector reprojection errors, and the camera and projector stereo characterisation. Across a mixed dataset of clean and degraded images, the camera reprojection error is reduced from 1.237 pixels to 0.259 pixels, while the projector reprojection error is reduced by roughly 50%. Dimensional evaluation of reconstructed artefacts shows improved geometric accuracy compared with those resulting from the conventional pipeline. Overall, the findings indicate increased robustness of system-level characterisation under challenging imaging conditions, thereby enabling more reliable industrial FPP measurements.

Figures & tables

Image dataset	RACE-FPP				OpenCV
Image dataset	PCK@0.5	PCK@1	PCK@5	PCK@10	PCK@0.5	PCK@1	PCK@5	PCK@10
Noise-free	19.70	100.00	100.00	100.00	12.12	100.00	100.00	100.00
Mixed images	16.67	58.33	100.00	100.00	16.67	49.24	96.21	97.73
Noisy images	3.03	19.70	99.24	100.00	5.30	15.15	77.27	87.88

Table 1: Percentage of correct keypoints (PCK) at different thresholds for RACE-FPP and OpenCV detections.

Error	RACE-FPP						OpenCV
	Noise-free		partial defocus		full defocus		Noise-free		partial defocus		full defocus
	Mean	std	Mean	std	Mean	std	Mean	std	Mean	std	Mean	std
Dx	-0.42	0.23	-0.28	1.17	1.02	1.32	-0.43	0.20	0.17	3.41	1.00	3.71
Dy	-0.42	0.16	-0.12	0.96	0.34	1.36	-0.44	0.13	0.34	3.65	1.00	3.84
Euclidean	0.63	0.19	1.22	0.94	2.00	0.86	0.64	0.16	2.27	4.47	3.30	4.43

Table 2: Mean and standard deviation (std) of the directional (

D_{x},D_{y}

) and Euclidean localisation errors in pixels (px) relative to the manually annotated ground truth (GT).

Dataset	Images detected (RACE-FPP)	RPE (RACE-FPP)	Images detected (OpenCV)	RPE (OpenCV)
Noise-free	16/16	0.2281	16/16	0.0806
Mixed images	16/16	0.2591	10/16	1.2365
Noisy images	16/16	0.4500	5/16	2.8514

Table 3: Detection rates and reprojection error (RPE) in pixels (px) for camera characterisation using RACE-FPP and OpenCV pipelines.

Method	Dataset	$f_{x}\times 10^{3}$ px	$f_{y}\times 10^{3}$ px	$c_{x}\times 10^{3}$ px	$c_{y}\times 10^{3}$ px
RACE-FPP	Noise-free	6.708	6.705	1.432	0.910
	Mixed images	6.733	6.728	1.429	0.896
	Noisy images	6.714	6.711	1.420	0.903
OpenCV	Noise-free	6.712	6.711	1.446	0.913
	Mixed images	6.597	6.634	1.397	1.018
	Noisy images	6.352	6.472	1.293	1.207

Table 4: Estimated camera intrinsic parameters obtained using the RACE-FPP and OpenCV pipeline for the three image datasets.

Dataset	RACE-FPP RPE		OpenCV RPE
Dataset	Projector	Stereo	Projector	Stereo
Noise-free	0.0512	0.1646	0.0357	0.1323
Mixed images	0.3935	0.3778	0.8571	1.6647
Noisy images	0.6305	0.1518	1.6849	3.1638

Table 5: Reprojection error (RPE) in pixels (px) for projector and stereo characterisation using RACE-FPP and the OpenCV pipeline.

Feature	Dataset	CMM	RACE-FPP	Deviation	Traditional FPP	Deviation
SD	Noise-free	49.945	49.89	-0.056	49.93	-0.015
	Mixed images		49.87	-0.075	51.37	+1.425
	Noisy images		49.83	-0.115	54.33	+4.385
Sphere 1 radius	Noise-free	4.997	5.03	+0.033	5.01	+0.013
	Mixed images		4.99	-0.007	5.12	+0.123
	Noisy images		5.00	+0.003	5.59	+0.593

Table 6: Sphere radius and spacing distance (SD) measurements (mm) of the precision dumbbell sphere, with deviations from the corresponding CMM measurements.

Dataset	CMM	RACE-FPP	std	Deviation	Traditional FPP	std	Deviation
Noise-free	4.917	4.91	0.04	-0.007	4.90	0.05	-0.017
Mixed images		4.91	0.03	-0.007	5.07	0.05	+0.153
Noisy images		4.92	0.04	+0.003	4.97	0.05	+0.053

Table 7: Average hood step-height measurements (mm) for the multi-feature artefact, with standard deviations (std) and deviations from the CMM reference.

Explore similar work

Jul 16, 2026cs.CV

Ray-based phase error correction for miniaturized DOE projector-based FPP under single-directional hyperbolic projection

Fringe Projection Profilometry (FPP) systems using miniaturized DOE pro-jectors often suffer from severe phase artifacts due to nonlinear projection characteristics and limited pattern controllability. We propose a ray-based phase error correction framework that models phase artifacts along projection rays from the projector pinhole, incorporating projector geometry without re-lying on image-domain processing or neighboring pixels. A projector pinhole estimation method based on a single-directional hyperbolic fringe pattern is introduced, through which projector geometry can be recovered without stereo calibration. In addition, a data-efficient strategy constructs the re-finement model from a single calibration pose. Experiments on miniaturized DOE projector-based FPP systems demonstrate significant improvements in reconstruction accuracy under nonlinear projection conditions, confirming the robustness and physical consistency of the proposed approach.

Seung-Jae Son, Yatong An, Jae-Sang Hyun

School of Mechanical Engineering Yonsei University, 03722, Seoul, South Korea · Meta Reality Labs, Redmond, 98052, Washington, USA

Vision-Language Reprojection Consistency Projection Heads Error Correction Category-Level Object Pose Estimation Correction

Ask PDF

Jun 10, 2026cs.CV

Spatially Coupled Phase-to-Depth Calibration for Fringe Projection Profilometry

In fringe projection profilometry (FPP), depth is commonly recovered by fitting a phase-to-depth relation independently at each camera pixel. Although such pixel-wise calibration achieves high local accuracy, neighboring pixels can acquire markedly different calibration functions even when they observe the same smooth surface, producing spatially inconsistent geometry and structured surface artifacts. We propose a spatially coupled phase-depth transformation in which all pixels share a single low-dimensional mapping-global phase scalars combined with affine spatial terms on the undistorted reference-camera grid-rather than independent per-pixel fits, optionally augmented by a bounded, spatially smooth correction field. We further introduce a native-grid pairing scheme that constructs phase-depth calibration pairs directly on the reference-camera grid: when depth supervision comes from a rectified active-stereo pipeline, planes are fitted in stereo 3D and sampled back onto the camera grid along native rays, so the phase maps are never rectified. On a dental target with high-resolution scanner ground truth, the proposed model attains point-to-surface RMSE comparable to an active-stereo reference (about 12μm aggregate) while substantially improving spatial coherence over pixel-wise polynomial and rational calibration, and reduces the runtime mapping to a few element-wise operations per pixel with negligible parameter storage.

Sehoon Tak, Jae-Sang Hyun

Department of Mechanical Engineering, Yonsei University, Seoul 03722, South Korea.

Projection Bird'S-Eye View

Ask PDF

Jun 13, 2026cs.LG

Diagnosing Shape-Prior Shortcuts in Long-Range Single-Shot Fringe Projection Profilometry

Learning-based single-shot fringe projection profilometry (FPP) has been studied almost entirely at close range, and the networks used are evaluated only on aggregate error, leaving open whether they recover depth from fringe phase or from object-level shape cues that correlate with depth. This paper diagnoses that question mechanistically in the long-range regime (standoff beyond 1 m). Using FPP-ML-Bench, an open photorealistic synthetic benchmark (15,600 fringe images, 50 objects at 1.5--2.1 m), we first formalize why the single-shot fringe-to-depth mapping is more severely ill-posed at long range: it is non-injective without fringe-order information, and the depth error from an incorrect fringe order grows as

Z^2

in the working distance. Systematic ablations, extended with a multi-frame study, establish a best UNet baseline at 14.54 mm object mean absolute error (MAE), 18% of the 80 mm object depth range, with only a 1.9

\times

spread across four architectures, indicating a representational rather than a capacity-bound limit. A mechanistic interpretability study, the first applied to an FPP network, localizes the cause: linear probing shows edges are 2.82

\times

more decodable than depth, Grad-CAM shows attention favoring boundaries over fringes by 1.28

\times

, and an in-range flat-plane test collapses a featureless plane to background depth despite valid fringes. The baseline solves the task via object-boundary shape priors rather than fringe-phase decoding. Because the shortcut is a hypothesis-space property, additional data or larger models will not remove it, motivating an architectural repair that removes the shape-prior solution by construction.

Adam Haroon, Anush Lakshman, Cody Fleming +1

Department of Mechanical Engineering, Iowa State University, 2529 Union Drive, Ames, 50011, Iowa, USA · College of Engineering, University of Georgia, 597 D. W. Brooks Drive, Athens, 30602, Georgia, USA

Vision-Language Reprojection Consistency 3D Priors Signal-To-Noise Ratio Diagnosis

Ask PDF