Affine Correspondences in Stereo Vision: Theory, Practice, and Limitations

Levente Hajder

cs.CVMar 2, 2026

Affine Correspondences in Stereo Vision: Theory, Practice, and Limitations

Authors: Levente Hajder

Organizations: Faculty of Informatics, Eötvös Loránd University, Pázmány Péter stny. 1/C, Budapest, H-1117, Hungary

Abstract

Affine transformations have been recently used for stereo vision. They can be exploited in various computer vision application, e.g., when estimating surface normals, homographies, fundamental and essential matrices. Even full 3D reconstruction can be obtained by using affine correspondences. First, this paper overviews the fundamental statements for affine transformations and epipolar geometry. Then it is investigated how the transformation accuracy influences the quality of the 3D reconstruction. Besides, we propose novel techniques for estimating the local affine transformation from corresponding image directions; moreover, the fundamental matrix, related to the processed image pair, can also be exploited. Both synthetic and real quantitative evaluations are implemented based on the accuracy of the reconstructed surface normals. For the latter one, a special object, containing three perpendicular planes with chessboard patterns, is constructed. The quantitative evaluations are based on the accuracy of the reconstructed surface normals and it is concluded that the estimation accuracy is around a few degrees for realistic test cases. Special stereo poses and plane orientations are also evaluated in detail.

Figures & tables

Figure 1: Two images of a spatial scene. A small planar surface is perspectively projected to images $P_{1}$ and $P_{2}$ . The shape deformation between projected patches are transformed by affine transformation $\mathbf{A}$ . Green : Point cloud can be reconstructed from PCs. Green+Red : Oriented point cloud obtained from ACs

Figure 2: An affine transformation transforms the shape of the red quadrilaterals from the first to the second images. It also connects normals of corresponding epipolar lines. The direction and scale changes of normals $\mathbf{n}_{1}$ and $\mathbf{n}_{2}$ are determined by the affine transformation. $\mathbf{C}_{1}$ , $\mathbf{C}_{2}$ , $\mathbf{e}_{1}$ , and $\mathbf{e}_{2}$ are focal points and epipoles, respectively.

Figure 3: Block diagram of proposed pipeline for AC-based 3D reconstruction. This pipeline is designed for the quantitative evaluation. Its input are only the chessboard corners. The output are the reconstructed oriented point clouds: spatial locations and normals of reconstructed surfaces.

Panel of Figure 4 — Figure 4: Left. Three different directions are considered, each connects chessboard corners: horizontal (red), vertical (green), diagonal (purple). The horizontal and vertical directions are exploited for affine transformation estimation when two directions are used (methods F2UDIR and 2SDIR); diagonal directions are applied for three direction-based estimation (methods F3UDIR, DET3UDIR, 3SDIR). For the diagonal direction, only one out of the two possible directions is utilized. Right. The structure of the planes for special motions. Plane #1, #2, #3 are perpendicular to the axes $Z$ , $Y$ , and $X$ , respectively.

Figure 5: Directional error of the normals w.r.t. plane normals. General stereo pose is considered. Directional noise, given in degrees, is added to the angle of 2D directions. The affine transformations are computed by the F2UDIR algorithm. The average error for all planes are plotted.

Figure 6: Quantitative comparison of the proposed affine estimators in case of synthetic input. General stereo pose is considered. The noise was added to the chessboard directions by a zero-mean Gaussian error with spread $3.0^{\circ}$ . The average directional error of the surface normals are computed. The affine transformations are estimated by the proposed algorithms.

Panel of Figure 7 — Figure 7: Synthetic testing results for special motion. Three different testing cases are considered: planar motion, forward motion and standard stereo. The reconstruction error of tree planes are considered: Plane #1 : Parallel with camera image, Plane #2 : Horizontal plane, and Plane #3 : Vertical plane, parallel to the other ones

Panel of Figure 8 — Figure 8: Testing environment for real-world experiments. Images are taken at ’home laboratory’.

Panel of Figure 9 — Figure 9: Four test cases for quantitative comparison on real-world images. Regular optics are used. Top two rows: input images. Bottom row: a snapshot of the visualized 3D scene. Cameras are visualized by pyramids.

Panel of Figure 10 — Figure 10: Four test cases for quantitative comparison on real-world images. Fisheye lens is used with high field of view. Top two rows: input images. Bottom row: a snapshot of the visualized 3D scene. Cameras are visualized by pyramids.

		General Motion		Planar Motion
		Reg.	Fish.	Reg.	Fish.
Orthogonality Error	Plane #1–#2	3.6010	0.7112	4.7177	0.5222
	Plane #1–#3	0.2599	0.2068	2.1562	2.0352
	Plane #2–#3	4.1743	1.1883	3.1345	2.0037
Normal Dir. Err. (mean)	Plane #1	0.8622	2.8618	0.4945	4.7222
	Plane #2	5.1772	0.9905	4.1530	8.1205

Table 1: Quantitative results for real images. Four stereo pose types are tested. Both regular (Reg.) and fisheye (Fish.) lens are applied. Orthogonality error : The captured planes are perpendicular, the deviation from

90^{\circ}\textdegree

is reported. Normal Dir. Err. The estimated normals compared to the normal of the planes fitted by PCA algorithm. Angular errors are reported in degrees.

		Measured	Reconstructed
			F2UDIR	F3UDIR	DET3UDIR	2SDIR	3SDIR
Regular	Std. Stereo	300	292.90	289.35	293.46	293.44	290.70
Regular	Forw. Motion	150	142.81	142.54	147.62	146.29	148.93
Fisheye	Std. Stereo	300	307.10	309.22	308.91	308.66	308.62
Fisheye	Forw. Motion	200	180.28	179.65	179.5	179.77	179.81

Table 2: Measured and reconstructed baseline lengths. Five variants of the affine transformation estimation, proposed in Section 3 , are run. Distances given in millimeters. Best values are highlighted by bold font.

Stereo Vision

Stereo vision uses images from two or more viewpoints to estimate depth, recover 3D structure, and support spatial perception.

See all 43 papers

GeoStereo: A Unified Stereo Geometry Estimation Framework for Disparity and Surface Normal
Jul 27, 2026Qizhe Wei, Xianda Guo, Shaocong Xu +3Surface Normal Estimation Stereo Matching
Beijing Institute of Technology BAAI Beijing, China · School of Computer Science, Wuhan University Wuhan, China · BAAI Beijing, China +3
WAFT-Stereo: Warping-Alone Field Transforms for Stereo Matching
Mar 25, 2026Yihan Wang, Jia DengStereo Matching Stereo Vision
Department of Computer Science, Princeton University
Spherical-to-ERP Epipolar Rectification for Single-Axis Disparity in 360 Stereo
Jun 23, 2026Sahereh Obeidavi, Dieter LandesStereo Matching Stereo Vision
Faculty of Electrical Engineering and Computer Science, Coburg University of Applied Science, German

Jul 27, 2026J/K move · Enter open · S save

GeoStereo: A Unified Stereo Geometry Estimation Framework for Disparity and Surface Normal

Qizhe Wei, Xianda Guo, Shaocong Xu, Hong Li+2

Beijing Institute of Technology BAAI Beijing, China · School of Computer Science, Wuhan University Wuhan, China · BAAI Beijing, China+3

Surface Normal Estimation Stereo Matching

Open paper AskPDF arXiv

Stereo matching and surface normal estimation are fundamental tasks in 3D vision. However, existing feed-forward stereo methods still struggle to produce reliable predictions in challenging regions, mainly due to the lack of strong geometric priors. In this paper, we propose

\textbf{GeoStereo}

, a unified stereo geometry estimation framework that leverages powerful diffusion priors to jointly predict disparity and surface normals. Specifically, GeoStereo couples a feed-forward stereo matching pipeline with a diffusion-based normal estimation branch. To enable effective interaction between the two tasks, we introduce a disparity to normal initialization strategy and construct a warp to left-view condition for the diffusion process. This coupled design allows the diffusion branch to provide strong structural priors that enhance disparity estimation in ill-posed regions, while the feed-forward branch offers reliable geometric guidance for accurate normal prediction. Extensive experiments show that GeoStereo performs reliably in challenging scenarios, including low-light environments, highly reflective surfaces, and transparent objects. Under zero-shot settings, it achieves Rank-1 disparity estimation on multiple benchmarks, including KITTI and NYUv2, and delivers the best normal estimation accuracy on many real indoor benchmarks, such as iBims-1 and ScanNet. Project page: https://qz-wei.github.io/GeoStereo.github.io/

Abstract

Stereo Vision

Explore similar work

GeoStereo: A Unified Stereo Geometry Estimation Framework for Disparity and Surface Normal

WAFT-Stereo: Warping-Alone Field Transforms for Stereo Matching

Spherical-to-ERP Epipolar Rectification for Single-Axis Disparity in 360 Stereo