cs.CVJun 24, 2026

Cross-View Variance Correlation in Path-Traced Stereo:A Hidden Shortcut in Synthetic Training Data

Authors: Po-Ting Lin

Organizations: Independent Researcher · Taiwan,Tainan

Abstract

Path-traced synthetic stereo data underlie a large fraction of modern disparity-estimation training pipelines. We report a previously unrecognised property of such data: while the Monte Carlo (MC) noise streams of the two cameras are statistically independent, the underlying \emph{variance fields} -- deterministic per-pixel functions of the rendering integrand -- are highly correlated once aligned by the ground-truth disparity warp. Across 20 scenes rendered with Mitsuba~3, the warped Pearson correlation reaches ρ=0.754±0.016ρ{=}0.754{\pm}0.016 across 20 scenes at SPP=512\mathrm{SPP}{=}512, and on a representative scene remains essentially invariant (ρ=0.778±0.001ρ{=}0.778{\pm}0.001) over a 16×16\times range of samples per pixel. The effect is strongest in Lambertian regions (ρ0.78ρ{\approx}0.78) and substantially weaker in glass (ρ0.30ρ{\approx}0.30), as predicted by an integrand decomposition into view-independent and view-dependent components. A residual-shuffle intervention that breaks the cross-view alignment while preserving the clean image degrades the GT cost margin by 33%33\% on non-glass and the variance-based winner-take-all accuracy on glass by 4.3×4.3\times, confirming the structure functions as a matching cue. This signal is unique to MC-rendered data and constitutes a candidate sim-to-real shortcut whose impact on trained networks remains to be quantified.

Explore similar work

Apr 6, 2026cs.CV

SimpleProc: Fully Procedural Synthetic Data from Simple Rules for Multi-View Stereo

Generating procedural synthetic data for multi-view stereo (MVS) usually requires writing complex rules to match the realism of curated datasets. We demonstrate that we can generate effective training data using SimpleProc: a new, fully procedural generator driven by a very small set of rules based on Non-Uniform Rational Basis Splines (NURBS), as well as simple displacement and texture patterns. At a modest scale of 8,000 images, our approach achieves superior results compared to manually curated images at the same scale sourced from games and real-world objects. When scaled to 352,000 images, our approach achieves similar, and in some benchmarks, even better results than the current state-of-the-art trained on over 692,000 manually curated images. The source code and the data are available at https://github.com/princeton-vl/SimpleProc.
Zeyu Ma, Alexander Raistrick, Jia Deng
May 22, 2026cs.CV

StereoGenBench: A Synthetic Multi-Camera Benchmark for Stereo Generation under Controlled Baseline Regimes

Stereo image and video generation, stereo geometry estimation, and condition-controlled view synthesis require paired data in which the variables that determine binocular geometry -- camera baseline, intrinsics, scene depth, and camera motion -- are known and controllable. Existing stereo resources provide subsets of these variables, but resources commonly used for stereo generation evaluation do not, to our knowledge, provide scene-paired, calibrated multi-baseline right-view ground truth with jointly recorded intrinsics, dense metric depth, and per-frame poses in a single controlled source. We introduce StereoGenBench, a synthetic Unreal Engine benchmark designed to make baseline-regime sensitivity and target-camera consistency measurable under matched scene content. Each scene is rendered with a rigid six-camera lateral array, yielding up to 15 calibrated view pairs; adjacent baselines are sampled from inter-pupillary to wide-baseline regimes; focal length is sampled independently; and every view is released with RGB, metric depth, intrinsics, per-pair baselines, and per-frame poses. The splits include two evaluation families for narrow and wide baseline regimes and a train-only family for broader all-pairs coverage. We release the dataset, evaluation code, reference results, Croissant metadata, and generation code/configuration for extension with compatible assets. The dataset is available at https://huggingface.co/datasets/stereo-dataset/stereo-dataset
Yangzhi Cui, Feng Qiao, Nathan Jacobs
Mar 25, 2026cs.CV

WAFT-Stereo: Warping-Alone Field Transforms for Stereo Matching

We introduce WAFT-Stereo, a simple and effective warping-based method for stereo matching. WAFT-Stereo demonstrates that cost volumes, a common design used in many leading methods, are not necessary for strong performance and can be replaced by warping with improved efficiency. WAFT-Stereo ranks first on ETH3D (BP-0.5), Middlebury (RMSE), and KITTI (all metrics), reducing the zero-shot error by 81% on ETH3D, while being 1.8-6.7x faster than competitive methods. Code and model weights are available at https://github.com/princeton-vl/WAFT-Stereo.
Yihan Wang, Jia Deng