StreamRig: Exploiting Intra-Rig Geometry for Streaming Multi-Camera Odometry

Yufei Wei; Shuhao Ye; Qi Wang; Xin Zheng; Qing Huang; Rong Xiong; Yue Wang

cs.CVSep 30, 2026

StreamRig: Exploiting Intra-Rig Geometry for Streaming Multi-Camera Odometry

Authors: Yufei Wei, Shuhao Ye, Qi Wang, Xin Zheng, Qing Huang, Rong Xiong, Yue Wang

Abstract

Mobile robots and vehicles carry synchronized multi-camera rigs, yet many streaming 3D foundation models are designed for monocular input, leaving efficient use of rig geometry a challenge. We present StreamRig, a freeze-and-stream framework that builds causal streaming odometry for calibrated rigs on a frozen multi-view 3D foundation model. The frozen front-end jointly perceives the synchronized views using rig calibration. A Rig-Resampler compresses their features, a CausalBridge applies causal attention with a key-value cache, and a lightweight head regresses rig poses. A periodic re-anchoring protocol supports stable pose estimation over long sequences. Only these modules are trained, 74.6M parameters in total, with relative poses as the sole supervision. Our two-stage training strategy combines group relocalization pretraining with causal rig training to transfer the geometric priors of the frozen front-end and the alignment ability of the pretrained modules to streaming odometry. We evaluate on NCLT, TartanGround, KITTI-360, and our self-collected humanoid-robot dataset ZJH, where training uses only simulation and real-world evaluation is zero-shot. Across all four datasets, StreamRig achieves lower translation and rotation drift than the evaluated non-oracle monocular streaming and rig-aware offline models, while maintaining low inference cost. Ablations and controlled camera-count experiments identify the sources of these gains. We further examine how longer training windows affect inference over longer horizons. Code has been released at https://github.com/WeiYuFei0217/StreamRig.

Figures & tables

Fig. 1: Accurate and efficient surround-view odometry. (a) StreamRig follows the NCLT ground-truth trajectory more closely than G2G-chain and CUT3R. (b) RGB-colored LiDAR points stitched using our predicted poses visualize the boxed region. (c) Five-camera StreamRig achieves lower ATE, GPU memory use, and latency than monocular CUT3R. Accuracy and costs are measured in separate configurations (Sec. IV-E ).

Fig. 2: StreamRig overview. (a) The frozen front-end perceives each rig with its calibration, the Rig-Resampler compresses every camera into $L$ latent tokens, the CausalBridge attends causally over the compressed rigs, and the pose head regresses $T_{a\leftarrow t}$ from the bridge outputs. Bottom: inside the bridge, tokens are tagged with camera, group, and anchor embeddings, attention across rigs is causal with the keys and values of earlier rigs cached, and every query rig receives its own anchor snapshot; the optional long-history mode prunes the latents of distant rigs. (b) The two training stages, group relocalization with bidirectional attention and causal training on rig windows with the same modules, and the streaming loop that re-anchors every $N$ rigs.

			NCLT			TartanGround			KITTI-360			ZJH (real robot) ‡
			5-cam surround, co-centered			4 $\times$ 90 ∘ co-located ring			stereo 0.59 m $+$ 2 fisheye			stereo 7 cm $+$ 2 oblique
			2 traj. / 12.29 km			10 scenes / 12.60 km			2 scenes / 13.63 km			3 scenes / 0.36 km
Method	In.	Tr.	$t_{rel}\!\downarrow$	$r_{rel}\!\downarrow$	ATE $\downarrow$	$t_{rel}\!\downarrow$	$r_{rel}\!\downarrow$	ATE $\downarrow$	$t_{rel}\!\downarrow$	$r_{rel}\!\downarrow$	ATE $\downarrow$	$t_{rel}\!\downarrow$	$r_{rel}\!\downarrow$	ATE $\downarrow$
Non-streaming (offline / pairwise)
Reloc3R-rig † [ 15 ]	full	M	24.26	11.06	207.4	36.39	79.18	28.6	47.12	15.02	345.4	16.49	163.17	3.77
Rig3R † [ 11 ]	full	M	48.80	24.31	216.7	158.89	231.79	65.4	471.96	34.78	4257.0	28.02	409.79	5.23

TABLE I: STREAMING ODOMETRY ON FOUR RIGS (

t_{rel}

IN %,

r_{rel}

IN ∘ /100 M, ATE IN M).

Cameras	$t_{rel}\!\downarrow$	$r_{rel}\!\downarrow$	SE3 $\downarrow$	Sim3 $\downarrow$
Mono	4.84	1.92	85.4	78.5
Stereo	3.59	1.33	67.2	65.6
Stereo $\bm{+}$ 2 side	2.59	0.98	63.7	59.1

TABLE II: MONO, STEREO, AND FULL RIG ON KITTI-360.

Fig. 3: Example trajectories from the Table I models, after full-sequence Sim(3) alignment and translation to a common start; KITTI-360 follows the same GT gap-bridging protocol as Table I .

Fig. 4: Re-anchoring interval and training horizon on NCLT. Translation drift (filled, left axis) and rotation drift (open, right axis) versus re-anchoring interval. Stars mark extrapolation beyond training limits. Top bars show estimated training costs in RTX 4090 GPU-hours.

Table 7

Streaming (monocular)
Method	Input	ms $\downarrow$	Thr. $\uparrow$	Mem $\downarrow$
CUT3R (recurrent)	$224{\times}224$	31.3	31.9	3.9
STream3R (window 5)	$224{\times}224$	57.6	17.4	6.6
STream3R (full causal)	$224{\times}224$	121.4	8.2	OOM
LingBot-Map	$518{\times}378$	150.5	6.6	13.8
HorizonStream	$504{\times}378$	935.3	1.1	15.0

TABLE V: COMPUTE AND MEMORY BUDGET PER ARRIVAL.

Explore similar work

HorizonStream: Long-Horizon Attention for Streaming 3D Reconstruction
May 22, 2026Chong Cheng, Peilin Tao, Nanjie Yao +93D Reconstruction Shape Correspondence
HKUST(GZ) · Horizon Robotics · CASIA +2
$R^3$ : 3D Reconstruction via Relative Regression
May 26, 2026Congrong Xu, Huachen Gao, Xingyu Chen +3Feed-Forward 3D Reconstruction 3D Reconstruction
University of Michigan · Westlake University · NVIDIA
Anchor3R: Streaming 3D Reconstruction with Transient Anchors for Long-Horizon Visual Mapping
Jun 3, 2026Peilin Tao, Chong Cheng, Yuansen Du +8Feed-Forward 3D Reconstruction 3D Reconstruction
1CASIA · 2UCAS · 3Horizon Robotics +1

May 22, 2026, cs.CVJ/K move · Enter open · S save

HorizonStream: Long-Horizon Attention for Streaming 3D Reconstruction

Chong Cheng, Peilin Tao, Nanjie Yao, Guanzhi Ding+8

HKUST(GZ) · Horizon Robotics · CASIA+2

3D Reconstruction Shape Correspondence Streaming Causal Attention Horizon

Open paper AskPDF arXiv

Online 3D reconstruction requires estimating camera pose and scene geometry under strict causal and bounded-memory constraints. Existing methods often suffer from drift, jitter, or collapse on long sequences. We trace these failures to a fundamental mismatch. Streaming geometry is inherently temporally heterogeneous, with evidence ranging from short-lived correspondences to persistent global scale. However, current architectures impose uniform and pathological influence patterns. For example, sliding windows enforce hard cutoffs, while ungated recurrence and causal attention cause cache saturation and spike-like attention sinks. To resolve this, we formalize geometric propagation as an \emph{evidence influence kernel} and propose HorizonStream, a long-horizon Transformer that explicitly factorizes this kernel. For the long-range temporal factor, Geometric Linear Attention learns channel-wise decay rates to enable bounded, multi-timescale propagation of geometric evidence. For the short-range spatial factor, Geometric Local Attention with Spatiotemporal RoPE performs reliable 3D matching while suppressing attention sinks. Finally, Metric Readout Tokens recover stable scale and rigid pose directly from the persistent geometric state. Extensive experiments show that HorizonStream, trained on only 48-frame clips, generalizes stably to sequences exceeding 10,000\ frames with constant memory and linear time, achieving state-of-the-art streaming 3D reconstruction performance. Project Page: https://3dagentworld.github.io/horizonstream/

Abstract

Explore similar work

HorizonStream: Long-Horizon Attention for Streaming 3D Reconstruction

R3R^3R3: 3D Reconstruction via Relative Regression

Anchor3R: Streaming 3D Reconstruction with Transient Anchors for Long-Horizon Visual Mapping

$R^3$ : 3D Reconstruction via Relative Regression