MemReg: Streaming Outdoor LiDAR Point Cloud Registration with Hybrid Memory Buffers

Jiuming Liu; Mengmeng Liu; Guangming Wang; Lihao Liu; Michael Ying Yang; Yunpeng Zhang; Francesco Nex; Hao Cheng

cs.CVOct 8, 2026

MemReg: Streaming Outdoor LiDAR Point Cloud Registration with Hybrid Memory Buffers

Authors: Jiuming Liu, Mengmeng Liu, Guangming Wang, Lihao Liu, Michael Ying Yang, Yunpeng Zhang, Francesco Nex, Hao Cheng,

Organizations: School of Automation and Intelligent Sensing, Shanghai Jiao Tong University · University of Twente · University of Cambridge · University of Bath ·

Abstract

Prior outdoor LiDAR registration works are commonly limited by the pair-wise input paradigm, which neglects temporal correlations inherent within streaming sequences. In this paper, we propose a novel multi-frame outdoor point cloud registration network for streaming LiDAR scans enhanced with pose-related memory buffers. The key observation is that long-term temporal LiDAR sequences can provide rich global contextual information to complete sparse measurements, filter outliers, and address low-overlap problems, thereby boosting registration performance. Specifically, two hybrid memory buffers are designed, including an implicit memory feature buffer and an explicit memory pose buffer, to store and dynamically update pose-related temporal features. Moreover, a novel dynamic history weighting module is developed to adaptively fuse current and history pose-related features. Extensive experiments on three outdoor datasets, including KITTI, nuScenes, and Apollo-Southbay, demonstrate state-of-the-art performance of MemReg, surpassing all previous pair-wise methods and also multi-frame SLAM systems. Our method also generalizes surprisingly well to multiview indoor registration scenarios with rather competitive performance on 3DMatch, 3DLoMatch, and ScanNet. Code will be released at https://github.com/IRMVLab/MemReg.

Figures & tables

Figure 1: Comparison with prior pair-wise registration methods. Prior methods are commonly constrained by pair-wise registration (a), lacking temporal exploration in LiDAR sequences. In contrast, we propose a streaming multi-frame registration network using long-term temporal priors from hybrid memory buffers (b).

Figure 2: The overview architecture of MemReg. We leverage the long-term temporal features from history frames to enhance the current registration. During the current registration, memory features and memory poses are first read-out and interacted with current motion features through the Mamba-based Temporal Encoding module and Dynamic History Weighting module. Then, the current refined poses will, in turn, update memory buffers for next-timestamp estimation.

Table 3

Figure 3: The registration recall under different RRE or RTE thresholds on the KITTI and nuScenes datasets.

Table 5

Figure 4: Visualization of point cloud registration on KITTI. Yellow and blue points indicate input point pair frames, with green lines showing accurate correspondences (inliers) and red lines showing inaccurate ones (outliers). This figure is best viewed in zoomed-in color mode. Compared to HRegNet ∗ Lu et al. (2023b) , our method MemReg estimates more accurate correspondences.

Figure 5: Visualization of dynamic history weighting.

Figure 6: Multiview indoor registration on 3DMatch Zeng et al. (2017) . We adapt our model to indoor scenes Wang et al. (2023a) , where our MemReg has more accurate pose estimation for areas with the red circle than the baseline method SGHR.

Table 9

Model	RTE (m)	RRE (deg)	Time (ms)
w/o DHW	$0.038\pm 0.038$	$0.136\pm 0.113$	138
with estimator in Liu et al. (2024c)	$0.036\pm 0.040$	$0.132\pm 0.104$	135
with DHW	$\mathbf{0.032\pm 0.036}$	$\mathbf{0.122\pm 0.096}$	133

Table 7: Ablation studies of the Dynamic History Weighting (DHW) on KITTI.

Table 11

Method	Intervals	RTE (m)	RRE (deg)	Recall
HRegNet* Lu et al. (2023b)	20-frame	$0.085\pm 0.152$	$0.235\pm 0.213$	96.2%
Ours	20-frame	$\bf{0.041\pm 0.039}$	$\bf{0.148\pm 0.127}$	100.0 %
HRegNet* Lu et al. (2023b)	30-frame	$0.123\pm 0.195$	$0.315\pm 0.302$	76.3%
Ours	30-frame	$\bf{0.046\pm 0.042}$	$\bf{0.169\pm 0.141}$	100.0 %

Table 10: Evaluation on low-overlap input pairs with 20-frame and 30-frame intervals on KITTI.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: The architecture of the Mamba module. We utilize the standard Mamba ( Gu and Dao, 2023 ) module without any task-specific design.

Module	Layer	N	MLP1	MLP2
Coarse Matching	Layer 1	8	[128, 128, 128]	[128, 128, 1]
Module	Layer	N	MLP1	MLP2
Pose Refinement	Layer 2	8	[256, 256, 256]	[256, 256, 1]
Pose Refinement	Layer 3	8	[512, 512, 512]	[512, 512, 1]

Appendix

Table 11: Implementation details of coarse matching and pose refinement layers. In the coarse matching and pose refinement layers, ‘N’ denotes the number of the selected neighboring points in KNN. ‘MLP1’ and ‘MLP2’ represent the output dimension of two MLP layers, respectively, for correspondence prediction.

Methods	KITTI Odometry
Methods	RTE (m)	RRE (deg)	Recall
PosDiffNet ( She et al., 2024 )	$0.066$	$0.240$	99.80%
TrT-Net ( Chen et al., 2024a )	$0.063$	$0.230$	99.80%
ML-SemReg ( Yan et al., 2024 )	$0.052$	$0.200$	97.91%
RAP ( Pan et al., 2026 )	$0.043$	$0.139$	100.0%
Ours	$\bf{0.032}$	$\bf{0.122}$	100.0 %

Appendix

Table 12: More comparison results on the KITTI odometry ( Geiger et al., 2012 ) .

$\theta_{T}$ (m)	Method	RTE (m)	RRE (deg)	Recall
0.05	HRegNet ∗	$0.0302\pm 0.0110$	$0.1583\pm 0.1155$	72.08%
0.05	Ours	$\bf{0.0243\pm 0.0109}$	$\bf{0.1175\pm 0.0837}$	87.50%
0.1	HRegNet ∗	$0.0406\pm 0.0201$	$0.1606\pm 0.1195$	95.60%
0.1	Ours	$\bf{0.0294\pm 0.0172}$	$\bf{0.1184\pm 0.0889}$	97.93%
0.2	HRegNet ∗	$0.0444\pm 0.0271$	$0.1613\pm 0.1192$	97.62%
0.2	Ours	$\bf{0.0310\pm 0.0225}$	$\bf{0.1198\pm 0.0881}$	98.13%

Appendix

Table 13: Comparison results with HRegNet ∗ ( Lu et al., 2023b ) under different translation thresholds

\theta_{T}

on the KITTI dataset.

\theta_{R}

is set to 1deg.

$\theta_{R}$ (deg)	Method	RTE (m)	RRE (deg)	Recall
0.25	HRegNet ∗	$0.0302\pm 0.0109$	$0.1210\pm 0.0627$	57.20%
0.25	Ours	$\bf{0.0241\pm 0.0109}$	$\bf{0.1006\pm 0.0538}$	81.10%
0.5	HRegNet ∗	$0.0304\pm 0.0109$	$0.1524\pm 0.0973$	67.38%
0.5	Ours	$\bf{0.0242\pm 0.0109}$	$\bf{0.1158\pm 0.0781}$	87.12%
1	HRegNet ∗	$0.0304\pm 0.0109$	$0.1593\pm 0.1121$	68.41%
1	Ours	$\bf{0.0243\pm 0.0109}$	$\bf{0.1192\pm 0.0882}$	87.71%

Appendix

Table 14: Comparison results with HRegNet ∗ ( Lu et al., 2023b ) under different rotation thresholds

\theta_{R}

on the KITTI dataset.

\theta_{T}

is set to 0.05m.

Model	RTE (m)	RRE (deg)	Recall	Time (ms)
Temporal Averaging	$0.045\pm 0.041$	$0.143\pm 0.117$	100%	131
GRU	$0.034\pm 0.039$	$0.133\pm 0.115$	100%	135
Attention	$0.033\pm 0.039$	$0.138\pm 0.121$	100%	136
Mamba (Ours)	$\mathbf{0.032\pm 0.036}$	$\mathbf{0.122\pm 0.096}$	100%	133

Appendix

Table 15: Ablation studies of temporal encoding methods on KITTI.

Mamba layers	RTE (m)	RRE (deg)	Recall
1	$\mathbf{0.032\pm 0.036}$	$0.122\pm 0.096$	100%
2	$\mathbf{0.032}\pm 0.037$	$\mathbf{0.120}\pm 0.103$	100 %
3	$\mathbf{0.032}\pm 0.037$	$0.121\pm\mathbf{0.092}$	100%

Appendix

Table 16: Ablation studies for different Mamba layers on KITTI.

Method	RTE (m)	RRE (deg)	Recall
Ours w/ fixed $W_{1}$	$0.036\pm 0.039$	$0.131\pm 0.103$	100 %
Ours w/ fixed $W_{2}$	$0.037\pm 0.041$	$0.134\pm 0.107$	100 %
Ours	$\mathbf{0.032\pm 0.036}$	$\mathbf{0.122\pm 0.096}$	100%

Appendix

Table 17: Ablation studies of the learnable parameters (

W_{1}

W_{2}

) in Dynamic History Weighting.

Figure 10: A dynamic case in KITTI. In a highway scene, partial observations hinder the recognition of the highly dynamic car.

Figure 11: An occlusion case in KITTI. A pairwise pose regression can fail in occluded scenarios, e.g., the static car in the red circle in Frame 2, breaking the correlation in Frames 1&2 and 2&3.

Figure 12: Registration error visualization. Our estimation (blue points) shows much smaller errors (red line) compared to HRegNet ∗ ( Lu et al., 2023b ) (yellow points) relative to GT (green points).

Figure 13: The visualization of Dynamic History Weighting. We show some samples about 2D trajectories (upper figures) and their corresponding weights (lower figures), which are generated from the PCA analysis of the weight matrix $W_{2}$ .

Figure 14: The visualization of point cloud registration on KITTI. Yellow and blue points indicate input point pair frames, with green lines showing accurate correspondences (inliers) and red lines showing inaccurate ones (outliers). This figure is best viewed in zoomed-in color mode.

Point Cloud Registration

Point cloud registration aligns 3D point sets by estimating the transformations and correspondences that make their geometry consistent.

See all 110 papers

Aug 20, 2026

CVSD-Reg: Cross-Modal Visual Semantic Prior Distillation for Robust LiDAR Registration

Learning-based global point cloud registration has achieved remarkable progress, yet its reliance on geometric representations makes existing methods sensitive to variations in point density, scan pattern, viewpoint, and sensor characteristics. We propose CVSD-Reg, a robust global LiDAR registration framework that distills visual semantic priors from a vision foundation model into LiDAR representations. In Stage 1, a Point Transformer V3 student learns from a frozen DINOv2 teacher through contrastive distillation and spherical-manifold alignment, which preserves the hyperspherical geometry of the teacher embedding space. Self-supervised InfoNCE consistency and soft

\mathrm{SE}(3)

invariance further encourage viewpoint-robust descriptors. In Stage 2, the distilled representation is adapted to registration through correspondence learning, density-aware point-dropout augmentation, and end-to-end pose optimization. With a single checkpoint, CVSD-Reg generalizes to both single-sensor and zero-shot cross-sensor scenarios without sensor-specific adaptation and remains entirely camera-free at inference. On KITTI, nuScenes, and HeLiPR, CVSD-Reg achieves strict success rate ([email protected],m/

1^\circ

) of 97.7

\%

, 99.0

\%

, and 99.3

\%

, respectively, including 97.3

\%

on sparse 16-beam Velodyne scans. It outperforms state-of-the-art geometric registration methods by up to 44.0 percentage points without requiring camera inputs or post-hoc ICP refinement.

Eunsoo Im, Junghun Suh, Gyeonggwan Lee +1

Point Cloud Registration Point Cloud Learning LiDAR Localization Cross-Modal Knowledge Distillation

Ask PDF

Mar 15, 2026

RegFormer++: An Efficient Large-Scale 3D LiDAR Point Registration Network with Projection-Aware 2D Transformer

Although point cloud registration has achieved remarkable advances in object-level and indoor scenes, large-scale LiDAR registration methods has been rarely explored before. Challenges mainly arise from the huge point scale, complex point distribution, and numerous outliers within outdoor LiDAR scans. In addition, most existing registration works generally adopt a two-stage paradigm: They first find correspondences by extracting discriminative local descriptors and then leverage robust estimators (e.g. RANSAC) to filter outliers, which are highly dependent on well-designed descriptors and post-processing choices. To address these problems, we propose a novel end-to-end differential transformer network, termed RegFormer++, for large-scale point cloud alignment without requiring any further post-processing. Specifically, a hierarchical projection-aware 2D transformer with linear complexity is proposed to project raw LiDAR points onto a cylindrical surface and extract global point features, which can improve resilience to outliers due to long-range dependencies. Because we fill original 3D coordinates into 2D projected positions, our designed transformer can benefit from both high efficiency in 2D processing and accuracy from 3D geometric information. Furthermore, to effectively reduce wrong point matching, a Bijective Association Transformer (BAT) is designed, combining both cross attention and all-to-all point gathering. To improve training stability and robustness, a feature-transformed optimal transport module is also designed for regressing the final pose transformation. Extensive experiments on KITTI, NuScenes, and Argoverse datasets demonstrate that our model achieves state-of-the-art performance in terms of both accuracy and efficiency.

Jiuming Liu, Guangming Wang, Zhe Liu +7

School of Automation and Intelligent Sensing, State Key Laboratory of Avionics Integration and Aviation System-of-Systems Synthesis, Shanghai Jiao Tong University, Shanghai 200240, China. · Department of Engineering, University of Cambridge, UK. · China University of Mining and Technology. +4

Point Cloud Registration 3D ViTs

Ask PDF

Jul 16, 2026

Image-to-Point Cloud Registration Made Easy with Rectified Flow-based LiDAR Upsampling

Image-to-Point Cloud Registration (I2P) is essential for integrating camera and LiDAR in perception and autonomous systems, yet the modality gap between images and point clouds makes it difficult to achieve both high accuracy and strong generalization. In this paper, we propose a simple yet effective I2P method that treats LiDAR as an imaging sensor: from a single sparse LiDAR scan, we generate a dense LiDAR intensity image using Conditional Rectified Flow, match it with a camera image using a pre-trained feature matcher, and estimate the 6-DoF relative pose via PnP-RANSAC. The proposed model is pre-trained through a self-supervised image completion task and fine-tuned on a small amount of LiDAR data (neither image-point cloud pairs nor ground-truth sensor poses are required), enabling it to scale to diverse LiDAR and camera configurations. Experiments on the R3LIVE dataset show that the proposed method achieves a mean error of 4.89° / 1.63 m, outperforming existing methods, while completing a single registration in approximately 0.68 s.

Reon Tabata, Kenji Koide, Shuji Oishi +4

Department of Computer Science and Engineering, Toyohashi University of Technology, Toyohashi, Aichi, Japan · National Institute of Advanced Industrial Science and Technology, Tsukuba, Ibaraki, Japan

Camera-LiDAR Fusion Point Cloud Registration Cross-Modal Image Registration

Ask PDF

Abstract

Point Cloud Registration

Explore similar work

CVSD-Reg: Cross-Modal Visual Semantic Prior Distillation for Robust LiDAR Registration

RegFormer++: An Efficient Large-Scale 3D LiDAR Point Registration Network with Projection-Aware 2D Transformer

Image-to-Point Cloud Registration Made Easy with Rectified Flow-based LiDAR Upsampling