RVLoss: Runoff Vote Loss for Self-Supervised LiDAR Scene Flow Estimation

Shiming Wang; Liangliang Nan; Julian Kooij; Holger Caesar; Yancong Lin

cs.CVAug 19, 2026

RVLoss: Runoff Vote Loss for Self-Supervised LiDAR Scene Flow Estimation

Authors: Shiming Wang, Liangliang Nan, Julian Kooij, Holger Caesar, Yancong Lin

Organizations: TU Delft, the Netherlands · University of Nottingham, UK

Abstract

LiDAR scene flow estimates point-wise motion between two consecutive scans, referred to as the source and target. Leading self-supervised methods typically minimize the Chamfer loss, the nearest neighbor distance between the flow-compensated source and the target. However, nearest-neighbor search does not enforce motion rigidity, often leading to inconsistent flows within object instances. Existing approaches address this issue with additional regularization terms, but flow consistency among points remains limited, especially for large objects. We propose RVLoss, a self-supervised loss that incorporates motion rigidity by design through a runoff vote mechanism. Our key observation is that the point-wise motion, calculated from nearest neighbor search, can often be grouped into a small set of dominant flow candidates by voting (top-k voting). Furthermore, when compensating the source by these candidates, the flow that best represents the underlying rigid motion often yields the highest consensus after a second voting (top-1 voting). Based on this insight, we incorporate the two-stage runoff vote into loss design and create cluster-wise rigid flows and free-form flows as pseudo-labels for self-supervised learning. RVLoss can be seamlessly integrated into existing feedforward architectures. Experiments on the Argoverse2 2026 Challenge show that models trained with RVLoss achieve state-of-the-art performance among self-supervised approaches, outperforming baseline models trained with alternative loss designs by 20%. Moreover, cross-dataset evaluations demonstrate consistent performance improvements across four additional datasets. Code will be released upon acceptance.

Figures & tables

Figure 1: Overview of a runoff vote. a) A car moves rigidly and thus exhibits consistent flows among points. b) The widely used Chamfer distance breaks motion rigidity due to the local nearest neighbor search. As a result, multiple flow candidates emerge within the object. Although the correct flow is among these candidates, it may not be the top-voted one ( $1^{\text{st}}$ stage). c) We translate the source by those flow candidates and measure how well it aligns with the target by an additional voting ( $2^{\text{nd}}$ stage). The flow receiving the highest number of votes is selected as the rigid flow, representing the underlying motion of the car.

Figure 2: Overview of RVLoss. a) A typical feedforward scene flow estimator $\mathcal{G}_{\theta}$ takes as input a sequence of LiDAR scans ( $\geq 2$ frames), $\mathbf{P}_{0-h},\dots,\mathbf{P}_{0},\mathbf{P}_{1}$ , and predicts per-point flows $\mathbf{F}_{0}$ . b) RVLoss employs a runoff vote mechanism to estimate rigid and free-form flows over clustered points during each forward pass, and uses the resulting flows as pseudo-labels. The runoff vote consists of a two-stage voting procedure built upon a shared operation, termed Guided Rigid Flow Estimation (GRF) . Specifically, GRF first uses the predicted flow $\mathbf{F}_{0}$ as guidance to select the top- $k$ rigid flow candidates $\mathbf{T}^{{}^{\prime}}_{0}$ (Stage 1). Subsequently, GRF takes these top- $k$ candidates as guidance and identifies the top- $1$ rigid flow $\mathbf{T}^{{}^{\prime\prime}}_{0}$ (Stage 2). In addition to rigid flow, RVLoss also computes a free-form flow $\mathbf{F}^{{}^{\prime\prime}}_{0}$ as complementary supervision to relax the rigid-motion assumption and to mitigate the quantization error introduced by voting. The objective is to minimize the residual between the predicted flow and two supervision flows: the top- $1$ rigid flow and the free-form flow.

Figure 3: Overview of GRF. a) Guided NN Search first translates the source points using the guidance flow and then finds their nearest neighbors in the target point cloud. The point-wise motion between each source point and its matched target point is used for the subsequent processing. b) Vote for Rigid Motion quantizes the point-wise flows within each cluster and casts them into a predefined voting space. The top- $k$ flow candidates are then selected and assigned to all points within the cluster.

Backbone	Loss	#F input	#F loss	RT (s)	Dynamic Bucket-Normalized $\downarrow$					Three-way EPE (cm) $\downarrow$
Backbone	Loss	#F input	#F loss	RT (s)	Mean	CAR	O.V.	PED.	VRU	Mean	FD	FS	BS
Supervised methods
DeltaFlow	GT	5	-	8	0.103	0.083	0.132	0.122	0.075	1.76	3.16	1.31	0.81
SSF	GT	2	-	-	0.160	0.115	0.170	0.217	0.137	2.36	4.71	1.69	0.67
Self-supervised methods
DeFlow	SeFlow	2	2	7.2	0.313	0.236	0.295	0.441	0.273	4.76	10.93	2.23	1.12

Table 1: In-domain performance on the AV2 test split. Numerical results on scene flow prediction come from the AV2 2026 Scene Flow Challenge leaderboard. ‘#F input ’ denotes the number of input frames to the network. ‘#F loss ’ denotes the number of input frames during loss calculation. RT stands for runtime per sequence (around 157 frames) in seconds, retrieved from TeFlow [ 10 ] . ( Best , Second Best )

Figure 4: Cross-domain evaluation performance. We report the mean Dynamic Bucket-Normalized EPE on multiple datasets used in the AV2 2026 scene flow challenge [ 11 ] . DeltaFlow+RVLoss demonstrates consistent performance improvement over all baselines.

Figure 5: Qualitative comparison . Visualization of scene flow predictions from the DeltaFlow model trained with the TeFlow loss and the proposed RVLoss. Point colors denote flow direction, and color saturation indicates flow magnitude. Objects are expected to exhibit coherent color patterns. Compared with the TeFlow [ 10 ] loss, RVLoss produces more uniform coloring within each object (highlighted in boxes), demonstrating more coherent scene flow estimation.

Free-form Loss	Rigid Loss	Bidirect Optim	Dynamic Bucket-Normalized $\downarrow$					Three-way EPE (cm) $\downarrow$
Free-form Loss	Rigid Loss	Bidirect Optim	Mean	CAR	O.V.	PD.	VRU	Mean	FD	FS	BS
✓	✓	✗	0.284	0.129	0.267	0.442	0.299	3.59	7.31	1.99	1.48
✗	✓	✓	0.338	0.138	0.268	0.604	0.346	3.70	8.27	1.69	1.14
✓	✗	✓	0.275	0.139	0.266	0.405	0.288	4.68	7.48	3.62	2.94
✓	✓	✓	0.262	0.121	0.246	0.405	0.277	3.47	6.84	2.05	1.53

Table 2: Ablation of each loss component on the Argoverse 2 val split [ 6 ] . ( Best , Second Best )

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Loss	nuScenes [ 53 ]					Waymo [ 24 ]
Method	Loss	Mean	CAR	O.V.	PD.	VRU	Mean	CAR	PD.	VRU
DeFlow	SeFlow	0.544	0.396	0.653	0.726	0.419	0.351	0.212	0.551	0.289
VoteFlow	SeFlow	0.538	0.355	0.605	0.780	0.410	0.347	0.197	0.548	0.298
DeFlow $++$	SeFlow $++$	0.509	0.327	0.583	0.716	0.409	0.323	0.201	0.521	0.247
DeltaFlow	TeFlow	0.395	0.303	0.461	0.474	0.344	0.275	0.157	0.469	0.195
DeltaFlow	TeFlow (Re)	0.433	0.347	0.492	0.628	0.266	0.285	0.164	0.486	0.206

Appendix

Table 3: Dynamic Bucket-Normalized EPE (

\downarrow

) on the nuScenes [ 53 ] val set and the Waymo [ 24 ] valid set. ( Best , Second Best )

Stage 1	Stage 2	Dynamic Bucket-Normalized $\downarrow$					Three-way EPE (cm) $\downarrow$
Stage 1	Stage 2	Mean	CAR	O.V.	PD.	VRU	Mean	FD	FS	BS
-	1	0.499	0.373	0.667	0.540	0.418	7.19	18.5	1.81	1.29
3	1	0.317	0.176	0.303	0.474	0.316	4.44	9.84	2.03	1.47
5	1	0.277	0.137	0.260	0.418	0.291	3.73	7.75	1.98	1.46
10	1	0.262	0.121	0.246	0.405	0.277	3.47	6.84	2.05	1.53
20	1	0.297	0.127	0.276	0.500	0.285	3.67	7.41	2.04	1.57

Appendix

Table 4: Ablation of

k

in voting modules on the Argoverse 2 val split [ 6 ] . ( Best , Second Best )

Method	Loss	Predicted flow $\downarrow$					(Refined) rigid flow $\downarrow$					(Refined) free-form flow $\downarrow$
Method	Loss	Mean	CAR	O.V.	PD.	VRU	Mean	CAR	O.V.	PD.	VRU	Mean	CAR	O.V.	PD.	VRU
DeFlow	SeFlow	0.342	0.205	0.340	0.510	0.315	0.237	0.140	0.273	0.348	0.189	0.293	0.194	0.294	0.411	0.272
DeFlow $++$	SeFlow $++$	0.300	0.180	0.340	0.395	0.288	0.236	0.131	0.260	0.363	0.189	0.289	0.187	0.282	0.416	0.270
DeltaFlow	TeFlow	0.223	0.140	0.268	0.304	0.179	0.220	0.119	0.243	0.354	0.162	0.278	0.178	0.267	0.414	0.255

Appendix

Table 5: RVLoss as offline Flow Refinement. We use RVLoss to refine the predicted flows and thus obtain rigid flows and free-form flows. We compare these flows on the AV2 val split [ 6 ] using Dynamic Bucket-Normalized EPE. ( Best / Second Best in each row and category)

Self-Supervised Learning

Self-supervised learning uses structure within unlabeled data to train representations and models for downstream tasks.

See all 259 papers

UniFlow: Zero-Shot LiDAR Scene Flow for Autonomous Vehicles
Nov 23, 2025Siyi Li, Qingwen Zhang, Ishan Khatri +4Point Cloud Learning Autonomous Driving Perception
University of Pennsylvania · KTH Royal Institute of Technology · StackAV +1
Motion Cues from Image-based Point Tracking for LiDAR Scene Flow Estimation
May 16, 2026Youngdong Jang, Gyeongrok Oh, Jong Wook Kim +6Point Tracking Motion Segmentation
Korea University · Purdue University · KAIST +1
CorrelationFlow: A Training-Free Geometric Approach for LiDAR Scene Flow Estimation
Jul 31, 2026Minh-Quan Dao, Yancong Lin, Shiming Wang +2Scene Flow Estimation
Nantes Université, LS2N · University of Nottingham · TU Delft +1

Nov 23, 2025J/K move · Enter open · S save

UniFlow: Zero-Shot LiDAR Scene Flow for Autonomous Vehicles

Siyi Li, Qingwen Zhang, Ishan Khatri, Kyle Vedder+3

University of Pennsylvania · KTH Royal Institute of Technology · StackAV+1

Point Cloud Learning Autonomous Driving Perception Cross-Domain Generalization 3D Scene Understanding

Open paper AskPDF arXiv

LiDAR scene flow is the task of estimating per-point 3D motion between consecutive point clouds. Recent methods achieve centimeter-level accuracy on popular autonomous vehicle (AV) datasets, but are typically only trained and evaluated on a single sensor. In this paper, we aim to learn general motion priors that transfer to diverse and unseen LiDAR sensors. However, prior work in LiDAR semantic segmentation and 3D object detection demonstrate that naively training on multiple datasets yields worse performance than single dataset models. Interestingly, we find that this conventional wisdom does not hold for motion estimation, and that state-of-the-art scene flow methods greatly benefit from cross-dataset training without architectural modification. We posit that low-level tasks such as motion estimation may be less sensitive to sensor configuration; indeed, our analysis shows that models trained on fast-moving objects (e.g., from highway datasets) perform well on fast-moving objects, even across different datasets. Informed by our analysis, we propose UniFlow, a feedforward model that unifies and trains on multiple large-scale LiDAR scene flow datasets with diverse sensor placements and point cloud densities. Our frustratingly simple solution establishes a new state-of-the-art on Waymo and nuScenes, improving over prior work by 5.1% and 35.2% respectively. Moreover, UniFlow achieves state-of-the-art accuracy on unseen datasets like TruckScenes and AEVAScenes, outperforming prior dataset-specific models by 30.1% and 22.5% respectively.

Abstract

Self-Supervised Learning

Explore similar work

UniFlow: Zero-Shot LiDAR Scene Flow for Autonomous Vehicles

Motion Cues from Image-based Point Tracking for LiDAR Scene Flow Estimation

CorrelationFlow: A Training-Free Geometric Approach for LiDAR Scene Flow Estimation