Scene flow estimation is essential for many autonomous driving (AD) and robotic applications, but obtaining ground-truth scene flow for non-rigid objects such as pedestrians is highly difficult. We present MeshBind, an automatic annotation method that uses mesh models to generate per-point scene flow for non-rigid objects. The novelty of this work is threefold: 1) developing a multi-view module that reconstructs an articulated mesh model from six surround cameras and registers it to the LiDAR frame; 2) developing a local-frame binding technique that attaches each point to the mesh via a per-triangle tangent-normal frame, invariant under rigid motion yet adaptive to articulated deformation; 3) developing a three-layer flow registration pipeline that combines ego-vehicle motion compensation, box binding, and mesh binding to produce full-scene per-point flow. We implement MeshBind in SF-Studio, an interactive annotation tool that visualises flow and allows human mesh corrections with automatic recomputation. We also release SFB (SUScape-Flow Benchmark), a large-scale benchmark built on the public SUScape dataset, covering 41k frames with per-point non-rigid pedestrian annotations and a Rigid/Semi-Rigid/Non-Rigid evaluation protocol. Experiments show that fine-tuning on SFB reduces three-way endpoint error (EPE) by 2-3x, and brings non-rigid EPE to the same level as rigid EPE.
Figures & tables
Fig. 1: Scene Flow Annotation for walking pedestrians (SUScape scene-001081, Δt=0.5 s, 105 returns). (a) Raw returns, coloured by height. (b) Binding distance: green ≤0.10 m / orange 0.10 – 0.25 m. (c) The rigid convention gives every point red one translation; error concentrates at the swinging foot. (d) MeshBind separates a blue torso-aligned rigid part from a green non-rigid residual at the extremities, which deviates from the rigid flow by 0.30 m on average (max 0.66 m) over a 2.13 m motion.
Fig. 2: MeshBind annotation pipeline. Top : the MeshBind stages—multi-view crop selection and mesh reconstruction, LiDAR-frame alignment, point-to-mesh binding with its reliability filter, and the mesh-based reconstruction that yields per-point non-rigid flow. Bottom : the three flow layers ( L1 Background , L2 Bounding box , L3 Non-rigid object ) produced by the method, carried in SF-Studio for review, together with the MeshBind-annotated dataset showing one instance per motion regime.
Fig. 3: Multi-view module: the camera mesh is generated and unmatched with the LiDAR points (a), rotated through the 96 candidate axis corrections θ=(d,π,σ) —reversing the direction d , permuting the axes π , flipping the signs σ (b)—and matched to the best candidate θ∗ (c), then propagated to the full point cloud (d).
Fig. 4: Binding distance on an example scene: each LiDAR point is coloured by its distance to the mesh surface, and points beyond the cut-off dmax=0.25 m are excluded from binding.
Fig. 5: SF-Studio. Top : the three-tier layout with the dashed Feedback path returning the persisted flow to the editor. Bottom : browser-editor UI on a sample frame.
Fig. 6: Cross-modal projection consistency across four scenes (front-right camera, one pedestrian per row). Columns: RGB ; RAFT camera-motion-compensated optical flow; representative zero-shot predictions of Flow4D (feed-forward) and ICP-Flow (learning-free); rigid-box ; and MeshBind (Ours) projected flow in colour (hue = direction, brightness = magnitude). MeshBind renders limb-level articulation (multi-coloured), whereas rigid-box and the baselines appear single-colour.
Methods
Fr.
FD ↓
BS ↓
FS ↓
3-way ↓
IoU ↑
Rig. ↓
Sem. ↓
NonR. ↓
Feed-forward (Argoverse-pretrained)
DeFlow [ 32 ]
2
2.571
0.0099
0.123
0.901
0.339
2.378
0.981
0.418
DeltaFlow [ 33 ]
2
2.784
0.0046
0.053
0.947
0.237
2.602
0.980
0.329
FastFlow3D [ 15 ]
2
3.498
0.0204
0.259
1.259
0.269
3.312
1.021
0.461
Flow4D [ 34 ] (5-fr)
5
2.715
0.0140
0.153
0.961
0.459
2.545
0.965
0.334
SeFlow [ 6 ]
2
2.580
0.0557
0.287
0.974
0.272
2.407
0.945
0.428
TABLE I: Zero-shot scene-flow baselines on SUScape-Flow test. Top: eight feed-forward estimators pre-trained on Argoverse 2; bottom: three learning-free baselines. EPE in metres. Fr. = input sweeps; ‘—’ = full-scan optimisation. Column abbreviations follow Sec. V-B : FD/BS/FS , 3-way , Rig./Sem./NonR. Best in bold.
Fine-tuned (FT), 30 ep from Argoverse weights
From-scratch (Sc), 15 ep from random init
Methods
Fr.
FD ↓
BS ↓
FS ↓
3-way ↓
IoU ↑
Rig. ↓
Sem. ↓
NonR. ↓
FD ↓
BS ↓
FS ↓
3-way ↓
IoU ↑
Rig. ↓
Sem. ↓
NonR. ↓
DeFlow [ 32 ]
2
0.203
0.0058
1.077
0.429
0.695
0.228
0.200
0.250
0.219
0.0062
1.090
0.439
0.686
0.242
0.217
0.255
DeltaFlow [ 33 ]
2
0.242
0.0093
1.087
0.446
0.546
0.269
0.209
0.241
0.236
0.0131
1.076
0.442
0.441
0.255
0.214
0.260
FastFlow3D [ 15 ]
2
0.267
0.0082
1.138
0.471
0.637
0.287
0.251
0.292
0.252
0.0078
1.147
0.469
0.647
0.271
0.262
0.315
Flow4D [ 34 ]
5
0.259
0.0161
1.125
0.467
0.496
0.286
0.240
0.259
0.333
0.0112
1.082
0.475
0.578
0.346
0.279
0.274
SeFlow [ 6 ]
2
0.201
0.0062
1.096
0.434
0.692
0.227
0.194
0.249
0.212
0.0063
1.120
0.446
0.684
0.236
0.225
0.258
TABLE II: Estimators trained on the SFB training split: FT = fine-tuned from Argoverse weights, Sc = trained from scratch (30 vs. 15 epochs). All EPE values in metres; ↓ lower, ↑ higher. Fr. = input sweeps. Column abbreviations follow Sec. V-B : FD/BS/FS, 3-way EPE, foreground IoU, and the motion-regime EPEs. Best per column in bold.
Perturbation
Coverage
Recovery drift (m)
(m)
(%)
No-residual
With-residual (Ours)
0.00
99.9
0.061
0.000
0.02
67.5
0.146
0.060
0.05
66.8
0.158
0.063
0.08
63.7
0.156
0.059
0.12
64.1
0.174
0.068
TABLE III: Residual tolerance under rigid mesh perturbation. “No-residual” forces ri,t=0 ; “With-residual” is the full MeshBind representation. Drift in metres ( ↓ lower).
Autonomous driving develops vehicles that perceive road scenes, predict interactions, and plan and control safe movement with limited or no human input.
Scene flow estimation is an extremely important task in computer vision to support the perception of dynamic changes in the scene. For robust scene flow, learning-based approaches have recently achieved impressive results using either image-based or LiDAR-based modalities. However, these methods have tended to focus on the use of a single modality. To tackle these problems, we present a deep learning architecture, SF3D-RGB, that enables sparse scene flow estimation using 2D monocular images and 3D point clouds (e.g., acquired by LiDAR) as inputs. Our architecture is an end-to-end model that first encodes information from each modality into features and fuses them together. Then, the fused features enhance a graph matching module for better and more robust mapping matrix computation to generate an initial scene flow. Finally, a residual scene flow module further refines the initial scene flow. Our model is designed to strike a balance between accuracy and efficiency. Furthermore, experiments show that our proposed method outperforms single-modality methods and achieves better scene flow accuracy on real-world datasets while using fewer parameters compared to other state-of-the-art methods with fusion.
Rajai Alhimdiat, Ramy Battrawy, René Schuster +2
Islamic University of Gaza, Gaza, Palestine · German Research Center for Artificial Intelligence (DFKI), Kaiserslautern, Germany
LiDAR scene flow is the task of estimating per-point 3D motion between consecutive point clouds. Recent methods achieve centimeter-level accuracy on popular autonomous vehicle (AV) datasets, but are typically only trained and evaluated on a single sensor. In this paper, we aim to learn general motion priors that transfer to diverse and unseen LiDAR sensors. However, prior work in LiDAR semantic segmentation and 3D object detection demonstrate that naively training on multiple datasets yields worse performance than single dataset models. Interestingly, we find that this conventional wisdom does not hold for motion estimation, and that state-of-the-art scene flow methods greatly benefit from cross-dataset training without architectural modification. We posit that low-level tasks such as motion estimation may be less sensitive to sensor configuration; indeed, our analysis shows that models trained on fast-moving objects (e.g., from highway datasets) perform well on fast-moving objects, even across different datasets. Informed by our analysis, we propose UniFlow, a feedforward model that unifies and trains on multiple large-scale LiDAR scene flow datasets with diverse sensor placements and point cloud densities. Our frustratingly simple solution establishes a new state-of-the-art on Waymo and nuScenes, improving over prior work by 5.1% and 35.2% respectively. Moreover, UniFlow achieves state-of-the-art accuracy on unseen datasets like TruckScenes and AEVAScenes, outperforming prior dataset-specific models by 30.1% and 22.5% respectively.
Siyi Li, Qingwen Zhang, Ishan Khatri +4
University of Pennsylvania · KTH Royal Institute of Technology · StackAV +1
Reliable 3D dynamic perception requires models that can anticipate motion beyond predefined categories, yet progress is hindered by the scarcity of dense, high-quality motion annotations. While self-supervision on unlabeled real data offers a path forward, empirical evidence suggests that scaling unlabeled data fails to close the performance gap due to noisy proxy signals. In this paper, we propose learning robust real-world motion priors entirely from scalable simulation. We introduce SynFlow, a data generation pipeline for large-scale synthetic LiDAR scene flow. Unlike prior works that prioritize sensor-specific realism, SynFlow employs a motion-oriented strategy to synthesize diverse kinematic patterns across 4,000 sequences (∼940k frames), termed SynFlow-4k. This represents a 34× scale-up in annotated volume over existing real-world benchmarks. Our experiments demonstrate that SynFlow-4k provides a highly domain-invariant motion prior. In a zero-shot regime, models trained only on our synthetic data generalize across multiple real-world benchmarks, comparable to in-domain supervised baselines on nuScenes and outperforming state-of-the-art methods on TruckScenes by 31.8%. Furthermore, SynFlow-4k serves as a label-efficient foundation: fine-tuning with only 5% of real-world labels surpasses models trained from scratch on the full available budget. We open-source the pipeline and dataset to facilitate research in generalizable 3D motion estimation. More detail can be found at https://kin-zhang.github.io/SynFlow.
Qingwen Zhang, Xiaomeng Zhu, Chenhan Jiang +1
KTH Royal Institute of Technology, Stockholm, Sweden · Hong Kong University of Science and Technology, Hong Kong, China