MeshBind: Point-to-Mesh Binding for Non-Rigid Scene Flow Automatic Annotation

Yinqi Chen; E Li; Shuaijun Wang; Weiyuan Peng; Meiying Zhang; Qi Hao

cs.CVOct 9, 2026

MeshBind: Point-to-Mesh Binding for Non-Rigid Scene Flow Automatic Annotation

Authors: Yinqi Chen, E Li, Shuaijun Wang, Weiyuan Peng, Meiying Zhang, Qi Hao

Abstract

Scene flow estimation is essential for many autonomous driving (AD) and robotic applications, but obtaining ground-truth scene flow for non-rigid objects such as pedestrians is highly difficult. We present MeshBind, an automatic annotation method that uses mesh models to generate per-point scene flow for non-rigid objects. The novelty of this work is threefold: 1) developing a multi-view module that reconstructs an articulated mesh model from six surround cameras and registers it to the LiDAR frame; 2) developing a local-frame binding technique that attaches each point to the mesh via a per-triangle tangent-normal frame, invariant under rigid motion yet adaptive to articulated deformation; 3) developing a three-layer flow registration pipeline that combines ego-vehicle motion compensation, box binding, and mesh binding to produce full-scene per-point flow. We implement MeshBind in SF-Studio, an interactive annotation tool that visualises flow and allows human mesh corrections with automatic recomputation. We also release SFB (SUScape-Flow Benchmark), a large-scale benchmark built on the public SUScape dataset, covering 41k frames with per-point non-rigid pedestrian annotations and a Rigid/Semi-Rigid/Non-Rigid evaluation protocol. Experiments show that fine-tuning on SFB reduces three-way endpoint error (EPE) by 2-3x, and brings non-rigid EPE to the same level as rigid EPE.

Figures & tables

Panel of Figure 1 — Fig. 1: Scene Flow Annotation for walking pedestrians (SUScape scene-001081, $\Delta t{=}0.5$ s, 105 returns). (a) Raw returns, coloured by height. (b) Binding distance: green $\leq 0.10$ m / orange $0.10$ – $0.25$ m. (c) The rigid convention gives every point red one translation; error concentrates at the swinging foot. (d) MeshBind separates a blue torso-aligned rigid part from a green non-rigid residual at the extremities, which deviates from the rigid flow by $0.30$ m on average (max $0.66$ m) over a $2.13$ m motion.

Fig. 2: MeshBind annotation pipeline. Top : the MeshBind stages—multi-view crop selection and mesh reconstruction, LiDAR-frame alignment, point-to-mesh binding with its reliability filter, and the mesh-based reconstruction that yields per-point non-rigid flow. Bottom : the three flow layers ( L1 Background , L2 Bounding box , L3 Non-rigid object ) produced by the method, carried in SF-Studio for review, together with the MeshBind-annotated dataset showing one instance per motion regime.

Fig. 3: Multi-view module: the camera mesh is generated and unmatched with the LiDAR points (a), rotated through the 96 candidate axis corrections $\theta=(d,\pi,\sigma)$ —reversing the direction $d$ , permuting the axes $\pi$ , flipping the signs $\sigma$ (b)—and matched to the best candidate $\theta^{*}$ (c), then propagated to the full point cloud (d).

Fig. 4: Binding distance on an example scene: each LiDAR point is coloured by its distance to the mesh surface, and points beyond the cut-off $d_{\max}{=}0.25$ m are excluded from binding.

Fig. 5: SF-Studio. Top : the three-tier layout with the dashed Feedback path returning the persisted flow to the editor. Bottom : browser-editor UI on a sample frame.

Fig. 6: Cross-modal projection consistency across four scenes (front-right camera, one pedestrian per row). Columns: RGB ; RAFT camera-motion-compensated optical flow; representative zero-shot predictions of Flow4D (feed-forward) and ICP-Flow (learning-free); rigid-box ; and MeshBind (Ours) projected flow in colour (hue = direction, brightness = magnitude). MeshBind renders limb-level articulation (multi-coloured), whereas rigid-box and the baselines appear single-colour.

Feed-forward (Argoverse-pretrained)
Methods	Fr.	FD $\downarrow$	BS $\downarrow$	FS $\downarrow$	3-way $\downarrow$	IoU $\uparrow$	Rig. $\downarrow$	Sem. $\downarrow$	NonR. $\downarrow$
DeFlow [ 32 ]	2	2.571	0.0099	0.123	0.901	0.339	2.378	0.981	0.418
DeltaFlow [ 33 ]	2	2.784	0.0046	0.053	0.947	0.237	2.602	0.980	0.329
FastFlow3D [ 15 ]	2	3.498	0.0204	0.259	1.259	0.269	3.312	1.021	0.461
Flow4D [ 34 ] (5-fr)	5	2.715	0.0140	0.153	0.961	0.459	2.545	0.965	0.334
SeFlow [ 6 ]	2	2.580	0.0557	0.287	0.974	0.272	2.407	0.945	0.428

TABLE I: Zero-shot scene-flow baselines on SUScape-Flow test. Top: eight feed-forward estimators pre-trained on Argoverse 2; bottom: three learning-free baselines. EPE in metres. Fr. = input sweeps; ‘—’ = full-scan optimisation. Column abbreviations follow Sec. V-B : FD/BS/FS , 3-way , Rig./Sem./NonR. Best in bold.

		Fine-tuned (FT), 30 ep from Argoverse weights								From-scratch (Sc), 15 ep from random init
Methods	Fr.	FD $\downarrow$	BS $\downarrow$	FS $\downarrow$	3-way $\downarrow$	IoU $\uparrow$	Rig. $\downarrow$	Sem. $\downarrow$	NonR. $\downarrow$	FD $\downarrow$	BS $\downarrow$	FS $\downarrow$	3-way $\downarrow$	IoU $\uparrow$	Rig. $\downarrow$	Sem. $\downarrow$	NonR. $\downarrow$
DeFlow [ 32 ]	2	0.203	0.0058	1.077	0.429	0.695	0.228	0.200	0.250	0.219	0.0062	1.090	0.439	0.686	0.242	0.217	0.255
DeltaFlow [ 33 ]	2	0.242	0.0093	1.087	0.446	0.546	0.269	0.209	0.241	0.236	0.0131	1.076	0.442	0.441	0.255	0.214	0.260
FastFlow3D [ 15 ]	2	0.267	0.0082	1.138	0.471	0.637	0.287	0.251	0.292	0.252	0.0078	1.147	0.469	0.647	0.271	0.262	0.315
Flow4D [ 34 ]	5	0.259	0.0161	1.125	0.467	0.496	0.286	0.240	0.259	0.333	0.0112	1.082	0.475	0.578	0.346	0.279	0.274
SeFlow [ 6 ]	2	0.201	0.0062	1.096	0.434	0.692	0.227	0.194	0.249	0.212	0.0063	1.120	0.446	0.684	0.236	0.225	0.258

TABLE II: Estimators trained on the SFB training split: FT = fine-tuned from Argoverse weights, Sc = trained from scratch (30 vs. 15 epochs). All EPE values in metres;

\downarrow

lower,

\uparrow

higher. Fr. = input sweeps. Column abbreviations follow Sec. V-B : FD/BS/FS, 3-way EPE, foreground IoU, and the motion-regime EPEs. Best per column in bold.

Perturbation	Coverage	Recovery drift (m)
(m)	(%)	No-residual	With-residual (Ours)
0.00	99.9	0.061	0.000
0.02	67.5	0.146	0.060
0.05	66.8	0.158	0.063
0.08	63.7	0.156	0.059
0.12	64.1	0.174	0.068

TABLE III: Residual tolerance under rigid mesh perturbation. “No-residual” forces

\mathbf{r}_{i,t}{=}0

; “With-residual” is the full MeshBind representation. Drift in metres (

\downarrow

lower).

Autonomous Driving

Autonomous driving develops vehicles that perceive road scenes, predict interactions, and plan and control safe movement with limited or no human input.

See all 538 papers

SF3D-RGB: Scene Flow Estimation from Monocular Camera and Sparse LiDAR
Feb 25, 2026Rajai Alhimdiat, Ramy Battrawy, René Schuster +2Scene Flow Estimation 3D Scene Understanding
Islamic University of Gaza, Gaza, Palestine · German Research Center for Artificial Intelligence (DFKI), Kaiserslautern, Germany
UniFlow: Zero-Shot LiDAR Scene Flow for Autonomous Vehicles
Nov 23, 2025Siyi Li, Qingwen Zhang, Ishan Khatri +4Point Cloud Learning Autonomous Driving Perception
University of Pennsylvania · KTH Royal Institute of Technology · StackAV +1
SynFlow: Scaling Up LiDAR Scene Flow Estimation with Synthetic Data
Apr 10, 2026Qingwen Zhang, Xiaomeng Zhu, Chenhan Jiang +1Synthetic Data Pretraining Synthetic Data Generation
KTH Royal Institute of Technology, Stockholm, Sweden · Hong Kong University of Science and Technology, Hong Kong, China

Feb 25, 2026J/K move · Enter open · S save

SF3D-RGB: Scene Flow Estimation from Monocular Camera and Sparse LiDAR

Rajai Alhimdiat, Ramy Battrawy, René Schuster, Didier Stricker+1

Islamic University of Gaza, Gaza, Palestine · German Research Center for Artificial Intelligence (DFKI), Kaiserslautern, Germany

Scene Flow Estimation 3D Scene Understanding Camera-LiDAR Fusion

Open paper AskPDF arXiv

Scene flow estimation is an extremely important task in computer vision to support the perception of dynamic changes in the scene. For robust scene flow, learning-based approaches have recently achieved impressive results using either image-based or LiDAR-based modalities. However, these methods have tended to focus on the use of a single modality. To tackle these problems, we present a deep learning architecture, SF3D-RGB, that enables sparse scene flow estimation using 2D monocular images and 3D point clouds (e.g., acquired by LiDAR) as inputs. Our architecture is an end-to-end model that first encodes information from each modality into features and fuses them together. Then, the fused features enhance a graph matching module for better and more robust mapping matrix computation to generate an initial scene flow. Finally, a residual scene flow module further refines the initial scene flow. Our model is designed to strike a balance between accuracy and efficiency. Furthermore, experiments show that our proposed method outperforms single-modality methods and achieves better scene flow accuracy on real-world datasets while using fewer parameters compared to other state-of-the-art methods with fusion.

Abstract

Autonomous Driving

Explore similar work

SF3D-RGB: Scene Flow Estimation from Monocular Camera and Sparse LiDAR

UniFlow: Zero-Shot LiDAR Scene Flow for Autonomous Vehicles

SynFlow: Scaling Up LiDAR Scene Flow Estimation with Synthetic Data