Organizations: KTH Royal Institute of Technology · Hong Kong University of Science and Technology · Scania · University of Warwick · Linköping University
Self-supervised feed-forward methods for scene flow estimation offer real-time efficiency, but their supervision from two-frame point correspondences is unreliable and often breaks down under occlusions. Multi-frame supervision has the potential to provide more stable guidance by incorporating motion cues from past frames, yet naive extensions of two-frame objectives are ineffective because point correspondences vary abruptly across frames, producing inconsistent signals. In the paper, we present TeFlow, enabling multi-frame supervision for feed-forward models by mining temporally consistent supervision. TeFlow introduces a temporal ensembling strategy that forms reliable supervisory signals by aggregating the most temporally consistent motion cues from a candidate pool built across multiple frames. Extensive evaluations demonstrate that TeFlow establishes a new state-of-the-art for self-supervised feed-forward methods, achieving performance gains of up to 33% on the challenging Argoverse 2 and nuScenes datasets. Our method performs on par with leading optimization-based methods, yet speeds up 150 times. The code is open-sourced at https://github.com/Kin-Zhang/TeFlow along with trained model weights.
Figures & tables
Figure 1: (a) Multi-frame supervision maintains more stable guidance during occlusion by querying past frames, while two-frame supervision fails due to missing points. (b) Temporal consistency of supervision signals, measured as the change in motion direction (in degrees) between consecutive frames. The two-frame supervision [ 42 ] exhibits abrupt variations with frequent direction shifts, while our five-frame TeFlow produces more stable signals that stay closer to the ground truth.
Figure 2: Accuracy vs. Runtime. Prior feed-forward methods are fast but less accurate, while optimization-based methods are accurate but too slow. TeFlow achieves both real-time speed and high accuracy.
Figure 3: An overview of the TeFlow , a multi-frame feedforward scene flow estimation pipeline, shown in the top row. Our self-supervised pipeline tackles the main challenge of deriving reliable supervision fˉ from dense multi-frame inputs. For each dynamic cluster Cj , we constructs a motion candidate pool (internal f^Cj and external fCj,k ). Candidates are processed via a weighted consensus voting scheme using directional consistency M and magnitude-based reliability w to find a consensus winner ( Equation 7 ). The final supervision fˉCj is a weighted average of the winner and agreeing candidates, which filters inconsistent outliers (e.g., f^Cj,1 ) for stable training.
Methods
#F
Runtime per seq
Three-way EPE (cm) ↓
Dynamic Bucket-Normalized ↓
Mean
FD
FS
BS
Mean
CAR
OTHER
PED.
VRU
Ego Motion Flow
-
-
18.13
53.35
1.03
0.00
1.000
1.000
1.000
1.000
1.000
Optimization-based
FastNSF [ 22 ]
2
12m
11.18
16.34
8.14
9.07
0.383
0.296
0.413
0.500
0.322
NSFP [ 21 ]
2
60m
6.06
11.58
3.16
3.44
0.422
0.251
0.331
0.722
0.383
ICP-Flow [ 24 ]
2
-
6.50
13.69
3.32
2.50
0.331
0.195
0.331
0.435
0.363
Table 1: Performance comparisons on the Argoverse 2 test set leaderboard [ 1 ] . TeFlow achieves state-of-the-art performance in real-time scene flow estimation. ‘#F’ denotes the number of input frames. Runtime is reported per sequence (around 157 frames), with ‘-’ indicating unreported values. Units are given in seconds (‘s’) and minutes (‘m’).
Methods
#F
Runtime per seq
Three-way EPE (cm) ↓
Dynamic Bucket-Normalized ↓
Mean
FD
FS
BS
Mean
CAR
OTHER
PED.
VRU
Ego Motion Flow
-
-
12.34
35.94
1.07
0.00
1.000
1.000
1.000
1.000
1.000
Optimization-based
NSFP [ 21 ]
2
3.5m
10.79
20.26
4.88
7.23
0.602
0.463
0.456
0.829
0.662
ICP-Flow [ 24 ]
2
3.2m
8.81
17.53
3.51
5.38
0.569
0.430
0.569
0.749
0.530
FastNSF [ 22 ]
2
2.6m
12.16
18.20
6.11
12.18
0.560
0.436
0.523
0.737
0.543
Table 2: Performance comparisons on the nuScenes validation set with a 10Hz LiDAR frequency. TeFlow achieves state-of-the-art accuracy in scene flow estimation. Runtime is reported per sequence ( ≈ 200 frames) using the same device.
Loss Type
#Frame
Dynamic Bucket-Normalized ↓
Three-way EPE (cm) ↓
Mean
CAR
OTHER
PED.
VRU
Mean
FD
FS
BS
SeFlow
2
0.408
0.319
0.412
0.369
0.531
6.35
16.63
1.48
0.92
TeFlow
2
0.353
0.271
0.389
0.329
0.424
5.98
13.93
2.53
1.46
4
0.283
0.204
0.342
0.295
0.293
4.57
10.77
1.87
1.08
5
0.265
0.198
0.275
0.295
0.293
4.43
10.36
1.86
1.08
6
0.269
0.197
0.305
0.290
0.284
4.55
10.66
1.87
1.12
Table 3: Ablation on the number of input frames on the Argoverse 2 validation set. All experiments use the same DeltaFlow backbone for a fair comparison. TeFlow surpasses SeFlow even with two frames, and performance peaks at five frames, indicating the optimal temporal window. The best results are shown in bold .
Loss item
Dynamic Bucket-Normalized ↓
Three-way EPE (cm) ↓
Lgemo
Lstatic
Ldcls
Mean
CAR
OTHER
PED.
VRU
Mean
FD
FS
BS
✓
0.386
0.317
0.586
0.297
0.343
8.85
17.26
4.45
4.85
✓
✓
0.458
0.321
0.654
0.481
0.377
6.37
17.15
1.25
0.73
✓
0.303
0.254
0.310
0.285
0.362
8.53
12.28
7.17
6.14
✓
✓
0.313
0.233
0.402
0.296
0.321
4.84
11.99
1.73
0.80
✓
✓
✓
0.265
0.198
0.275
0.295
0.293
4.43
10.36
1.86
1.08
Table 4: Ablation study of proposed self-supervised loss items (See Equation 10 ). Results are evaluated on the Argoverse 2 validation set using default hyperparameters. Bold indicates the best performance and purple highlights settings with a significant performance drop.
Formulation
Dynamic Bucket-Normalized ↓
Mean
CAR
OTHER
PED.
VRU
Point-level
0.351
0.258
0.331
0.352
0.463
Cluster-level
0.356
0.222
0.603
0.284
0.316
Ldcls (Ours)
0.265
0.198
0.275
0.295
0.293
Table 5: Ablation study of our dynamic cluster loss (See Equation 9 ). Results are evaluated on the Argoverse 2 validation set.
Figure 4: Qualitative results on Argoverse 2 (left) and nuScenes (right). Rows show ground truth, SeFlow, and TeFlow predictions across time. Scene flow is visualized with hue indicating direction and saturation representing speed. Compared to SeFlow, TeFlow produces flow estimates that are more accurate and temporally consistent, particularly for dynamic objects (red circles).
Candidates Pool
Dynamic Bucket-Normalized ↓
Mean
CAR
OTHER
PED.
VRU
Internal Only
0.455
0.232
0.481
0.505
0.601
External Only
0.321
0.278
0.403
0.281
0.321
Both (Proposed)
0.265
0.198
0.275
0.295
0.293
Table 6: Ablation study of our candidate pool (See Equation 4 ). Results are evaluated on the Argoverse 2 validation set.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Methods
#F
Runtime per seq
Dynamic Bucket-Normalized ↓
Three-way EPE (cm) ↓
Mean
CAR
PED
VRU
Mean
FD
FS
BS
Ego Motion Flow
-
-
1.000
1.000
1.000
1.000
17.10
50.85
0.46
0.00
Optimization-based
NSFP [ 21 ]
2
96m
0.574
0.315
0.823
0.584
10.05
17.12
10.81
2.21
ICP-Flow [ 24 ]
2
11m
0.328
0.305
0.485
0.195
8.50
20.81
2.14
2.57
FastNSF [ 22 ]
2
6.7m
0.458
0.236
0.719
0.418
9.24
18.19
2.56
6.98
Appendix
Table 7: Performance comparisons on the Waymo validation set . TeFlow achieves state-of-the-art accuracy in scene flow estimation. Runtime is reported per sequence ( ≈ 200 frames) using the same device. The best results are shown in bold .
Backbone
Dynamic Bucket-Normalized ↓
Three-way EPE (cm) ↓
Mean
CAR
OTHER
PED.
VRU
Mean
FD
FS
BS
Flow4D
0.330
0.254
0.326
0.329
0.411
5.70
12.98
2.67
1.46
Δ Flow
0.265
0.198
0.275
0.295
0.293
4.43
10.36
1.86
1.08
Appendix
Table 8: Ablation study on different multi-frame backbones within our self-supervised pipeline TeFlow. Results are evaluated on the Argoverse 2 validation set with five input frames. The results demonstrate that our self-supervised objective integrates seamlessly with different multi-frame backbones (Flow4D and Δ Flow), showing that the method is architecture-agnostic and applicable to future multi-frame scene flow designs.
Exp. Id
Ablation Variant
Dynamic Bucket-Normalized ↓
Three-way EPE (cm) ↓
Mean
CAR
OTHER
PED.
VRU
Mean
FD
FS
BS
1
w/o M (i.e., M=1 )
0.349
0.301
0.408
0.292
0.394
5.81
15.33
1.34
0.74
2
w/o w (i.e., w=1 )
0.271
0.199
0.300
0.285
0.302
5.70
12.98
2.67
1.46
3
w/o Aggregation
0.289
0.226
0.332
0.303
0.295
4.61
11.55
1.47
0.82
4
TeFlow (Ours)
0.265
0.198
0.275
0.295
0.293
4.43
10.36
1.86
1.08
Appendix
Table 9: Ablation of the candidate voting and aggregation pipeline. We evaluate three components: the directional consistency agreement matrix M in Equation 5 , the reliability weights w in Equation 6 , and the aggregation step in Equation 8 . Removing either directional consistency or reliability weighting degrades performance, with M playing the dominant role in stabilizing candidate votes. Skipping aggregation further reduces accuracy by discarding supportive consistent candidates. Results on the Argoverse 2 validation set demonstrate that the full pipeline is essential for producing reliable multi-frame supervision.
TeFlow Setting
Dynamic Bucket-Normalized ↓
Three-way EPE (cm) ↓
Mean
CAR
OTHER
PED.
VRU
Mean
FD
FS
BS
Default
0.265
0.198
0.275
0.295
0.293
4.43
10.36
1.86
1.08
τcos=0 (90°)
0.307
0.239
0.365
0.291
0.332
5.19
13.02
1.60
0.95
τcos=0.9 (20°)
0.289
0.207
0.356
0.294
0.297
4.42
10.41
1.80
1.04
K=20
0.353
0.283
0.355
0.312
0.463
5.88
14.97
1.68
1.00
K=10
0.307
0.241
0.314
0.296
0.377
5.11
12.39
1.83
1.12
Appendix
Table 10: Ablation study on the key hyperparameters of TeFlow, evaluated on the Argoverse 2 validation set. The default and best-performing configuration is cosine similarity τcos=0.7 (45°), Top-K = 5, and time decay γ=0.9 . In each row, only the specified parameter is varied from this setting.
Figure 5: Qualitative comparisons on the Argoverse 2 validation set. Left: A multi-vehicle scene. Right: A vehicle stopping for pedestrians. Our method robustly handles both scenarios, unlike the baseline. (Best viewed in color.) The scenes correspond to scene IDs ‘c85a88a8-c916-30a7-923c-0c66bd3ebbd3’ and ‘b6500255-eba3-3f77-acfd-626c07aa8621’.
Figure 6: Qualitative results on the Argoverse 2 validation set. Our method accurately captures the motion of multiple pedestrians, while all feed-forward baselines underestimate the flows of moving pedestrians. (Best viewed in color.) The scenes correspond to scene IDs ‘9f871fb4-3b8e-34b3-9161-ed961e71a6da’.
Figure 7: Qualitative results on the nuScenes validation set. On this sparser data, TeFlow provides complete motion for the vehicle and detects the pedestrians, whereas the baseline underestimates the car’s flow and misses the smaller actors. (Best viewed in color.) The scenes correspond to the scene IDs ‘scene-0025’.
Figure 8: Qualitative results on the Argoverse 2 validation set. Our method accurately captures the motion of vehicles in complex roundabout scenarios. (Best viewed in color.) The scenes correspond to scene IDs ‘bdb9d309-f14b-3ff6-ad1f-5d3f3f95a13e’.
Monocular scene flow estimation aims to recover dense 3D motion from image sequences, yet most existing methods are limited to two-frame inputs, restricting temporal modeling and robustness to occlusions. We propose RAFT-MSF++, a self-supervised multi-frame framework that recurrently fuses temporal features to jointly estimate depth and scene flow. Central to our approach is the Geometry-Motion Feature (GMF), which compactly encodes coupled motion and geometry cues and is iteratively updated for effective temporal reasoning. To ensure the robustness of this temporal fusion against occlusions, we incorporate relative positional attention to inject spatial priors and an occlusion regularization module to propagate reliable motion from visible regions. These components enable the GMF to effectively propagate information even in ambiguous areas. Extensive experiments show that RAFT-MSF++ achieves 24.14% SF-all on the KITTI Scene Flow benchmark, with a 30.99% improvement over the baseline and better robustness in occluded regions. The code is available at https://github.com/sunzunyi/RAFT-MSF-PlusPlus.
Xunpei Sun, Zuoxun Hou, Yi Chang +2
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China 510006. · Beijing Institute of Space Mechanics and Electricity, Beijing, China 100094. · School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China 430074.
Unsupervised optical flow methods typically lack reliable uncertainty estimation, limiting their robustness and interpretability. We propose U2Flow, the first recurrent unsupervised framework that jointly estimates optical flow and per-pixel uncertainty. The core innovation is a decoupled learning strategy that derives uncertainty supervision from augmentation consistency via a Laplace-based maximum likelihood objective, enabling stable training without ground truth. The predicted uncertainty is further integrated into the network to guide adaptive flow refinement and dynamically modulate the regional smoothness loss. Furthermore, we introduce an uncertainty-guided bidirectional flow fusion mechanism that enhances robustness in challenging regions. Extensive experiments on KITTI and Sintel demonstrate that U2Flow achieves state-of-the-art performance among unsupervised methods while producing highly reliable uncertainty maps, validating the effectiveness of our joint estimation paradigm. The code is available at https://github.com/sunzunyi/U2FLOW.
Xunpei Sun, Wenwei Lin, Yi Chang +1
Sun Yat-sen University, Guangzhou, China · Huazhong University of Science and Technology, Wuhan, China
Optical flow estimation is a fundamental problem in computer vision, yet the reliance on expensive ground-truth annotations limits the scalability of supervised approaches. Although unsupervised and semi-supervised methods alleviate this issue, they often suffer from unreliable supervision signals based on brightness constancy and smoothness assumptions, leading to inaccurate motion estimation in complex real-world scenarios. To overcome these limitations, we introduce \modelname, a novel framework that synthesizes large-scale, perfectly aligned frame--flow data pairs for supervised optical flow training without human annotations. Specifically, our method leverages a pre-trained depth estimation network to generate pseudo optical flows, which serve as conditioning inputs for a next-frame generation model trained to produce high-fidelity, pixel-aligned subsequent frames. This process enables the creation of abundant, high-quality synthetic data with precise motion correspondence. Furthermore, we propose an inconsistent pixel filtering strategy that identifies and removes unreliable pixels in generated frames, effectively enhancing fine-tuning performance on real-world datasets. Extensive experiments on KITTI2012, KITTI2015, and Sintel demonstrate that \modelname achieves competitive or superior results compared to existing unsupervised and semi-supervised approaches, highlighting its potential as a scalable and annotation-free solution for optical flow learning. We will release our code upon acceptance.
Yixuan Luo, Feng Qiao, Zhexiao Xiong +2
University of Chicago · Washington University in St. Louis