cs.CVFeb 26, 2026
SaveNo Labels, No Look-Ahead: Unsupervised Online Video Stabilization with Classical Priors
Organizations: School of Electronic and Optical Engineering, Nanjing University of Science and Technology · College of Geoexploration Science and Technology, Jilin University
Abstract
We propose a new unsupervised framework for online video stabilization. Unlike methods based on deep learning that require paired stable and unstable datasets, our approach instantiates the classical stabilization pipeline with three stages and incorporates a multithreaded buffering mechanism. This design addresses three longstanding challenges in end-to-end learning: limited data, poor controllability, and inefficiency on hardware with constrained resources. Existing benchmarks focus mainly on handheld videos with a forward view in visible light, which restricts the applicability of stabilization to domains such as UAV nighttime remote sensing. To fill this gap, we introduce a new multimodal UAV aerial video dataset (UAV-Test). Experiments show that our method consistently outperforms state-of-the-art online stabilizers in both quantitative metrics and visual quality, while achieving performance comparable to offline methods.
Figures & tables
Figure 1: Limitations of existing methods and an overview of our approach. (a) Traditional pipelines rely on sparse, spatially biased keypoints, leading to inaccurate motion estimation and blur/distortion in stabilized frames. (b) Over-smoothing and fixed filtering ignore scene motion trends, resulting in residual jitter, black borders, and warping in the output video. (c) Our approach uses dense, evenly distributed keypoints together with accurate optical flow to propagate motion estimates across frames; it operates online without future frames, enabling real-time processing. (d) In the bottom-right chart, our method (red star) outperforms both traditional offline methods and recent online methods in stabilization quality, achieving state-of-the-art performance for online unsupervised stabilization.
Figure 2: Overview of our video stabilization framework, consisting of three key modules: motion estimation, motion propagation, and motion compensation. The motion estimation module detects keypoints and calculates optical flow to estimate motion between frames. The motion propagation module transfers the estimated motion information to a global trajectory buffer, ensuring motion consistency across frames. The motion compensation module adjusts the frames based on the updated trajectories to produce a stabilized output. A multi-threaded processing structure utilizes three threads, each dedicated to one of the core modules, and leverages shared buffer queues for efficient parallel execution. This approach improves processing speed and online performance, enabling the stabilization of video frames with minimal latency.
| Scene Type | Sensor | Tilt Range | Resolution | FPS | Occlusion | Sequences |
|---|---|---|---|---|---|---|
| Urban areas | Visible/IR | - | 25 | Partial building occlusion | 19 | |
| Highways | Visible/IR | - | 30 | Minimal occlusion | 22 | |
| Forests/mountains | Visible/IR | - | 25 | Frequent canopy occlusion | 18 | |
| Waterfronts | Visible | - | 30 | Mild vapor occlusion | 20 | |
| Industrial sites | Visible/IR | - | 25 | Large machinery occlusion | 13 |
Table 1: Overview of the UAV-Test dataset.
| Method | NUS | DeepStab | Selfie | GyRo | UAV-Test | |||||||||||
| C↑ | D↑ | S↑ | C↑ | D↑ | S↑ | C↑ | D↑ | S↑ | C↑ | D↑ | S↑ | C↑ | D↑ | S↑ | ||
| Offline | Grundmann et al. [ 13 ] | 0.71 | 0.76 | 0.82 | 0.77 | 0.87 | 0.84 | 0.75 | 0.81 | 0.83 | 0.91 | 0.89 | 0.81 | 0.72 | 0.64 | 0.76 |
| Bundle [ 27 ] | 0.81 | 0.78 | 0.82 | 0.80 | 0.90 | 0.85 | 0.74 | 0.82 | 0.80 | 0.93 | 0.82 | 0.85 | 0.79 | 0.69 | 0.88 | |
| DIFRINT [ 7 ] | 1.00 | 0.87 | 0.84 | 1.00 | 0.91 | 0.78 | 1.00 | 0.78 | 0.84 | 1.00 | 0.83 | 0.85 | 1.00 | 0.86 | 0.87 | |
| Yu and Ramamoorthi [ 57 ] | 0.85 | 0.81 | 0.86 | 0.87 | 0.92 | 0.82 | 0.83 | 0.79 | 0.86 | 1.00 | 0.92 | 0.84 | 0.85 | 0.83 | 0.86 | |
| PWStableNet [ 60 ] | 0.91 | 0.97 | 0.86 | 0.97 | 0.97 | 0.87 | 0.99 | 0.92 | 0.89 | 0.99 | 0.96 | 0.83 | 0.90 | 0.88 | 0.90 | |
Table 2: Quantitative results on the NUS [ 27 ] , DeepStab [ 48 ] , Selfie [ 55 ] , GyRo [ 40 ] and UAV-Test datasets. We report Cropping Ratio (C), Distortion Value (D), and Stability Score (S). All metrics range from 0 to 1; higher is better. Best results in each group (offline or online) are in red , second best in blue . A runtime (fps) comparison among online methods is provided in the supplementary material (Sec. F).
Figure 3: Comparison of visual results across different online methods. Distorted regions and black borders are highlighted with red boxes.
Figure 4: User study preferences.
Figure 5: Normalized ablation study results on five datasets (NUS, DeepStab, Selfile, GyRo, UAV-Test). The x-axis denotes ablation configurations A1–A9 (variants with modules removed/modified) and A10 (our full model with window length ). The y-axis shows normalized scores (higher is better) for five metrics: Consistency (C), Distortion Value (D), Peak Signal-to-Noise Ratio (PSNR), Stability Score (S), and Structural Similarity (SSIM).
Figure 6: The architecture of our EfficientMotionPro network.
| Component | Operation | Output Size |
|---|---|---|
| Input | Distance & Motion channels | |
| Distance branch | GhostModule + DWConv + PWConv | |
| Feature reduction | ||
| Distance attention extractor | ||
| Motion branch | GhostModule + DWConv + PWConv | |
| ECA attention enhancement |
Table 3: Architecture of the proposed EfficientMotionPro module. The input tensor is split into distance and motion channels, embedded via Ghost modules, enhanced with ECA, fused, and finally decoded into grid displacements.
Figure 7: Training curves of EfficientMotionPro : the blue line shows training loss and the orange line shows validation loss.
| Keypoints | Estimated MACs / frame |
|---|---|
| M | |
| M | |
| M |
Table 4: Estimated MACs per frame of EfficientMotionPro under different keypoint counts .
Figure 8: The architecture of our OnlineSmoother network.
| Submodule | Tensor Shape (Input Output) | Structural Components |
|---|---|---|
| Trajectory Encoder | Linear + ReLU | |
| Lite LS-3D (Temporal) | DW-Conv3d , dilation + ReLU | |
| Lite LS-3D (Spatial) | DW-Conv3d + ReLU | |
| (Optional) Channel Mixing | PW-Conv3d , groups=4) + ReLU | |
| Kernel Decoder (x) | (per vertex) | Linear (gate), Sigmoid; Linear (amplitude) |
| Kernel Decoder (y) | (per vertex) | Same as |
Table 5: Architecture of the OnlineSmoother module. Per-vertex processing; outputs are per-vertex kernels and smoothed trajectories.
Figure 9: Training curves of OnlineSmoother : blue for training loss, orange for validation loss.
| Window | Grid | MACs / frame |
|---|---|---|
| M | ||
| M | ||
| M |
Table 6: Estimated MACs of the OnlineSmoother backbone at different grid sizes ( , , , ).
Figure 10: Left to right: Original unstable video; Stabilized (uncropped); Stabilized (cropped); Stabilized (with frame outpainting).
Figure 11: Representative examples of unstable UAV visible-light videos from diverse scenarios, including urban areas, highways, forested mountains, waterfronts, and industrial sites.
Figure 12: Representative examples of unstable UAV infrared videos from diverse scenarios, including urban areas, highways, forested mountains, and industrial sites.
| Method | Runtime (ms) | FPS |
|---|---|---|
| MeshFlow [ 26 ] | 168.95 | 5.92 |
| StabNet [ 48 ] | 179.88 | 5.56 |
| NNDVS [ 59 ] | 340.02 | 2.94 |
| Liu et al. [ 29 ] | 36.41 | 27.47 |
| Ours | 78.94 | 12.67 |
Table 7: Per-frame runtime comparison on Jetson AGX Orin. We report the average per-frame runtime (ms) and FPS. Our method is significantly faster than MeshFlow, StabNet, and NNDVS, and second only to the highly lightweight Liu et al. [ 29 ] .
Figure 13: Visualization of motion propagation. The sparse optical flow guided by initial keypoints is contrasted with the grid-wise motion after propagation by our EfficientMotionPro module. The results show that our propagation yields more consistent and structured motion, robust to disturbances caused by dynamic objects and noise.
Figure 14: Per-scene quantitative evaluation.
Figure 15: Example question from the user study survey.
Figure 16: Visualization of mesh vertex trajectories during online smoothing. Six vertices are randomly selected, showing comparisons between unsmoothed trajectories after motion propagation and those refined by our OnlineSmoother .
Figure 17: Comparison of trajectory variance before and after applying OnlineSmoother , demonstrating its effectiveness in suppressing fluctuations.
Figure 18: Frequency-domain analysis of trajectories via FFT, highlighting that OnlineSmoother effectively attenuates high-frequency noise.
Figure 19: Keypoint density heatmaps of representative detectors (SIFT [ 31 ] , SuperPoint [ 8 ] , RFdet [ 38 ] , DISK [ 45 ] , DoG-HardNet [ 6 ] , ALIKED [ 62 ] , XFeat [ 35 ] , DAD [ 11 ] , and Ours) on a sample scene. Our collaborative detector achieves more uniform coverage across the image grid.
Figure 20: Comparison of representative optical-flow algorithms on diverse examples. We report results from several state-of-the-art methods [ 4 , 17 , 41 , 43 , 58 , 39 , 9 ] , among which MemFlow [ 9 ] is adopted in our experiments for its balance of accuracy and memory efficiency.