Selective Event Evidence and Persistent Temporal Priors for Event-Based Point Tracking
Authors: Xiaokai Song, Zhang Li, Leqi Liu, Yibin Ye, Kun Wang, Xichao Teng
Organizations: College of Aerospace Science and Engineering, National University of Defense Technology, Changsha 410000, China · Hunan Provincial Key Laboratory of Image Measurement and Vision Navigation, National University of Defense Technology, Changsha 410000, China · Videogrammetry Innovation Center, Hunan Institute of Advanced Technology
Event cameras enable tracking under fast motion and challenging illumination. However, stable long-term point tracking remains challenging: event activity is sparse and varies with scene content and motion, complicating correspondence, while trajectory estimates can drift over long sequences. Together, these challenges motivate a tracker that selectively uses informative event evidence and retains trajectory constraints across temporal windows. We propose Selective Event Evidence and Persistent Temporal Priors (SEPT), an event-only framework for tracking any point (TAP) over long sequences. SEPT introduces two complementary modules: Task-Adaptive Event Evidence Modulation (TEEM) and Persistent Trajectory Prior (PTP). Using only tracking supervision, TEEM learns a bounded reliability field that weights the event tensor across space, time, and channels before feature extraction. PTP carries geometric information across overlapping windows by marginalizing a fixed local quadratic trajectory model with a Schur complement. Its visibility-aware correction refines overlap coordinates, while a clipped terminal velocity initializes coordinates at newly exposed timestamps. SEPT builds on the ETAP architecture. SEPT is trained exclusively on synthetic EventKubric data and evaluated on five datasets across TAP and feature tracking. Results show that SEPT outperforms the retrained ETAP baseline in average Jaccard for TAP and in feature age and expected feature age for feature tracking; it also surpasses several methods combining event and frame inputs on EC and EDS. The source code is publicly available at https://github.com/sssxxxkkkk/SEPT.
Figures & tables
Fig. 1: Bottlenecks and corresponding solutions in event-based point tracking. (a) Sparse, nonuniform event activity yields competing local matches; TEEM learns task-adaptive event modulation before feature extraction. (b) As the tracking window advances, historical constraints are discarded and trajectories may drift; PTP carries a Schur-style trajectory prior into the next window.
Fig. 2: Overview of SEPT. TEEM learns spatiotemporal and channel-wise weights for the event tensor before feature extraction. The tracker iteratively refines query trajectories using local correlations and joint attention. At a window transition, PTP transfers overlap coordinates, visibility logits, and a clipped terminal displacement; its fixed Schur precision couples overlap-coordinate deviations during refinement in the next window. Refined latent descriptors are not reused across windows.
Fig. 3: Schur-style persistent trajectory prior. Expired states are eliminated from a fixed quadratic perturbation model, leaving precision P over the retained overlap. The next window copies overlap coordinates and visibility logits and initializes coordinates at newly exposed timestamps using a clipped terminal displacement. After each transformer update, a validity-masked and visibility-weighted preconditioned gradient softly corrects only overlap coordinates; coordinates at newly exposed timestamps receive no prior correction. State counts are illustrative.
Fig. 4: Synthetic EventKubric examples with varied motion. In each pair (a)–(f), the left panel shows the rendered RGB frame and the right panel its event representation. Colored markers and curves indicate query locations and ground-truth trajectories; an × marks a query point that is occluded at the displayed time.
Dataset
Method
AJ ↑
δavgx↑
OA ↑
EventKubric
ETAP †
0.512
0.633
0.951
SEPT †
0.578
0.693
0.955
EVIMO2
ETAP †
0.635
0.751
0.943
SEPT †
0.673
0.786
0.936
TABLE I: TAP results on EventKubric and EVIMO2. ETAP [ 16 ] is our retrained baseline; SEPT combines TEEM and PTP.
Method
Input
EC
EDS
FA
EFA
FA
EFA
ICP [ 21 ]
E
0.256
0.245
0.060
0.040
EKLT [ 12 ]
E+F
0.811
0.775
0.325
0.205
DDFT [ 14 ]
E+F
0.825
0.818
0.576
0.472
FE-TAP [ 40 ]
E+F
0.844
0.838
0.676
0.589
EM-ICP [ 22 ]
E
0.337
0.334
0.161
0.120
TABLE II: Feature tracking on EC and EDS. E denotes events; E+F denotes events and frames. MATE results are quoted from its original paper. M3ED results are reported by sequence in Table III .
Sequence
HASTE [ 41 ]
AEB-Tracker [ 43 ]
DDFT [ 14 ]
ETAP †
SEPT † (ours)
car_urban_day_ penno_small_loop
0.733/0.669
0.530/0.455
0.813/0.807
0.831 / 0.829
0.824/0.822
car_urban_night_ penno_big_loop
0.693/0.632
0.553/0.504
0.683/0.657
0.836/0.828
0.854 / 0.845
car_urban_night_ penno_small_loop
0.577/0.541
0.457/0.429
0.557/0.540
0.736 / 0.735
0.731/0.729
falcon_indoor_ flight_1
0.576/0.502
0.426/0.387
0.652/0.644
0.740/0.739
0.742 / 0.741
falcon_indoor_ flight_2
0.611/0.558
0.539/0.482
0.750/0.741
0.827 / 0.825
0.802/0.801
falcon_outdoor_day_ fast_flight_2
0.614/0.606
0.434/0.396
0.799/0.704
0.878/0.877
0.881 / 0.881
TABLE III: Per-sequence feature tracking on the nine evaluated M3ED left-camera segments. Entries are FA/EFA, averaged over localization thresholds from 1 to 31 pixels. DDFT [ 14 ] uses its event-only configuration. The final row is the unweighted mean across sequences.
Fig. 5: Qualitative feature-tracking results on M3ED. Rows show four sequences; columns compare SEPT, DDFT [ 14 ] , and HASTE [ 41 ] from left to right. Colored curves show predicted tracks, and × marks the ground-truth point position at each timestamp.
TAP
Feature tracking
EventKubric
EVIMO2
EC
EDS
M3ED
Model
AJ
δavgx
OA
AJ
δavgx
OA
FA
EFA
FA
EFA
FA
EFA
Standard training split (80:15:5)
ETAP
0.524
0.643
0.949
0.622
0.737
0.940
0.839
0.833
0.699
0.599
0.789
0.787
ETAP+TEEM
0.531
0.649
0.944
0.655
0.771
0.937
0.797
0.792
0.719
0.605
0.795
0.795
ETAP+PTP
0.566
0.681
0.946
0.663
0.777
0.943
0.866
0.861
0.726
0.614
0.797
0.794
TABLE IV: Module ablation and training-data sensitivity across TAP (EventKubric and EVIMO2) and feature tracking (EC, EDS, and M3ED). Bold marks the best score for each dataset and metric within a training split.
Fig. 6: TEEM visualizations on real event recordings across datasets and timestamps. Each example shows the input event tensor, the learned event-weight map, and the modulated event evidence. Color bars indicate the weight values.
Fig. 7: TEEM evidence at visible object boundaries on EventKubric. Segmentation-derived ground-truth contours (green) are overlaid on the aligned RGB frame, original event tensor, and TEEM-modulated event evidence.
Fig. 8: EventKubric AJ versus (a) mean inference time per forward pass and (b) peak reserved GPU memory for ETAP and the ablated variants. Peak allocated memory changes little across variants and is not plotted. Results use the expanded training split.
Point tracking follows selected or dense image points through video, enabling motion analysis, 3D reconstruction, and precise spatiotemporal understanding.