SICAF: Time-Varying Focus Bottleneck for Self-Supervised Event-Based Optical Flow with Spiking Neural Network

Shuangming Yang; Qing He; Shangqi Guo; Badong Chen

cs.CVOct 21, 2026

SICAF: Time-Varying Focus Bottleneck for Self-Supervised Event-Based Optical Flow with Spiking Neural Network

Authors: Shuangming Yang, Qing He, Shangqi Guo, Badong Chen

Organizations: Tianjin University, Tianjin, China · Tsinghua University, Beijing, China · Xi’an Jiaotong University, Xi’an, China

Abstract

Event-based optical flow estimation achieves high temporal resolution and dynamic range by exploiting asynchronous events, while self-supervised learning provides an effective solution to the absence of ground-truth labels. Spiking neural networks (SNNs), as brain-inspired models, process information through asynchronous spikes, naturally aligning with optical flow events. However, the temporal irregularity across event streams hinders the compact extraction of spatiotemporal features, while the absence of explicit labels leads to spurious correlations, thereby increasing redundancy within latent representations. In this paper, we propose a novel 'Compactly Compress, Time-varying Focus' training strategy. We present SICAF (Spikes Information Compression And Focus) framework that formulates the learning process as a constrained optimization with temporal modulation. It encourages self-supervised SNNs to extract compact latent representations while preventing over-compression that would discard temporal saliency. To better enable feature selectivity across time steps, we design an estimator with learnable temporal attention, achieving context awareness with cross-time dependencies. Experimental results on several datasets demonstrate that SICAF achieves state-of-the-art performance in self-supervised SNNs, with a 8.14% improvement in AEE on the MVSEC dataset and a 13.49% increase in robustness under white-box adversarial attacks.

Figures & tables

Figure 1: Visualization of performance evaluation with previous self-supervised SNN (top) and the proposed SICAF (bottom). Our SICAF enhances the estimation accuracy by emphasizing temporal saliency. Concurrently, our SICAF effectively captures structured yet informative motion features, improving the issues of incorrect flow direction estimation and the loss of fine-grained motion details.

Figure 2: Comparison between the general self-supervised SNNs [ 7 , 6 , 15 ] , SeLHIB [ 16 ] , and ours for handling optical flow events with self-supervised learning strategy. (a) General self-supervised SNNs typically rely on contrast maximization. This objective enhances the spatial contrast of motion-compensated events to drive learning, but it neglects the temporal non-uniformity of event data. (b) SeLHIB presents an information bottleneck constraint to encourage compact latent representations. However, because it enforces a uniform compression constraint across time, it struggles to preserve temporally salient features. (c) SICAF addresses these issues by formulating a constrained optimization process with temporally modulated focus. Within this framework, a learnable temporal-attention–based estimator allows SICAF to explicitly model temporal heterogeneity, encouraging the formation of compact yet more temporally informative representations.

Figure 3: Overview of the time-sensitive Rényi’s $\alpha$ -entropy estimator. Temporal attention scores are derived from spike firing rates of latent representations to capture temporal heterogeneity. Rényi’s $\alpha$ -entropy provides a robust and flexible measure of information content, enabling selective feature compression when combined with temporal attention.

Figure 4: (a) Comparison of feature maps in latent space. Both SeLHIB and our SICAF demonstrates sparser feature maps compared to baseline self-supervised SNN, indicating the acquisition of more structured spatiotemporal features. (b) The radar charts of SSIM for self-supervised SNN, SeLHIB, and SICAF, respectively.

Methods	Average Endpoint Error (AEE) $\downarrow$
Methods	Indoor (flying1)	Indoor (flying2)	Indoor (flying3)	Outdoor (day1)	Avg.
LIF-EV-FlowNet [ 6 ]	0.71	1.44	1.16	0.53	0.96
LIF-FireNet [ 6 ]	0.98	1.77	1.50	0.57	1.21
ALIF-EV-FlowNet [ 6 ]	1.00	1.78	1.55	0.57	1.23
PLIF-EV-FlowNet [ 6 ]	0.75	1.52	1.23	0.60	1.03
XLIF-EV-FlowNet [ 6 ]	0.73	1.45	1.17	0.45	0.95

Table 1: Experimental results of SICAF on MVSEC compared to previous work. (lower is better, best in red, runner-up in blue).

Figure 5: Qualitative evaluation of optical flow estimation results on sequences from the MVSEC dataset.

Noise type	Methods	AEE
Noise type	Methods	Indoor (flying1)	Indoor (flying2)	Indoor (flying3)	Outdoor (day1)	Average
SP	LIF-EV-FlowNet [ 6 ]	0.91	1.68	1.39	0.56	1.14
	XLIF-EV-FlowNet [ 6 ]	0.86	1.76	1.21	0.62	1.11
	SICAF-LIF	0.82	1.44	1.20	0.60	1.01
	SICAF-XLIF	0.75	1.43	1.18	0.44	0.95
Gaussian	LIF-EV-FlowNet [ 6 ]	0.95	1.72	1.44	0.62	1.18

Table 2: Experimental results of SICAF on MVSEC under different noise (lower is better, best in red, runner-up in blue).

Methods	$L_{contrast}$	$L_{smooth}$	SICAF	AEE
Methods	$L_{contrast}$	$L_{smooth}$	SICAF	Indoor (flying1)	Indoor (flying2)	Indoor (flying3)	outdoor (day1)	Avg.
(a)	✓	✗	✗	0.77	1.50	1.25	0.56	1.02
(b)	✓	✓	✗	0.81	1.45	1.19	0.57	1.01
(c)	✓	✗	✓	0.83	1.60	1.31	0.54	1.07
(d)	✓	✓	✓	0.71	1.37	1.09	0.43	0.90

Table 3: Ablation study on the various components of loss functions (lower is better, best in red, runner-up in blue).

Methods	Compact Compression	Time-Varying Focus	Uniform Compression	AEE
Methods	Compact Compression	Time-Varying Focus	Uniform Compression	Indoor (flying1)	Indoor (flying2)	Indoor (flying3)	Outdoor (day1)	Avg.
(a)	✓	✓	✗	0.72 $\pm$ 0.031	1.38 $\pm$ 0.026	1.09 $\pm$ 0.03	0.47 $\pm$ 0.053	0.91 $\pm$ 0.012
(b)	✓	✗	✓	0.81 $\pm$ 0.047	1.53 $\pm$ 0.044	1.28 $\pm$ 0.064	0.55 $\pm$ 0.041	1.04 $\pm$ 0.019
(c)	✗	✓	✗	0.81 $\pm$ 0.075	1.51 $\pm$ 0.029	1.24 $\pm$ 0.010	0.56 $\pm$ 0.061	1.03 $\pm$ 0.043
(d)	✓	✗	✗	0.81 $\pm$ 0.021	1.52 $\pm$ 0.059	1.28 $\pm$ 0.025	0.61 $\pm$ 0.055	1.06 $\pm$ 0.036

Table 4: Ablation study on the key components of SICAF on the MVSEC dataset (lower is better, best in red, runner-up in blue).

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Initialization range
$\alpha_{0}$ (learnable)	0.01
$\alpha_{1}$ (learnable)	1
$\nu$ (learnable)	$\nu=\frac{1}{1+\exp(-a)},a\in\mathcal{N}(-4,0.1)$
$\delta$	$\delta\in\{0,1\}$
$\eta$ (learnable)	$\eta=\frac{1}{1+\exp(-c)},c\in\mathcal{N}(-4,0.1)$
$m$	0.8

Appendix

Table 5: SNN hyperparameter initialization table

Methods	ECD		HQF
Methods	FWL $\uparrow$	RSAT $\downarrow$	FWL $\uparrow$	RSAT $\downarrow$
LIF-EV-FlowNet [ 6 ]	1.21	0.95	1.24	0.94
ALIF-EV-FlowNet [ 6 ]	1.17	0.98	1.21	0.98
PLIF-EV-FlowNet [ 6 ]	1.24	0.95	1.28	0.93
XLIF-EV-FlowNet [ 6 ]	1.23	0.95	1.25	0.93
SICAF	1.26	0.94	1.30	0.92

Appendix

Table 6: Experimental results on ECD and HQF datasets. (Higher FWL and lower RSAT indicate better performance. Best in red, runner-up in blue.)

Figure 6: Qualitative evaluation of optical flow estimation results on sequences from the ECD and HQF datasets.

Noise type	SP		Gaussian		Black-box attack		White-box attack
Methods	[ 28 ]	SICAF	[ 28 ]	SICAF	[ 28 ]	SICAF	[ 28 ]	SICAF
boxes	0.97	0.94	0.97	0.94	1.01	1.00	0.99	0.96
0calibration	0.97	0.96	0.98	0.96	1.00	1.00	0.99	0.98
dynamic_6dof	0.94	0.92	0.95	0.92	1.00	0.98	0.97	0.94
office_zigzag	0.97	0.96	0.98	0.97	1.00	1.00	0.99	0.98
poster_6dof	0.97	0.94	0.97	0.94	1.01	0.99	0.99	0.90

Appendix

Table 7: Experimental results of SICAF on ECD under different noise (lower is better, best in red).

Noise type	SP		Gaussian		Black-box attack		White-box attack
Methods	[ 28 ]	SICAF	[ 28 ]	SICAF	[ 28 ]	SICAF	[ 28 ]	SICAF
bike_bar_hdr	0.97	0.95	0.98	0.95	1.00	0.98	0.98	0.95
boxes	0.94	0.92	0.95	0.93	1.00	0.97	0.97	0.93
desk_6k	0.92	0.90	0.93	0.91	0.99	0.95	0.95	0.92
desk_fast	0.92	0.89	0.93	0.90	0.99	0.95	0.95	0.91
desk_hand_only	1.00	0.98	1.00	0.98	0.99	0.99	0.98	0.97

Appendix

Table 8: Experimental results of SICAF on HQF under different noise (lower is better, best in red).

Model	Latency (ms/batch)	Inference energy per batch (J)	Memory (MiB)	Spikes per inference batch
LF-EV-FlowNet	$29.764\pm 7.881$	$4.354\pm 1.159$	$241.548\pm 6.885$	$197{,}724.67\pm 769.86$
SICAF	$27.164\pm 6.645$	$3.873\pm 0.436$	$247.503\pm 6.582$	$83{,}978.00\pm 274.99$

Appendix

Table 9: Computational efficiency comparison between LF-EV-FlowNet and SICAF. Lower values are preferable for all reported metrics.

Event-Based Optical Flow

Event-based optical flow estimates pixel motion from asynchronous brightness-change events, enabling low-latency vision in fast or difficult lighting.

See all 13 papers

SpikeCLR: Contrastive Self-Supervised Learning for Few-Shot Event-Based Vision using Spiking Neural Networks
Mar 17, 2026Maxime Vaillant, Axel Carlier, Lai Xing Ng +2Self-Supervised Visual Representation Learning Event-Based Vision
CNRS, IPAL IRL 2955, Singapore · Université de Toulouse, IRIT, France · Institute for Infocomm Research, A*STAR, Singapore +2
Sequence-SOD: Bio-inspired Sequence-aware Spiking ObjectDetection for Event Cameras
Jul 29, 2026Katharina Bendig, René Schuster, Didier StrickerSpiking Neural Networks Event-Based Object Detection
RPTU – Technical University Kaiserslautern-Landau, Gottlieb-Daimler-Straße 47, Kaiserslautern 67663, Germany · DFKI – German Research Center for Artificial Intelligence, Trippstadter Str. 122, Kaiserslautern 67663, Germany
PAFS: Perturbation-Aware Firing Scaffolds for Robust Event-Based SNNs
Oct 23, 2026Ning Liu, Jun Sun, Hao Guo +6Spiking Neural Networks Neural Network Robustness
The University of Western Australia, Australia · Nanjing University, China · The Chinese University of Hong Kong, Hong Kong SAR, China +2

Mar 17, 2026J/K move · Enter open · S save

SpikeCLR: Contrastive Self-Supervised Learning for Few-Shot Event-Based Vision using Spiking Neural Networks

Maxime Vaillant, Axel Carlier, Lai Xing Ng, Christophe Hurter+1

CNRS, IPAL IRL 2955, Singapore · Université de Toulouse, IRIT, France · Institute for Infocomm Research, A*STAR, Singapore+2

Self-Supervised Visual Representation Learning Event-Based Vision Contrastive Learning Spiking Neural Networks

Open paper AskPDF arXiv

Event-based vision sensors provide significant advantages for high-speed perception, including microsecond temporal resolution, high dynamic range, and low power consumption. When combined with Spiking Neural Networks (SNNs), they can be deployed on neuromorphic hardware, enabling energy-efficient applications on embedded systems. However, this potential is severely limited by the scarcity of large-scale labeled datasets required to effectively train such models. In this work, we introduce SpikeCLR, a contrastive self-supervised learning framework that enables SNNs to learn robust visual representations from unlabeled event data. We adapt prior frame-based methods to the spiking domain using surrogate gradient training and introduce a suite of event-specific augmentations that leverage spatial, temporal, and polarity transformations. Through extensive experiments on CIFAR10-DVS, N-Caltech101, N-MNIST, and DVS-Gesture benchmarks, we demonstrate that self-supervised pretraining with subsequent fine-tuning outperforms supervised learning in low-data regimes, achieving consistent gains in few-shot and semi-supervised settings. Our ablation studies reveal that combining spatial and temporal augmentations is critical for learning effective spatio-temporal invariances in event data. We further show that learned representations transfer across datasets, contributing to efforts for powerful event-based models in label-scarce settings.

Abstract

Event-Based Optical Flow

Explore similar work

SpikeCLR: Contrastive Self-Supervised Learning for Few-Shot Event-Based Vision using Spiking Neural Networks

Sequence-SOD: Bio-inspired Sequence-aware Spiking ObjectDetection for Event Cameras

PAFS: Perturbation-Aware Firing Scaffolds for Robust Event-Based SNNs