Organizations: School of Data Science, The Chinese University of Hong Kong, Shenzhen, China · School of Computer Science, Peking University, Beijing, China · College of Computing and Data Science, Nanyang Technological University, Singapore · School of Intelligence Science and Engineering, Harbin Institute of Technology, Shenzhen, China · State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, Beijing, China
Spike cameras encode dynamic scenes into high-temporal-resolution binary spike streams for efficient high-speed vision. Although spiking neural networks (SNNs) can process such signals with sparse and accumulate-only computation, existing SNN-based reconstruction methods are typically shallow and still lag behind ANN-based models in reconstruction fidelity. This paper proposes MADSNet, a Multi-scale Axial-attentive Dense Spiking Network for spike camera image reconstruction. MADSNet adopts a hierarchical spiking encoder--decoder, where a multi-scale spike stem extracts early features, axial spike attention captures long-range dependencies, a dense spiking bottleneck enhances feature reuse, and a spike-to-continuous decoder reconstructs continuous intensity images from spike-domain representations. Experiments on gray-scale and color spike reconstruction show that MADSNet outperforms SSIR by 4.3 dB PSNR and Spk2ImgNet by 1.94 dB on SREDS. Compared with Spk2ImgNet, it achieves about 15× lower latency and uses only about 1/40 of the estimated energy, demonstrating that carefully designed SNN architectures can improve reconstruction quality while preserving computational efficiency.
Figures & tables
Fig. 1: Overall architecture of MADSNet and detailed diagrams of each sub-module. Top: the full encoder-decoder pipeline from input spike stream to reconstructed frames. Bottom: detailed structures of MS3Stem,DS-SEW Block,Axial Spike Attention (ASA), Light Spike Block, Dense Spike Bottleneck, Spike Pixel-Shuffle Upsampling (SPSU), and the Spike-to-Continuous Fusion Block.
Paradigm
Method
SREDS Dataset
PSNR ↑
SSIM ↑
LPIPS ↓
NIQE ↓
BRISQUE ↓
Non-SNN
TFP [ 3 ]
28.027
0.687
0.269
7.445
40.656
TFI [ 3 ]
24.258
0.691
0.263
5.074
40.069
TFSTP [ 22 ]
20.845
0.640
0.271
5.163
36.948
MAHTF [ 4 ]
31.566
0.904
0.089
2.631
15.997
ANN
ETNet [ 32 ]
31.088
0.899
0.095
3.477
14.357
TABLE I: Comparison on the gray-scale spike reconstruction benchmark. Results of existing methods are taken from SSIR.
Fig. 2: Qualitative comparison on real-captured gray-scale spike streams (Binary Spikes, ET-Net, HyperE2VID, Spk2ImgNet, SSIR, Ours). Zoom in for more details. Gamma transformation with parameter 1.5 is used for visualization.
Paradigm
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Non-SNN
TFP [ 3 ]
23.78
0.6207
0.3986
TFI [ 3 ]
23.97
0.6294
0.3731
TFSTP [ 22 ]
23.32
0.5978
0.3759
3DRI [ 18 ]
26.95
0.8029
0.1946
ANN
SJRE [ 9 ]
24.84
0.5499
0.4050
SSL-CSC [ 19 ]
29.95
0.8417
0.1829
TABLE II: Transferability test on RBSS ( α=1.0 ).
Fig. 3: Qualitative comparison on real-captured color spike streams (BSS) (Binary Spikes, ET-Net, HyperE2VID, Spk2ImgNet, SSIR, Ours). Zoom in for more details.Gamma transformation with parameter 2.2 is used for visualization.
Variant
PSNR ↑
SSIM ↑
LPIPS ↓
MADSNet (Full)
37.232
0.956
0.028
w/o MS3Stem (single 3×3 SConv)
35.406
0.948
0.034
w/o ASA
36.601
0.955
0.030
w/o DS-SEWBlock (standard SEWBlock)
36.990
0.954
0.032
w/o Dense Bottleneck (sequential)
36.566
0.954
0.032
w/o SPSU (transposed conv)
36.581
0.954
0.031
TABLE III: Component-wise ablation on SREDS.
Architecture
Neuron
PSNR ↑
SSIM ↑
LPIPS ↓
SSIR (baseline)
PLIF
32.930
0.922
0.050
IBRA-LIF
33.168
0.937
0.030
MADSNet (Ours)
PLIF
30.230
0.817
0.124
IBRA-LIF
37.232
0.956
0.028
TABLE IV: Cross-architecture neuron comparison on SREDS. IBRA-LIF brings only a marginal gain to SSIR, but is essential for preventing performance collapse in the deeper MADSNet architecture.
Layer Group
SSIR
SSIR
MADSNet
MADSNet
Neuron
PLIF
IBRA
PLIF
IBRA
SSIR
Stem (static_conv)
39.66
23.53
–
–
Encoder (enc1)
80.94
36.30
–
–
Bottleneck (eres1)
65.05
29.42
–
–
Decoder (dec3)
83.79
13.19
–
–
TABLE V: Layer-wise non-zero activation rate (%) on SREDS validation set. MADSNet-PLIF exhibits severe activation collapse in deep layers, while IBRA-LIF maintains healthy activations throughout.
Method
Latency ↓ (ms)
Memory ↓ (GB)
Energy ↓ (mJ)
Norm. Energy ↓
Spk2ImgNet
305.4
1.675
4690.0
1
SSIR
3.3
0.310
4.6
1/1019
MADSNet (Ours)
20.4
0.489
118.4
1/40
TABLE VI: Efficiency analysis. Spk2ImgNet and SSIR results are taken from SSIR.
Image reconstruction recovers images, volumes, or scenes from incomplete, noisy, indirect, or compressed measurements using physical models and learned priors.
ANN-based All-in-One image restoration (AiOIR) unifies diverse degradation handling but incurs high computational costs, limiting its real-time deployment. While Spiking Neural Networks (SNNs) offer a low-power alternative, applying them to static images remains challenging. This difficulty arises because explicit event signals are absent, and degradation cues are heavily entangled with scene structures, hindering the learning of reliable restoration-oriented spike events. To address these issues, we propose SpikeRestormer, an energy-efficient SNN for AiOIR that performs event reasoning over internally generated spike cues. Specifically, we propose a degradation-event perception process to extract spike-based degradation events through Subtractive Degradation Event Attention (SDEA). Moreover, we introduce Hierarchical Bayesian Skip Masking (HBSM) and Additive Restoration Event Attention (AREA) processes for event-reliability inference and restoration-event construction, respectively. By integrating these complementary processes, SpikeRestormer formulates restoration as a unified process of degradation-event perception, degradation-event reliability inference, and restoration-event construction, liberating the potential of SNNs for energy-efficient AiOIR. Extensive experiments show that SpikeRestormer delivers competitive performance against ANN-based methods and establishes new state-of-the-art results among SNN-based methods with significantly lower energy consumption.
Shengkai Hu, Jie Shao, Jiaqi Ma +5
School of Information Engineering, Zhongnan University of Economics and Law, Wuhan, China · Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates · School of Computer Science, Wuhan University, Wuhan, China +1
Decoding visual stimuli from the spiking activity of retinal ganglion cells (RGCs) informs both our understanding of the retinal code and the design of visual prostheses. Natural vision is dynamic, yet state-of-the-art decoders mostly reconstruct single frames with artificial neural networks that process dense, continuous-valued signals rather than the retina's sparse, asynchronous spikes. We asked whether dynamic scenes can be reconstructed and tracked entirely within the spike domain, and at what computational cost. We built an end-to-end spiking neural network (SNN) whose neurons integrate the recorded RGC spikes over time. Populations of up to 1836 RGCs were recorded ex vivo from mouse retina on high-density microelectrode arrays while flashed images, moving bars, drifting gratings and randomly moving handwritten digits were presented. For moving digits, one shared spiking encoder fed two readouts, one reconstructing the image and one decoding the digit's position. Compute energy was estimated from counted synaptic operations. The SNN decoded what and where at once. It reconstructed the stimuli with a structural similarity (SSIM) of up to 0.93, refining the reconstruction as spikes accumulated, and tracked a randomly moving target with a mean error of 1.33 pixels on a 32×32 grid. Its fidelity nearly matched that of the best of 17 larger spiking and artificial decoders (5.9 million to 1.7 billion parameters, versus 5.4 million), and at a mean firing rate of 14.2% it needed an estimated ∼70-fold less compute energy than a dense artificial network of the same architecture. Dynamic visual scenes can thus be decoded from retinal population activity in the spike domain, with a compact network and at a compute cost compatible with implantable and wearable neural interfaces, supporting spike-native decoders for visual prostheses.
Ning Zhang, Arto Nurmikko
School of Engineering, Brown University, Providence, RI 02912 USA
Spiking neural networks (SNNs) have garnered significant interest in computer vision due to their potential for efficiency and biological inspiration. While spiking CNN-based methods have shown promise for image restoration (IR) tasks, their performance is constrained by the inherent receptive field limitations of CNN operations. In the paper, we explore the benefits of discrete wavelet transformation and propose a spiking pyramid wavelet-based model (SPWM) for high-efficient and low-energy target. Specifically, we develop a spiking dual pyramid wavelet (SDPW) block to model long-range dependency and exploit the properties of the degradation in the wavelet domain. Experimental results on several benchmarks demonstrate that SPWM significantly lowers computational costs and energy consumption while maintaining image quality. Our method showcases the potential of SNNs in the field of IR, offering new insights for future applications of resource-limited devices.
Chen Zhao, Xiantao Hu, Song Wu +5
School of Intelligence Science and Technology, Nanjing University, Suzhou, 215163, Jiangsu, China · PCA Lab, Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, 210094, Jiangsu, China · China Mobile Institute, Beijing, 100032, China +1