cs.LGOct 27, 2026

Learning Without Clocks: Partition-Invariant Neural Network

Authors: Han Wang, Jingyuan Bao, Juntao Wu, Saiao Zhou, Yuman Nie

Organizations: Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China · Science Island Branch, Graduate School of University of Science and Technology of China, Hefei 230026, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Event cameras, spiking cochleas and similar sensors emit asynchronous events, yet most learning systems first bin them into frames or packets, so the output depends on how the stream was sliced. Networks whose state is updated exactly at every event and evolves in closed form between events are invariant to this slicing. We ask what more makes the output (i) independent of the processing order of simultaneous events and (ii) continuous in the time stamps, even when two events cross in time, while (iii) staying sensitive to temporal order. For arbitrary nonlinear event models we prove that (i) and (ii) are each equivalent to pairwise commutation of the event updates, and that once updates commute, all order dependence comes from their non-commutation with the inter-event dynamics and vanishes linearly in the time lag. This identifies a class of time-mediated non-commutative (TM) models and yields a resolution-robustness bound for any model. We construct the minimal TM layer, a learnable, provably Lipschitz generalization of the Hassenstein-Reichardt correlator; prove that no single additive layer can uniformly represent ordered pair interactions, whereas a three-dimensional TM state can; show that each TM layer adds two levels of temporal nesting versus one for additive layers; and give a single-layer higher-order construction (TM-k) and a stacking rule preserving all three properties. We instantiate these results as the Clock-Free Event (CFE) layer, linear in the number of events, give a topology-free variant for temporal point processes with closed-form likelihoods, and specify a preregistered clock-robustness protocol with a physical-time threat model for spike-retiming attacks. In preliminary development-set experiments, the TM state beats an otherwise identical additive state by 3.5-9.1 points on DVS128 Gesture and by 24.5 and 30.6 points on SHD and Speech Commands.

Figures & tables

Figure 1: Clocked versus clock-free processing. Top: binned pipelines slice the event stream into frames of width Δ\Delta ; the input tensor depends on Δ\Delta and on the phase of the grid, and moving one event across a bin edge by an arbitrarily small ε\varepsilon changes a count ( Proposition 11 ). Bottom: a CFE network updates its state exactly at every event (jumps, coloured by event type) and evolves it in closed form between events (blue); queries ( ⧫\blacklozenge ) can be placed at arbitrary times, and the result does not depend on how the stream is cut into packets ( Proposition 3 ). Right: output of a single site as two events of types aa and bb cross in time, computed from the model equations with illustrative parameters. A gated update ( Sel ) jumps at the crossing and returns two different values at an exact tie, depending on the processing order; the TM layer is continuous and still distinguishes the two orders. Tie invariance and crossing continuity are each equivalent to commuting jumps ( Theorem 5 ), and order information then enters only through the non-commutation of the jumps with the drift ( Theorem 8 ).
FamilyUpdate at an event(P)(T)(C)(O 2 )
Binned networks (frames, fixed time step; e.g. PLIF [ 4 ] )counts added to the current binnoyesnoyes, at bin resolution
Additive event SSMs (exact decay; NS )x←x+bcx\leftarrow x+b_{c}yesyesyesno ( Theorem 15 )
Selective or input-gated affine updates ( Sel )x←Gcx+bcx\leftarrow G_{c}x+b_{c} , generic GcG_{c}yesnonoyes
Gated nonlinear updates with ODE drift [ 25 , 26 ]x←GRU(x,c)x\leftarrow\mathrm{GRU}(x,c)yesnonoyes
Online log-signature models [ 15 ]interval composition, non-commuting generatorsyesexcl. †fixed order ‡yes
TM (this work)u←u+αcu\leftarrow u+\alpha_{c} , v←v+gcRcsv\leftarrow v+g_{c}R_{c}syesyesyesyes
Table 1: Properties of event-model families under exact computation, for generic parameters. (P) partition invariance; (T) tie invariance; (C) continuity in the time stamps across crossings; (O 2 ) ordered pair interactions within one layer. Justifications are in Section A.13 .
Figure 2: The CFE network. An event stream (top left; ON and OFF events of a schematic rotating gesture), or a sampled signal converted into events by a send-on-delta encoder (bottom left, Proposition 22 ), drives the CFE layer. The sensor is tiled into sites; an event (red pixel) applies a jump to the state of every site in its 3×33\times 3 neighbourhood, with a letter given by its polarity and relative offset, and every site state drifts in closed form between events. TM site: events write the fast trace uu and, through the mark-dependent gate gc(m)Rcg_{c}(m)R_{c} , read the delayed trace ss into the accumulator vv ; no event writes a coordinate that events read, so all jumps commute ( Corollary 7 ). Exact parallel scan: each event acts as an affine map, so the states are computed exactly by a segmented parallel scan, and tie groups may be processed in any order. Queries read the site states x(Tq)x(T_{q}) at arbitrary times; a readout shared by all arms (signed-log normalization with raw features, convolution over the site grid, anchor-based pooling, a memory across queries and a linear layer) gives a prediction at every query. For event sequences without topology, an intensity head with a closed-form compensator gives the exact likelihood ( Proposition 23 ). Bar heights are schematic.
CheckResultSupports
Closed form vs. recursion (200 streams, 5–60 events, ≈\approx 30% ties)max. relative error 2.5×10−152.5\times 10^{-15}Prop. 12
Shuffling within ties (100 streams, ≈\approx 60% ties)max. ∣Δx∣|\Delta x| : TM 1.7×10−131.7\times 10^{-13} , NS 1.4×10−141.4\times 10^{-14} , Sel-Merge 3.4×10−133.4\times 10^{-13} , Sel 43.543.5Thm. 5
Time-stamp perturbations 10−410^{-4} – 3.23.2 ms, with reordering (300)observed/bound ≤0.0089\leq 0.0089Thm. 14
Swap at lag δ∈[10−5,10−1]\delta\in[10^{-5},10^{-1}] msTM 3.84×10−3δ3.84\times 10^{-3}\delta ; NS 1.88×10−3δ1.88\times 10^{-3}\delta ; Sel , Sel-Merge 0.2526–0.2528Cor. 6 , Thm. 8
NS fiber ( d=5d=5 ; 50 bb -events)state residual ≤6.7×10−16\leq 6.7\times 10^{-16} while yordy_{\mathrm{ord}} falls from 37.00 to 32.30Thm. 15
TM represents yordy_{\mathrm{ord}} (200 streams)∣K−κ∣≤1.1×10−16|K-\kappa|\leq 1.1\times 10^{-16} ; all errors within bound; max. 6.4% (bound 9.5%)Prop. 16
Table 2: Numerical checks of the theory (FP64, seed 23).
Figure 3: State difference when two events of different types swap places at lag δ\delta (FP64). TM and NS change linearly in δ\delta , below the Lipschitz bound of Theorem 14 . Selective updates jump by a constant amount at every lag, as Corollary 6 predicts.
Setting (clips)Arm200400600800
V1, folds 0, 2 (198)TM170175177176
NS152165169169
Zero138156159162
V1, folds 0–3 (495)TM407419440440
best unordered367407418419
V2, folds 0, 2 (198)TM170177179179
Table 3: DVS128 Gesture, development folds: correct clips at the 200/400/600/800 ms prefixes. V1: 8×88\times 8 -pixel sites; V2: 4×44\times 4 -pixel sites with augmentation. “Best unordered” is the better of NS and Zero at each prefix.
Figure 4: DVS128 Gesture, folds 0 and 2 (198 calibration clips): accuracy against observation window for TM, NS and the stateless arm ( Zero ). The benefit of the ordered state is largest early.
Dataset, configurationArm10-point meanfinal
SHD, augmented (A)TM81.8087.40
NS64.3162.88
Zero–8.73
SHD, baseTM73.2576.45
NS51.5947.37
Zero11.368.86
Table 4: Audio benchmarks, validation splits (accuracy, %). SHD: mean over 10 query times and at 1000 ms. Speech Commands (SC): 9,981 validation clips; TM and NS are still training (epoch 20).
Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Variant10-point mean1000 ms
Base (no augmentation, 8 channels per site)73.2576.45
A: augmented, 8 channels per site81.8087.40
A with 2 channels per site63.8963.43
A with 4 channels per site (B)75.8382.27
B with channel-position letters (12 letters)71.8375.07
A with 16 channels per site74.6177.15
Appendix
Table 5: SHD validation accuracy (%) of TM variants.
More in this topic

Permutation-Invariant Learning

Permutation-invariant learning builds models whose outputs do not change when equivalent inputs are reordered.

See all 31 papers

Explore similar work

CardsList
Sep 16, 2026

REACT: A Fully Spiking State-Space Model for Real-Time Event-Driven Temporal Perception

Robotic systems operating in dynamic environments require visual perception that evolves continuously with the incoming sensory stream. Event cameras provide microsecond temporal resolution and asynchronous sensing, but most learning-based methods accumulate events into frames or temporal bins, introducing an integration delay that can limit fast reaction. Here we propose REACT, a fully spiking state-space model for event-driven temporal perception that processes raw events one by one, without temporal accumulation. REACT uses a complex-valued spiking neuron, C-SiLIF, whose continuous-time dynamics are driven by the physical inter-event interval, allowing its internal state to evolve at the temporal resolution of individual events. We evaluate REACT on gesture recognition and time-to-collision (TTC) estimation from full-field event streams, without a target bounding box or localization input. On EvTTC, REACT achieves a 9.59% relative TTC error with 4.6 ms end-to-end inference latency, within 0.15 percentage points of the best learned method while requiring no target prior. At the dataset's mean approach speed, this latency corresponds to only 4 cm of vehicle motion, compared with 1 m for the fastest competing learned method. REACT further supports anytime TTC prediction, zero-shot transfer to a different driving sequence, and INT8 quantization, reducing the estimated energy consumption from 18.5 to 2.8 mJ per 32,768 events. These results show that event-driven spiking state-space dynamics can provide low-latency, continuously updated temporal perception for reactive robotic systems.

Geoffroy Keime, Nicolas Cuperlier, Benoit R. Cottereau

CerCo, CNRS UMR5549 Université de Toulouse Toulouse, France · IPAL, CNRS IRL, Singapore · ETIS, CY Cergy Paris Université CNRS, ENSEA Cergy, France

Collision AvoidanceEvent-Based VisionRobotic PerceptionState Space Models
AskPDF
Sep 28, 2026

Stealth Is a Relation, Not a Property: How Event Representations Create Blind Spots for Timing Attacks in Event-Based Perception

An event camera produces an asynchronous stream, but what is visible in that stream depends on how a downstream consumer, such as a model or detector, processes time. The same timestamp change may leave a coarse temporal representation unchanged while changing the response of a model that preserves finer timing. We characterize this dependence as observer-relative stealth. For recorded event streams, retiming an event within its protected accumulation window leaves the accumulated integer tensor exactly unchanged. We use this exact blind space to construct Null, a gradient-guided timestamp-retiming attack, and define SC-ASR_A(tau) to measure attack success while bounding the change visible to observer A. On DVS Gesture at a 10% event budget, Null reaches 81.56 +/- 5.81% ASR on ConvSNN and 98.67 +/- 0.45% on a GRU while preserving the protected tensor exactly. On DailyDVS-200, a protocol-scale Multi-View Fusion Network variant reaches 99.28 +/- 0.11% exact-null ASR, compared with 9.70 +/- 1.06% for its matched control. In a five-attack comparison, Null is the only method with nonzero attack success at exact observer equality, reaching 81.4% on DVS Gesture and 87.35% on DailyDVS-200. We also search the same exact blind space with an independently implemented constrained projected-gradient optimizer, C-PGD. At matched victim-gradient evaluations, C-PGD reaches 84.50 +/- 2.89% ASR on DVS Gesture and 89.55 +/- 4.39% on DailyDVS-200, again with exact protected equality. Perturbations that are exactly hidden from the protected observer become visible under shifted, finer, overlapping, and randomized temporal views. Adding observer constraints reduces the real-valued blind-space fraction from 87.5% to 75.0% to 62.5%, while DVS ConvSNN ASR falls from 74.9% to 61.9% to 37.2%. These results show that stealth is not a property of the perturbation alone.

Shoaib Ahmed Dipu, Md. Shaown Miah, Kamrul Hasan +1

Indiana University Indianapolis · Bangladesh University of Engineering and Technology · Tennessee State University

Event-Based VisionAdversarial Attacks
AskPDF
Jun 18, 2026

Neural Events: Discrete Asynchronous Autoencoders for Event-Based Vision

Event cameras capture dynamic scenes with exceptional temporal fidelity by representing them as a continuous stream of microsecond resolution events. Each individual event, however, only carries minimal semantic value, merely signaling a localized brightness change. To derive meaningful signals, downstream algorithms need to quickly integrate cues from a potentially massive torrent of low-information events. Current architectures, however, are easily overwhelmed, struggling to balance capturing fine-grained temporal dynamics and maintaining a manageable data throughput. This paper proposes a framework to re-tokenize event streams into a small set of highly informative neural events, each representing a local spatio-temporal context window with a discrete learnable code. Every time this code flips, a neural event is triggered, yielding a highly compressed data stream. We demonstrate that, across object detection and classification, networks trained on neural events are on par or surpass the performance of state-of-the-art approaches while reducing the event rate by a factor of 2.0.

Roberto Pellerito, Daniel Gehrig, Shintaro Shiba +1

Robotics and Perception Group, University of Zurich · University of Pennsylvania · The University of Tokyo +1

Event-Based VisionRepresentation LearningAutoencodersEvent-Based Object Detection
AskPDF