Efficient and robust control with spikes that constrain free energy
Authors: André Urbano, Pablo Lanillos, Sander Keemink
Organizations: Cajal Neuroscience Center, Spanish National Research Council, Madrid, Spain · Department of Machine Learning and Neural Computing, Donders Institute for Brain Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
Animal brains exhibit remarkable efficiency in perception and action, while being robust to both external and internal perturbations. The means by which brains accomplish this remains, for now, poorly understood, hindering our understanding of animal and human cognition, as well as our own implementation of efficient algorithms for control of dynamical systems.A potential candidate for a robust mechanism of state estimation and action computation is the free energy principle, but existing implementations of this principle have largely relied on conventional, biologically implausible approaches without spikes. We propose a novel, efficient, and robust spiking control framework with realistic biological characteristics. The resulting networks function as free energy constrainers, in which neurons only fire if they reduce the free energy of their internal representation. The networks offer efficient operation through highly sparse activity while matching performance with other similar spiking frameworks, and have high resilience against both external (e.g. sensory noise or collisions) and internal perturbations (e.g. synaptic noise and delays or neuron silencing) that such a network would be faced with when deployed by either an organism or an engineer. Overall, our work provides a novel mathematical account for spiking control through constraining free energy, providing both better insight into how brain networks might leverage their spiking substrate and a new route for implementing efficient control algorithms in neuromorphic hardware.
Figures & tables
Figure 1: Control Loop of Spiking Free Energy Constrainer (SFEC) networks, displayed schematically, for controlling a single Spring-Mass-Damper linear dynamical system. A) The network receives the system observations ( y ) and a target state ( z ) as input, as well as recurrent connections of two different timescales ( Ω ), and outputs sequences of spikes that drive the network’s internal representation. B) The network’s objective is to maintain a bounded free energy, which is computed from the precision-weighed prediction errors ( F , top). This corresponds to a quadratic optimization process that maintains low prediction errors ( Geirhos et al., 2020 ; Mancoo et al., 2020 ) . Spikes are generated from this process during states with high free energy, as we can see by the fact that activity in the spike raster (bottom) is concentrated around the moments when the target changes instantaneously. C) SFEC networks maintain an internal estimate ( μ ), which is driven by the spikes. The weights of the network are arranged such that the neurons’ thresholds form a bounding box ( Calaim et al., 2022 ) around the point of minimal free energy and each neuron’s firing will return the signal closer to this point. The gray polygon within the box indicates the limits of where the internal errors are constrained to be ( ϵμ and ϵy ), thus constraining the free energy. When either error becomes to big, thus ”hitting” one of the colored edges, the corresponding neuron fires, moving the errors along the vector Di towards the center of the bounding box. The internal estimate is updated such that the free energy ( F ) remains bounded through a bounding box mechanism ( Calaim et al., 2022 ) . Because the free energy combines measures of the error with the measured state and the target state, the internal estimate remains between the two. D) Based on the internal estimate (which is biased towards the target state) a control action (force) is generated and applied to the system driving it from its actual state towards the internal state, ensuring it follows the target dynamics. E) The state variables evolve according to the Plant dynamics, the control input and a process noise term. The system used in this example is the linear SMD system. F) Observations are generated from the current state of the system and an observation noise term, and fed back into the network.
Figure 2: SFEC can successfully control coupled dynamical systems. A) (i) Schematic of a Coupled Oscillator system with 5 masses and 6 springs. The masses oscillate between two walls with fixed positions at x=0 and x=L . (ii) Control of a coupled Oscillator system showing the oscillator states converging to the target states (top), the control signals generated by the control algorithm (middle), and the spike raster showcasing network activity (bottom). B) (i) Schematic of a system of drones (modelled as 2D free masses). The green lines represent the attractive forces the controller exerts towards the target positions, the big X’s are those target positions and the dotted arrows are the trajectory the target signal is following (ii) Same as panel A (ii) but for control of a drone swarm system.
Figure 3: Flexible target dynamics enable different coordination behaviors. A) (i) Simple target dynamics: each mass is independently attracted to a target position (illustrated by springs from targets to drones, as illustrated by the green dotted lines); [explain green lines, arrows, X’s]. (ii) 2D trajectories of three drones following independent attractors — each drone moves directly to its target. B) (i) Formation target dynamics: masses are attracted to targets while repelling each other (green lines show attraction, red lines show repulsion). (ii) 2D trajectories showing formation maintenance — drones coordinate their motion to avoid collisions while following a central target which moves on a circle (dashed line).
Figure 4: SFEC robustness to cumulative perturbations. (Top row) A timeline indicating when each perturbation is activated). (Rows 2-4) Target and state changes, spike raster, free energy, and the MSE. A) Control robustness under external perturbations, consisting of random kicks (throughout), increased process noise ( t≥10s ) and increased observation noise ( t≥20s ). B) Control robustness under internal perturbations applied to the controller network, consisting of voltage noise ( t≥6s ), synaptic perturbations ( t≥12s ), 25% neuron silencing ( t≥18s ), synaptic delays ( t≥24s ).
Figure 5: Extensive analysis of the controller’s robustness to noise. A) Heat Map plot showing Control Noise in the x-axis and Observation Noise in the y-axis; total MSE of the entire control run ( T=30s,dt=0.001s , 2D masses) is shown as color, where brighter color represents more error. B) Example of a run with minimal Process Noise and very high Observation Noise, corresponding to the highlighted box on the top left. C) Example of a run with very high Process Noise and minimal Observation Noise, corresponding to the highlighted box on the bottom right. In both B and C, the top subplot shows the real positions of the masses, whilst the bottom subplot shows the observations fed to the controller.
Controller (#Neurons)
MSE
Spike Count
Comp. Time (ms/timestep)
Nengo (100)
(1.90±0.22)e4
11608±1209
0.06
Gradient SCN (100)
(1.68±0.01)e4
2032±131
0.02
SFEC (100)
(1.60±0.02)e4
642±7
0.02
Nengo (200)
(1.52±0.04)e4
19897±1904
0.06
Gradient SCN (200)
(1.66±0.01)e4
2915±71
0.03
SFEC (200)
(1.57±0.03)e4
1053±30
0.04
Table 1: Performance and efficiency evaluation: Mean-Square-Error, Total Spike Count and Computation time of each of the implementations averaged over 20 runs, under the same conditions as all other tests ( T=30s,dt=0.001s ), controlling the 2D masses system. Best values in bold and standard deviations reported.
Figure 6: Total network firing rates over time. Number of spikes per time step of each of the Spiking versions of the Free Energy Controller applied to the Drone Swarm system. Each firing rate is computed from it’s corresponding network’s spikes with a rolling window of 500 time steps. SFEC (blue) and Gradient SCN (red) are 1-2 orders of magnitude more efficient than the NENGO variant (black) in terms of spike counts, and SFEC displays the best performance, maintaining better efficiency than the Gradient SCN.
Figure 7: Comparison of the 3 controllers operating on the Drone Swarm system with 200 neurons and fixed conditions equal to Fig. 2B. Reporting the states, the generated control signal and the spiking activity.
This paper presents a Bayesian control framework that integrates spike-based dynamics with probabilistic inference for adaptive control. Bayesian inference is widely regarded as a core computational principle of brain function, providing a normative framework for perception, decision-making, and learning under uncertainty. By combining a biologically inspired spiking neural model with Bayesian inference principles, we propose a brain-like control algorithm capable of operating in uncertain environments. We use the mountain car parking problem as a benchmark with non-linear dynamics. Our results demonstrate that the proposed controller can successfully update states in real time and generate goal-directed action plans through spike-driven dynamics. The results highlight the proposed model's potential as a bridge between computational neuroscience and probabilistic control theory.
Sepideh Adamiat, Hongye Wang, Wouter M. Kouw +1
Electrical Engineering Department, Eindhoven University of Technology, Eindhoven, the Netherlands · Biomedical Signals and Systems, Faculty of Electrical Engineering, Mathematics and Computer Science (EEMCS), University of Twente, Enschede, The Netherlands
Biological neural systems achieve remarkable robustness and adaptability in uncertain environments through sparse, event-driven spike-based information processing and adaptive regulation. Inspired by this paradigm, this paper develops a neuromorhpic disturbance observer (NDO) and control framework that replaces conventional continuous-time signal representations with spike-timing encoding. Both disturbance estimates and control inputs are constructed via integrate-and-fire (IF) neuron dynamics from discrete spike events, yielding intrinsically event-driven updates. An adaptive-threshold triggering mechanism is inspired by spike-frequency adaptation (SFA), enabling history-dependent regulation of spike generation. Simulation results demonstrate that the proposed framework achieves neurally inspired robustness and adaptability, while the adaptive-threshold spiking scheme reduces spike events to 42.6% of the fixed-threshold case under noisy conditions.
Hongfu Xu, Xiaoyu Guo, Shengbo Wang +1
School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China · Department of Mechanical Engineering, City University of Hong Kong, Hong Kong · Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong +1
Spiking neural networks (SNNs) are promoted as an energy-efficient substrate because sparse, event-driven activity replaces dense multiply-accumulates with cheap accumulates. We argue the energy dividend of sparsity is not a property of SNNs but of the task. Holding architecture fixed and swapping only the hidden unit (continuous vs. leaky-integrate-and-fire), plus a two-sided target-firing-rate probe, we measure how far activity can be pushed down before quality breaks. Low-load feed-forward perception sparsifies to 5% firing at no accuracy cost; a recurrent language model cannot go below ~50% -- the recurrent state must stay active to carry information. A spiking Transformer, by contrast, sparsifies freely to 2% (3 seeds) -- so the ceiling is a property of recurrent compression, not sequence modeling. Attention escapes the floor only by storing the full key-value cache, trading a firing floor for a memory wall: on neuromorphic hardware, recurrence and attention pay on different axes, neither escapes. We formalize the ceiling with an information-theoretic bound rho >= H_b^{-1}(log2 M / H) and confirm its predictions: the floor rises with memory load, falls with state width, and (refuting a naive memory-only reading) rises with task difficulty. A layer-wise input floor further caps op reduction under dense input, isolating event-driven perception as where neuromorphic hardware wins.