Neuromorphic Hardware

We propose tokenization of events and present a tokenizer, Spiking Patches, specifically designed for event cameras. Given a stream of asynchronous and spatially sparse events, our goal is to discover an event representation that preserves these properties. Prior works have represented events as frames or as voxels. However, while these representations yield high accuracy, both frames and voxels are synchronous and decrease the spatial sparsity. Spiking Patches gives the means to preserve the unique properties of event cameras and we show in our experiments that this comes without sacrificing accuracy. We evaluate our tokenizer using a GNN, PCN, and a Transformer on gesture recognition and object detection. Tokens from Spiking Patches yield inference times that are up to 3.4x faster than voxel-based tokens and up to 10.4x faster than frames. We achieve this while matching their accuracy and even surpassing in some cases with absolute improvements up to 3.8 for gesture recognition and up to 1.4 for object detection. Thus, tokenization constitutes a novel direction in event-based vision and marks a step towards methods that preserve the properties of event cameras.

Memristor-based Spiking Neural Networks (SNNs) with temporal spike encoding enable ultra-low-energy computation, making them ideal for battery-powered intelligent devices. This paper presents a circuit-level memristive spiking neural network (SNN) architecture trained using a proposed novel supervised in-situ learning algorithm inspired by spike-timing-dependent plasticity (STDP). The proposed architecture efficiently implements lateral inhibition and the refractory period, eliminating the need for external microcontrollers or ancillary control hardware. All synapses of the winning neurons are updated in parallel, enhancing training efficiency. The modular design ensures scalability with respect to input data dimensions and output class count. The SNN is evaluated in LTspice for pattern recognition (using 5x3 binary images) and classification tasks using the Iris and Breast Cancer Wisconsin (BCW) datasets. During testing, the system achieved perfect pattern recognition and high classification accuracies of 99.11% (Iris) and 97.9% (BCW). Additionally, it has demonstrated robustness, maintaining an average recognition rate of 93.4% under 20% input noise. The impact of stuck-at-conductance faults and memristor device variations was also analyzed.

Recurrent neural networks are canonical models of biological memory. In these models, memories are represented by distributed patterns of neural activity that are stored in the recurrent connections between neurons, such that they become attractors of the network's dynamics. During memory recall, network dynamics thus converge toward one of these memory patterns when started from a noisy or partial cue. Therefore, memory performance critically hinges on the dynamical stability of the stored patterns. However, previous theoretical approaches only studied dynamical stability under highly restrictive conditions that do not readily apply to biological neural circuits. Here, we develop a theory of the local stability of discrete fixed points in a broad class of networks with graded neural activities and in the presence of noise. Using methods from random matrix theory, we analyze the bulk and outliers of the eigenvalue spectra of the Jacobians that characterize network dynamics around fixed points. We show that either all fixed points are stable or all of them are unstable, depending on whether their number is below a ``critical load for stability'', which is distinct from the classical critical capacity that measures the maximal number of achievable fixed points regardless of their stability. We further analyze the dependence of this critical load for stability on experimentally measurable quantities characterizing the statistics of memory patterns and the activation functions of neurons. Our analysis highlights the computational benefits of sparse-like patterns and threshold-linear activation functions and offers testable predictions for neural circuits supporting memory.

Neuromorphic and quantum computing have recently emerged as promising paradigms for advancing artificial intelligence, each offering complementary strengths. Neuromorphic systems built on spiking neurons excel at processing time series data efficiently through sparse, event-driven computation, consuming energy only upon input events. Quantum computing, on the other hand, operates on state spaces that grow exponentially in dimension with the number of qubits -- as a consequence of tensor-product composition -- with quantum states admitting superposition across basis states and entanglement between subsystems. Hybrid approaches combining these paradigms have begun to show potential, but existing quantum spiking models have important limitations. Notably, they implement classical memory mechanisms on single qubits, requiring repeated measurements to estimate firing probabilities, while relying on conventional backpropagation for training. In this paper, we propose a novel stochastic quantum spiking (SQS) neuron model that addresses these challenges. The SQS neuron uses multi-qubit quantum circuits to realize a spiking unit with internal quantum memory, enabling event-driven probabilistic spike generation in a single shot during inference. Furthermore, we study networks of SQS neurons, dubbed SQS neural networks (SQSNN), and demonstrate that they can be trained via a hardware-friendly local learning rule, eliminating the need for global classical backpropagation. The proposed SQSNN model is shown via experiments with both conventional and neuromorphic datasets to improve over previous quantum spiking neural networks, as well as over classical counterparts, when fixing the overall number of trainable parameters, highlighting its potential for event-driven applications such as neuromorphic integrated sensing and communications (N-ISAC).

Neuromorphic computing aims to improve the efficiency of artificial neural networks by taking inspiration from biological neurons and leveraging temporal sparsity, spatial sparsity, and compute near/in memory. Although these approaches have shown efficiency gains, training these spiking neural networks (SNN) remains difficult. The original attempts at converting trained conventional analog neural networks (ANN) to SNNs used the rate of binary spikes to represent neuron activations. This required many simulation time steps per inference, which degraded efficiency. Intel's Loihi 2 is a neuromorphic platform that supports graded spikes which can be used to represent changes in neuron activation. In this work, we use Loihi 2's graded spikes to develop a method for converting ANN networks to spiking networks, which exploits temporal and spatial sparsity. We evaluated the performance of this network on Loihi 2 and compared it to NVIDIA's Jetson Xavier edge AI platform. The results show that neuromorphic approaches achieve significant improvements in efficiency and latency (energy-delay product) over existing solutions.

Continuous adaptive learning, the ability to adapt to the environment and keep improving performance, is a hallmark of natural intelligence. Biological organisms excel in acquiring, transferring, and retaining knowledge while adapting to volatile environments, making them a source of inspiration for artificial neural networks (ANNs). This study explores how neuromodulation, a building block of learning in biological systems, can help address catastrophic forgetting and enhance the robustness of ANNs in continual learning. Driven by neuromodulators including dopamine (DA), acetylcholine (ACh), serotonin (5-HT) and noradrenaline (NA), neuromodulatory processes in the brain operate at multiple scales, facilitating dynamic responses to environmental changes through mechanisms ranging from local synaptic plasticity to global network-wide adaptability. Importantly, the relationship between neuromodulators and their interplay in modulating sensory and cognitive processes is more complex than previously expected, demonstrating a "many-to-many" neuromodulator-to-task mapping. To inspire neuromodulation-aware learning rules, we highlight (i) how multi-neuromodulatory interactions enrich single-neuromodulator-driven learning, (ii) the impact of neuromodulators across multiple spatio-temporal scales, and correspondingly, (iii) strategies for approximating and integrating neuromodulated learning processes in ANNs, and (iv) an architectural-general formulation of multi-neuromodulatory dynamics. We also present a conceptual study to showcase how neuromodulation-inspired mechanisms, such as DA-driven reward processing and NA-based cognitive flexibility, can enhance ANN performance in a Go/No-Go task. Though multi-scale neuromodulation, we aim to bridge the gap between biological and artificial learning, paving the way for ANNs with greater flexibility, robustness, and adaptability.

Reinforcement learning (RL) has demonstrated remarkable capabilities in training agents to solve complex tasks autonomously, such as mobile robots, UAVs/UGVs, and game-playing agents). However, scaling RL to master multiple tasks simultaneously (i.e., so-called multi-task RL) remains a significant challenge. Such a multi-task RL capability especially is important for agents to adapt to changes in real-world operational environments. State-of-the-art works show that, training agents with neural networks and shared structures across tasks promises improved generalization in simultaneous multi-task RL. However, they still suffer from task interference and incur high energy consumption due to intensive computation. To address this, we propose MTSpark, a novel methodology that enables energy-efficient simultaneous multi-task RL using spiking neural networks (SNNs) equipped with active dendrites for bio-inspired generalist agents. Specifically, MTSpark enhances a Deep Spiking Q-Network (DSQN) with active dendrites, a dueling structure, and task-specific context signals to dynamically form specialized sub-networks for individual tasks, while exploiting sparse operations for energy-efficient network processing. Experimental results demonstrate that MTSpark achieves higher performance and efficiency compared to state-of-the-art by obtaining high scores across three Atari games (i.e., Pong: -5.4, Breakout: 0.6, and Enduro: 371.2), approaching human-level performance (i.e., Pong: -3, Breakout: 31, Enduro: 368), while incurring similar memory and about 2x lower energy than state-of-the-art. These results show that our MTSpark potentially advances the frontiers toward energy-efficient generalist agents by combining RL and SNNs.

AI is rapidly advancing neuroscience, yet many laboratories fail to fully unleash its potential due to significant interdisciplinary barriers. While pre-trained neural models for physiological data are progressing quickly, their heterogeneous architectures and modality-specific constraints hinder systematic integration, selection, and evaluation. Despite recent advances in large language model (LLM)-based agent systems for intelligent scientific applications, existing approaches often still lack the domain expertise required to effectively select and coordinate diverse neuroscience pre-trained models and handle unique data types in this domain. We present NS-Copilot, an LLM-driven multi-agent system for neuroscience analysis that autonomously supports end-to-end workflows for diverse professional tasks. It unifies domain-specific pre-trained models and supports key neuroscience modalities, including EEG and extracellular spike data, through a natural-language interface. Given raw data and a task description, NS-Copilot orchestrates agents with specialized roles for planning, adaptive control, code generation, and result synthesis, enabling analysis without dataset-specific heuristics. We evaluate NS-Copilot on neuroscience benchmarks spanning Alzheimer's disease, Parkinson's disease, and working memory spike decoding. Across 8 trials per task, the system consistently outperforms strong baselines on the primary metric, demonstrating the ability of NS-Copilot for effective and scalable neuroscience analysis. Our code is publicly available at https://github.com/FrankLiu1102/ns-copilot.

The brain uses discrete spikes for dynamic computation, yet, how neural microcircuits (NMCs) solve temporal credit assignment using local spike timing remains a fundamental open question. Dominant spiking neural network (SNN) approaches circumvent this by approximating backpropagation through surrogate gradients, decoupling learning from biological spike timing. Here, we reformulate temporal credit assignment as a state separation problem: extracting task-required components induced by historical perturbations directly from the current neural state. This enables an online feedback learning framework for NMCs through a gradient tunneling (GT) algorithm and the lead-lag expansion technique that derives credit assignment from local synaptic spike timing, while remaining compatible with ANN-SNN hybrid architectures. Experimentally, GT-trained NMCs excel at long-timescale evidence integration and noise-robust memory retention, and perform comparably to leading SNN online learning methods on real-world benchmarks with far fewer parameters. The proposed framework addresses the two-decade-old NMC feedback learning problem and suggests a computationally plausible explanation for the brain's learning mechanisms.