cs.LGOct 29, 2026

AI at a Turning Point: From ChatGPT through Neural Networks to Neuromorphic AI

Authors: Volker Kempe

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The tension between the established technologies of connectionist AI and their neuromorphic challengers is attracting growing attention, above all because of the enormous energy demand of the former and the low power consumption of the latter. Developing and running large AI models consumes gigawatt- to terawatt-hours of energy, while the human brain operates on a continuous 20 watts. Spiking neural networks (SNNs) counter this energy demand with a frugal approach and bring elements of self-organization into the methodological repertoire of AI: local learning instead of global error backpropagation, event-driven computing, and time as a carrier of information. They do not replace today's AI but extend it; in hybrid systems each approach can be deployed where its strengths lie. This monograph explains the field in a generally accessible yet mathematically rigorous way. Using the text-generation module of ChatGPT as an example, the principle, elegance, and computational cost of transformer-based AI are presented and compared with the coding, learning, and network-design methods of SNNs. Throughout, relevant mechanisms of the human nervous system serve as model and reference point. Topics include spike-timing-dependent plasticity, population and temporal coding, the Neural Engineering Framework, neuromorphic hardware from Loihi 2 to commercial edge processors, and the prospects for classical and neuromorphic AI to complement and extend each other.

Figures & tables

Figure 1
Fig. 2.2 Transformer network
Figure 3
ParameterValues for GPT-3
Dimension of the model space dmodeld_{\mathrm{model}}12,288
Dimension of the FFN dffd_{\mathrm{ff}}4 dmodel=49,1524\,d_{\mathrm{model}}=49{,}152
Number of transformer blocks (layers)96
Model typedense autoregressive transformer
Total number of trainable parameters175 billion ( 1.75⋅10111.75\cdot 10^{11} )
Number of training tokensabout 300 billion
Table 4
Fig. 3.1 The neuron
Fig. 3.2 Time course of an action potential with refractory periods
Fig. 3.3 Structure and function of chemical synapses
Fig. 3.4 Interaction of excitatory and inhibitory potentials
Fig. 3.5 Spatial and temporal integration of synaptic inputs
Fig. 3.6 The LIF model
Fig. 3.7 Idealization of a spike event
Fig. 3.8 Schematic representation of an STDP learning window
Fig. 4.1 Principle of a learning SNN
Fig. 4.2 The firing rate
Fig. 4.3 Population coding
Fig. 4.4 Tuning curves for representing the input variable
Fig. 4.6 Schematic representation of the Loihi 2 system
Fig. 4.7 The processing pipeline of a Loihi 2 core
More in this topic

Neuromorphic Computing

Neuromorphic computing uses brain-inspired, event-driven algorithms and hardware to process information with low power and latency.

See all 197 papers

Explore similar work

CardsList
Mar 19, 2026

Brain-inspired AI for Edge Intelligence: a systematic review

While Spiking Neural Networks (SNNs) promise to circumvent the severe Size, Weight, and Power (SWaP) constraints of edge intelligence, the field currently faces a "Deployment Paradox" where theoretical energy gains are frequently negated by the inefficiencies of mapping asynchronous, event-driven dynamics onto traditional von Neumann substrates. Transcending the reductionism of algorithm-only reviews, this survey adopts a rigorous system-level hardware-software co-design perspective to examine the 2020-2025 trajectory, specifically targeting the "last mile" technologies - from quantization methodologies to hybrid architectures - that translate biological plausibility into silicon reality. We critically dissect the interplay between training complexity (the dichotomy of direct learning vs. conversion), the "memory wall" bottlenecking stateful neuronal updates, and the critical software gap in neuromorphic compilation toolchains. Finally, we envision a roadmap to reconcile the fundamental "Sync-Async Mismatch," proposing the development of a standardized Neuromorphic OS as the foundational layer for realizing a ubiquitous, energy-autonomous Green Cognitive Substrate.

Yingchao Cheng, Meijia Wang, Zhifeng Hao +1

Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ), Shenzhen, 518107, China · Cloud Computing and Distributed Systems (CLOUDS) Lab, School of Computing and Information Systems, University of Melbourne, Melbourne, VIC 3010, Australia · College of Mathematics and Computer Science, Shantou University, Shantou, 515063, China

Neuromorphic ComputingSpiking Neural NetworksEnergy-Efficient MLEdge ML
AskPDF
Apr 6, 2026

Neuromorphic Computing for Low-Power Artificial Intelligence

Classical computing is beginning to encounter fundamental limits of energy efficiency. This presents a challenge that can no longer be solved by strategies such as increasing circuit density or refining standard semiconductor processes. The growing computational and memory demands of artificial intelligence (AI) require disruptive innovation in how information is represented, stored, communicated, and processed. By leveraging novel device modalities and compute-in-memory (CIM), in addition to analog dynamics and sparse communication inspired by the brain, neuromorphic computing offers a promising path toward improvements in the energy efficiency and scalability of current AI systems. But realizing this potential is not a matter of replacing one chip with another; rather, it requires a co-design effort, spanning new materials and non-volatile device structures, novel mixed-signal circuits and architectures, and learning algorithms tailored to the physics of these substrates. This article surveys the key limitations of classical complementary metal-oxide-semiconductor (CMOS) technology and outlines how such cross-layer neuromorphic approaches may overcome them.

Keshava Katti, Pratik Chaudhari, Deep Jariwala

Electrical & Systems Engineering, University of Pennsylvania, Philadelphia, USA.

Analog ComputingNeuromorphic ComputingEnergy-Efficient MLCompute-in-Memory
AskPDF
Sep 4, 2026

Large Language Models with At Most One Spike per Neuron

Leveraging their inherent sparse event-driven computation, spiking neural networks (SNNs) offer a promising path toward energy-efficient large language models (LLMs). Time-to-first-spike (TTFS) coding generates at most one spike per neuron within a time window, yielding extremely low firing rates. However, conventional TTFS SNNs are restricted to specific structures, making it challenging to encode certain blocks in LLM -- such as layer normalization and matrix multiplication --using TTFS. To overcome this limitation, we introduce a reference-based strategy specifically to encode the four core LLM components: embedding layers, layer normalization, attention-related operations and dropout. We construct a fully TTFS-based SNN architecture and train it end-to-end. Experiments on modern LLMs like BERT and GPT-2 demonstrate that our approach achieves performance comparable to ANN counterparts on natural language understanding and common-sense reasoning, while a clear gap remains on language modeling perplexity. To the best of our knowledge, this is the first work to scale a spiking LLM to 1.5 billion parameters using TTFS coding. We also report an estimate of spike-related energy; this is a spike-count proxy under an established cost model rather than a measurement on neuromorphic hardware.

Zhuoya Zhao, Parsa Omidi, Aref Jafari +1

Spiking Language ModelsSpiking Neural NetworksSpike-Based Signal EncodingSpiking TransformersLanguage Modeling
AskPDF