Overview

Backpropagation uses the chain rule to send error signals from a model’s outputs backward through its layers, providing gradients for updating parameters. Work improves its efficiency and reach through memory-saving gradient methods, parallel scans, gradient approximations, and better scheduling, including for recurrent models and large language models. Alternatives such as feedback alignment, predictive coding, and forward-only local learning aim to reduce global backward communication or support biological and physical hardware. The central tension is that these alternatives may improve locality or memory use, but often struggle to match backpropagation’s accuracy, convergence, or measured end-to-end energy efficiency.

Latest papers

CardsList

48 papers6 papers in the last four weeks, up 20% on the four weeks before. 0.0% of all new papers.

  1. Augmented Lagrangian Predictive Coding

    May 29, 2026Jeffrey Seely, Julian GouldBackpropagationPredictive Coding

  2. PINNs Failure Modes are Overfitting

    May 29, 2026Nigel T. Andersen, Takashi MatsubaraNeural Network GeneralizationNeural PDE Solvers

  3. Misalignment Between Backpropagation and the Hierarchy of Brain Responses to Images

    May 27, 2026Joséphine Raugel, Maximilian Seitzer, Marc Szafraniec +6BackpropagationVisual Representation Learning

  4. Understanding Sample Efficiency in Predictive Coding

    May 12, 2026Gaspard Oliviers, Elene Lominadze, Rafal BogaczBackpropagationPredictive Coding

  5. Efficient and provably convergent end-to-end training of deep neural networks with linear constraints

    May 12, 2026Zonglin Yang, Zhexuan Gu, Yancheng YuanDeep Learning OptimizationBackpropagation

  6. LBI: Parallel Scan Backpropagation via Latent Bounded Interfaces

    May 9, 2026Shaun Christopher Lee, Sangeetha Abdu JyothiDeep Learning OptimizationBackpropagation

  7. Physics-Informed Neural Networks: A Didactic Derivation of the Complete Training Cycle

    Apr 20, 2026Abdeladhim TahimiBackpropagationOrdinary Differential Equations

  8. Local learning for stable backpropagation-free neural network training towards physical learning

    Mar 25, 2026Yaqi Guo, Fabian Braun, Bastiaan Ketelaar +3Physical Neural NetworksNeural Network Optimization

  9. A Variational Latent Equilibrium for Learning in Neuronal Circuits

    Mar 10, 2026Simon Brandt, Paul Haider, Walter Senn +2Temporal Credit AssignmentNeural Network Training Dynamics

  10. Riemannian Optimization in Modular Systems

    Mar 4, 2026Christian Pehle, Jean-Jacques SlotineContraction TheoryRiemannian Optimization

  11. Unbiased Approximate Vector-Jacobian Products for Efficient Backpropagation

    Feb 16, 2026Killian Bakong, Laurent Massoulié, Edouard Oyallon +1BackpropagationDeep Learning Optimization

  12. Perforated Backpropagation: A Neuroscience Inspired Extension to Artificial Neural Networks

    Jan 29, 2025Rorry Brenner, Laurent IttiBackpropagation