Backpropagation
Overview
Backpropagation uses the chain rule to send error signals from a model’s outputs backward through its layers, providing gradients for updating parameters. Work improves its efficiency and reach through memory-saving gradient methods, parallel scans, gradient approximations, and better scheduling, including for recurrent models and large language models. Alternatives such as feedback alignment, predictive coding, and forward-only local learning aim to reduce global backward communication or support biological and physical hardware. The central tension is that these alternatives may improve locality or memory use, but often struggle to match backpropagation’s accuracy, convergence, or measured end-to-end energy efficiency.
Latest papers
48 papers6 papers in the last four weeks, up 20% on the four weeks before. 0.0% of all new papers.
CAPSkip: Cancellation-Aware Prefix Skipping for Selective Backpropagation
Which Chains to Calibrate for Physical Backpropagation: Per-User Factors Versus Base-Station Chains
Detailed Derivation of The Backpropagation Algorithm
Neural LoFi with Backward Coupling: A Spectral Theory of Cross-Layer Feature Learning
How Much Gradient Does Truncated Backprop Lose in Looped Transformers?
FidLIF: Parallel Training of High-Fidelity Spiking Neurons with Exact BPTT
Connected Self Forcing: Beyond Local Learning in Video Autoregression
Backward-State Policy Is Part of the Learning Algorithm
The Price of Locality: Why Forward-Forward Underperforms Backpropagation?
LocalProp: Neuro-Localized Memory-Efficient Backpropagation
The Ups and Downs of Backprop Weights
Large Distant Gradients Need Not Be Reliable: reliability-weighted credit assignment for long-horizon autoregressive forecasting
Learning Length-Extrapolatable Recurrent Models
Wave Function Backpropagation with Explicit Temporal-Interval Dynamics
The Forward-Backward Disconnect: State Dynamics, Credit Assignment, and Biological Grounding in Neural Computation
ERASE: EaRly bAckpropagation SchEdule for Faster Training of Modern Recommendation Systems
Unifying Physical Backpropagation
Learning in Deep Networks under Dale's Constraint
Reconstructing Backpropagation from Forward Fluctuations in Noise-modulated Neural Networks
Continual-RL for Generalization in Autonomous Racing on the RoboRacer Platform
JAGG: Jacobian-Aggregated Group Gradient for Efficient GRPO Training of Diffusion Models
jagg\_frac) deploys JAGG only where the assumption holds. Experiments on T2I benchmarks show JAGG delivers 2 backward speedup with negligible quality degradation. The code for this work can be accessed through https://github.com/SchumiDing/JAGG.Building a Neural Network from Scratch: Implementation, Evaluation, and Optimization
Backpropagation-Free Trunk Training via the Split Forward Gradients
Backpropagation as a Nilpotent Linear System
K-ABENA: K-Adaptive Backpropagation with Error-based N-exclusion Algorithm : (Compensated Loss-Based Sample Exclusion with Unbiased Gradient Estimation)
FORGE: Fused On-Register Gradient Elimination for Memory-Efficient LLM Training
Automatic Differentiation from Scratch: How PyTorch Computes Gradients in Physics-Informed Neural Networks
create\_graph=True enables correct differentiation through the physics-informed residual. Every adjoint value is verified against the hand derivations of Tahimi (2026), connecting the sensitivity framework to the vector--Jacobian products used by PyTorch's autograd engine.