Backpropagation

Follow this topic Ask about this topic

Momentum

6 papers in the last four weeks, up 50% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 96

All topics

Increasing Width Allows Greedy Layer-wise Training to Rival End-to-End Backpropagation in Self-Supervised Learning
Sep 30, 2026Syon Mansur, Joel ZylberbergBackpropagation Layer-Wise
Jules Stein Eye Institute University of California, Los Angeles Los Angeles, CA 90095, USA
SOLO: Pretraining Billion-Parameter Language Models with Shared-Output Local Learning
Sep 28, 2026Bojian Yin, Shurong Wang, Yuqi Pan +1Large Language Model Pretraining Singular Learning Theory
Institute of Automation, Chinese Academy of Sciences
LVMT: Video Mask Transformer for Long-term Video Segmentation
Sep 28, 2026Narges Norouzi, Niccolò Cavagnero, Idil Esen Zulfikar +3Video Object Segmentation Mask2Former
Eindhoven University of Technology · RWTH Aachen University
Rank and computation of the pathlifting Jacobian of a DAG ReLU network
Sep 16, 2026Manon VerbockhavenRectified Linear Unit Networks Jacobian
OCKHAM
Learning Length-Extrapolatable Recurrent Models
Sep 8, 2026Hanwen JiangRecurrent Model Backpropagation
Adobe Research
MpSub: A Momentum $p$ -Dimensional Subspace Trust-Region Method for Derivative-Free Fine-Tuning of Large Language Models
Sep 7, 2026Yuyang Wang, Haoyu Yao, Pengcheng XieTrust Region Momentum
Xi’an Jiaotong University, Xi’an, China · Lawrence Berkeley National Laboratory, Berkeley, CA, USA
Wave Function Backpropagation with Explicit Temporal-Interval Dynamics
Sep 1, 2026Byunggu Yu, Justin KimComplex Wavefield Backpropagation
Dept. of Computer Science and Information Technology University of the District of Columbia Washington, DC USA
Program Learning with Verifiable Rewards: Symbolic Backpropagation for Post-Training LLMs
Aug 28, 2026Vishvesh BhatReinforcement Learning With Verifiable Reward Verifiable Rewards
CoreThink AI
Unifying Physical Backpropagation
Aug 12, 2026Cyrill Bösch, Yigithan Gediz, Clara C. Wanjura +1Backpropagation Adjoint Matching
Department of Computer Science, Princeton University, Princeton, NJ 08540, USA · Department of Electrical and Computer Engineering, Princeton University, Princeton, NJ 08540, USA
Evaluation Resolution Confounds Learning-Rule Comparisons in Model-Brain RSA of Early Visual Cortex
Aug 11, 2026Nils LeuteneggerHuman Visual Cortex Visual Perception
Independent Researcher, Switzerland
Learning in Deep Networks under Dale's Constraint
Aug 7, 2026Roy Abel, Shimon UllmanNeural Circuits Neurons
Weizmann Institute of Science
Test-Time Scaling for Safe Text-Guided Image Generation via Intermediate Clean Estimates
Aug 4, 2026Jinya Sakurai, Shueicheng Yan, Xun XuText-To-Image Diffusion Models Text-To-Image
1Nanyang Technological University · 2National University of Singapore · Institute of Advanced Intelligence and Computing (IAIC), A*STAR +1
LoCA: Forward-Only LLM Tuning after One-Shot Calibration with Local Credit Assignment
Aug 4, 2026Linhan Xia, Rui Liu, Zhaofeng Zhang +3Multi-Lora Per-Node Correction
University of Oklahoma · 4Tencent · 2Imperial College London +5
Crayotter: Learning Long-Horizon Video Editing Agents via Group-Relative Preference Backpropagation
Aug 3, 2026Lecheng Yan, Jianze Lin, Yichong Zhang +5Video Editing Video Agent
University of Science and Technology of China · Beijing Normal University · Jilin University +4
Reconstructing Backpropagation from Forward Fluctuations in Noise-modulated Neural Networks
Jul 29, 2026Shuhei IkemotoBackpropagation Neural Network
Graduate School of Life Science and Systems Engineering, Kyushu Institute of Technology, Kitakyushu, Japan
Physics-Informed Broad Learning System: An Efficient Backpropagation-Free Framework for Solving Partial Differential Equations
Jul 28, 2026Pinki Khatun, M. Sajid, Abhinav Jha +1Parametric Physics-Informed Neural Network Physics-Informed Learning
Department of Industrial Engineering, Università degli Studi di Firenze, Viale Morgagni 40/44, 50134 Firenze, Italy · Department of Mathematics, Indian Institute of Technology Indore, Indore 453552, India · Department of Mathematics, Indian Institute of Technology Gandhinagar, Palaj, Gandhinagar, 382055, Gujarat, India
End-to-End Learning of Safe Optimal Feedback Control in High Dimensions with Control Barrier Function Layers
Jul 22, 2026Xingjian Li, Kelvin Kan, Deepanshu Verma +3Control Barrier Functions Safety Filters
UT Austin · UCLA · Clemson University +1
Elicitation without Backpropagation: Steering Model Behavior by Optimizing the Latent Posterior
Jul 21, 2026Garrett Baker, Vinayak Pathak, Daniel Murfet +1Posterior Transformer Architectures
Timaeus · CORAL · Monash University
Conditioned Direct Feedback Alignment via Activity and Error Geometry
Jul 20, 2026Houman Safaai, Varun Reddy, Bernardo L. SabatiniBackpropagation Synaptic Plasticity
Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University · Department of Neurobiology, Howard Hughes Medical Institute, Harvard Medical School
Empowering On-Device Model Adaptation with an Edge AI Inference Accelerator
Jul 20, 2026Mateusz Piechocki, Alessandro Capotondi, Marek KraftEdge Devices Parameter-Efficient Adaptation
Poznan University of Technology, Institute of Robotics and Machine Intelligence, Poznan, Poland · University of Modena and Reggio Emilia, The High-Performance Real-Time Laboratory, Modena, Italy
The Art of Not Forgetting
Jul 20, 2026Ashmith Atmuri, Akshay Kumar, Yashaswini Rao BhogarajulaCatastrophic Forgetting Backpropagation
Arkadhi Labs
FlexiGrad: Adaptive Gradient Modulation for Hierarchical Fine-Grained Classification
Jul 20, 2026Zilu Zhou, Dongliang Chang, Junhan Chen +1Fine-Grained Perception Gradient Component
Beijing University of Posts and Telecommunications, Beijing 100876, China · Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance, Beijing 100876, China
Building a Neural Network from Scratch: Implementation, Evaluation, and Optimization
Jul 18, 2026Yuanzhe JiaBackpropagation Neural Network
University of Sydney, Australia
Backpropagation-Free Trunk Training via the Split Forward Gradients
Jul 18, 2026Tian Qin, Wei-Min HuangBackpropagation Gradient
Math department Lehigh University Bethlehem, PA 18015, USA
Local Additive Feature Attribution: A Mathematical Taxonomy and Reporting Checklist
Jul 15, 2026Rebecca Afriyie Sarpong, Daniel CommeyShapley Additive Explanations Explainable Artificial Intelligence
Weight Feedback Computes the Jacobian Transpose Locally in Modern Deep Networks
Jul 15, 2026Junlong Shen, Xingyu LiBackpropagation Predictive Coding
Department of Electrical and Computer Engineering, University of Alberta, Edmonton, Canada · Alberta Machine Intelligence Institute (Amii), Edmonton, Canada
From Preimage Search To Source-Grounded Feature Inversion
Jul 14, 2026Kaixiang ShuInverse Design Feature Learning
Independent Researcher
Backpropagation as a Nilpotent Linear System
Jul 13, 2026Ahmed BoughammouraBackpropagation Backward Passes
Higher Institute of Informatics and Mathematics of Monastir, University of Monastir, 5000 Monastir, Tunisia.
CARE-LoRA: Compressed Activation REconstruction for Memory-Efficient LoRA
Jul 11, 2026Gengyu Zhang, Haiyin Ran, Zhengbao He +4Low-Rank Adaptation Framework Large Language Model Compression
Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai 200240, China
Beyond Backpropagation: Monte Carlo Method Can Train Deep Neural Networks
Jul 9, 2026Hong ZhaoBackpropagation Gradient-Based Optimization
Department of Physics, Xiamen University, Xiamen 361005, China · Lanzhou Center for Theoretical Physics, Lanzhou University, Lanzhou 730000, China
On the Principles of Deep Feedforward ReLU Networks
Jul 8, 2026Changcun HuangRectified Linear Unit Networks Backpropagation
Shuitu Institute of Applied Mathematics, Chongqing 400700, P.R.C
K-ABENA: K-Adaptive Backpropagation with Error-based N-exclusion Algorithm : (Compensated Loss-Based Sample Exclusion with Unbiased Gradient Estimation)
Jul 7, 2026Jean-Francois BonbhelGradient Boosted Decision Tree Backpropagation
NeuroSoft IA, Qu´ebec City, Canada · YekoElite University, Brazzaville, Republic of Congo
SEAM: Smooth Execution of Action-Chunked Motion for Vision-Language-Action Policies
Jul 6, 2026Dijia Zhan, Xuemiao Xu, Jinyi Li +1Flow-Matching Vision-Language-Action Action Chunks
South China University of Technology
Cross-device Collaborative Test-time Adaptation with Zeroth-order Optimization and Model Merging
Jul 3, 2026Yu Mitsuzumi, Akisato Kimura, Yasuhiro Fujiwara +1Test-Time Adaptation Zeroth-Order Optimization
NTT, Inc. · Kyoto University
MedSaab-US: A Backpropagation-Free Multi-Scale Wavelet-Saab Framework for Thyroid Nodule Segmentation in Ultrasound Images
Jul 2, 2026Mohammad Amanour RahmanUltrasound Deep Learning
Department of Computer Science and Engineering Ahsanullah University of Science and Technology (AUST) Dhaka, Bangladesh
Diffusing Blame: Task-Dependent Credit Assignment in Biologically Plausible Dual-Stream Networks
Jun 30, 2026Yutaro Yamada, Luca Grillotti, Rujikorn Charakorn +3Backpropagation Credit Assignment
Sakana AI
GPU Parallelization Strategies for Forward and Backward Propagation in Shallow Neural Networks: A CUDA-Based Comparative Study
Jun 29, 2026Rania Zitouni, Nadine Bousdjira, Sarah Hasnaoui +2Cuda Cpu-Gpu Hybrid Designs
E´ cole Nationale Supe´rieure d’Informatique (ESI) · High Performance Computing (HPC)
Supervised Hebbian learning in Deep Counterstream Associative Networks
Jun 28, 2026Andreas KnoblauchHebbian Learning Backpropagation
Albstadt-Sigmaringen University, KEIM Institute, Poststrasse 6, 72458 Albstadt-Ebingen, Germany
Symplectic Neural Networks for learning Generalized Hamiltonians
Jun 25, 2026Harsh Choudhary, Vyacheslav Kungurtsev, Chandan Gupta +2Hamiltonian Neural Ordinary Differential Equations
Computer Science Department, Indraprasth Institute of Information Technology, New Delhi, India 110020 · Athena Research and Innovation Center, Marousi, Greece 15125 · HSBC Holdings, Singapore
Black-Box Continual Learning for Vision-Language Models
Jun 22, 2026Yuting Li, Weihang Fang, Haoyuan Gao +4Replay-Based Continual Learning Continual Learning
School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China · State Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI), Beijing, China +2
Backpropagating Through Simulation: Analytic Policy Gradients for Sample and Learning Efficient Differentiable Continuous Control
Jun 19, 2026Yueci DengPolicy Gradient Frictive Policy Optimization
School of Data Science, The Chinese University of Hong Kong, Shenzhen
Breaking chains with trees: Deep learning with $\mathcal{O}(\log N)$ parallel time complexity
Jun 19, 2026Neeraj Mohan Sushma, Aditya Nagarsekar, Cabrel Teguemne Fokam +4Deep Learning Architectures Backpropagation
Center for Cognitive Interaction Technology, Bielefeld University, Germany · Ruhr Universität Bochum, Germany · Birla Institute of Technology and Science, Pilani, Goa Campus +1
SDS-LoRA: Overcoming Anisotropic Gradient Scaling in Low-Rank Adaptation
Jun 15, 2026Junghun Oh, Sungyong Baik, Kyoung Mu LeeLow-Rank Structure Hip-Lora
Dept. of ECE, ASRI Seoul National University · Dept. of Artificial Intelligence Dept. of Data Science Hanyang University
Beyond Backbone Backpropagation: A Decoupled Strategy for Efficient Transfer Learning
Jun 13, 2026Daniel Vila-Cruz, Laura Morán-Fernández, Verónica Bolón-CanedoTransfer Learning Convolutional Neural Networks
CITIC, Universidade da Coruña, A Coruña, Spain
Graph-based Target Back-Propagation for Context Adaptation in Multi-LLM Agentic Systems
Jun 12, 2026Tan Zhu, Tong Yao, Kananart Kuwaranancharoen +4Prior Prompt Optimization Techniques Agentic Workflow Design
Retail Intelligence, Walmart Global Tech, Sunnyvale, CA, USA
A Programmer's Guide to Cascaded Adaptive Combiners: Online Learning by Biologically Accurate Models of Multilayer Neuron Networks
Jun 12, 2026Martin Nilsson, Denis KleykoNeurons Backpropagation
RISE Research Institutes of Sweden, Kista, Sweden · Örebro University, Örebro, Sweden
TimeROME-DLM: Temporal Causal Tracing and Low-Rank Inference-Time Knowledge Editing for Masked Diffusion Language Models
Jun 11, 2026Zhengtao Yao, Liuyang Song, Hongbo Zhang +4Masked Diffusion Language Models Multimodal Knowledge Editing
University of Southern California, USA · Peking University, China · Stevens Institute of Technology, USA +1
Context-Driven Incremental Compression for Multi-Turn Dialogue Generation
Jun 10, 2026Yeongseo Jung, Jaehyeok Kim, Eunseo Jung +5Dialogue Benchmarks Efficient Long-Context Inference
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong · NVIDIA AI Technology Center, NVIDIA · University of Tsukuba, Japan +1
$μ$ VLA: On Recurrent Memory for Partially Observable Manipulation in VLA Models
Jun 10, 2026Egor Cherepanov, Nikita Kachaev, Daniil Zelezetsky +6Diffusion-Based Vision-Language-Actions Recurrent Model
CogAI Lab, Moscow, Russia · MIRAI, Moscow, Russia
Fine-tuning Multi-modal LLMs with ART: Art-based Reinforcement Training
Jun 10, 2026Michal Chudoba, Sergey Alyaev, Petra Galuscakova +1Multimodal Large Language Models Model Fine-Tuning
University of Stavanger · Stavanger, Norway · NORCE Research +1
Overcoming Rank Collapse in Feedback Alignment
Jun 9, 2026Gauthier Boeshertz, Razvan Pascanu, Claudia ClopathBackpropagation Orthogonality
Department of Bioengineering Imperial College London · Mila
Exploring the Design Space of Reward Backpropagation for Flow Matching
Jun 9, 2026Ruoyu Wang, Boye Niu, Xiangxin Zhou +3Backpropagation Performance Metric
1Westlake University · 2Tencent Hunyuan · University of Sydney +1
Accelerating NeurASP with vectorization and caching
Jun 9, 2026Alexander Philipp Rader, Alessandra RussoNeuro-Symbolic Framework Backpropagation
Imperial College London
Predictive Coding with Bayesian Priors via Proximal Gradients
Jun 6, 2026Francesco BulloPredictive Coding Posterior Predictive Distribution
Mechanical Engineering, 2325 Engineering Bldg II, UCSB, Santa Barbara, CA 93106-5070, USA
Pretraining Recurrent Networks without Recurrence
Jun 4, 2026Akarsh Kumar, Phillip IsolaRecurrent Neural Networks Backpropagation
MIT
Instance-Level Post Hoc Uncertainty Quantification in Object Detection
Jun 3, 2026Chongzhe Zhang, Zifan Zeng, Qunli Zhang +2Uncertainty Quantification Object Detection
RAMS Lab, Huawei Heisenberg Research Center, Munich, Germany · Technische Universität Berlin, Berlin, Germany · Technische Universität München, Munich, Germany
FFR: Forward-Forward Learning for Regression
Jun 2, 2026Xinyang Liu, Xuanyu Liang, Shiqi Ding +4Backpropagation Forward Propagation
University of Bristol · University College London · University of Cambridge +1
Drifting Preference Optimization for One-Step Generative Models
Jun 1, 2026Zhou Jiang, Yandong Wen, Zhen LiuDiffusion Alignment Few-Step Generation
Westlake University · The Chinese University of Hong Kong, Shenzhen
Score Broadcast and Decorrelation: A General Framework for Broadcast-Based Credit Assignment
May 28, 2026Mustafa Uzun, Mete Erdogan, Cengiz Pehlevan +1Loss Function Credit Assignment
KUIS AI Center, Koc University, Turkey · Electrical and Electronics Engineering, Koc University, Turkey · Department of Electrical Engineering, Stanford University, USA +3
AMDP: Asynchronous Multi-Directional Pipeline Parallelism for Large-Scale Models Training
May 28, 2026Ling Chen, Houming Wu, Wenjie YuCpu-Gpu Hybrid Designs Asynchronous Execution
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China · College of Computer Science and Technology, Zhejiang University, Hangzhou, China

Sep 30, 2026, cs.LGJ/K move · Enter open · S save

Increasing Width Allows Greedy Layer-wise Training to Rival End-to-End Backpropagation in Self-Supervised Learning

Syon Mansur, Joel Zylberberg

Jules Stein Eye Institute University of California, Los Angeles Los Angeles, CA 90095, USA

Backpropagation Layer-Wise Self-Supervised Learning Deep Learning

Open paper AskPDF arXiv

End-to-end backpropagation has been the dominant mode of training in deep learning, allowing for the coordination of parameter updates across layers of a neural network. Prior studies have explored alternative -- and, in some cases, simpler -- training mechanisms, showing that they can sometimes achieve performance similar to backpropagation. However, the architectural conditions under which locally optimized networks, which avoid end-to-end backpropagation of error, can learn representations comparable to those learned through end-to-end training remain unclear. We aim to answer this question in the context of self-supervised learning, an important framework for large-scale pretraining in artificial intelligence. Here, we investigate how network width and depth affect the efficacy of greedy layer-wise and end-to-end self-supervised training in convolutional networks. We find that in wider networks, the benefits of end-to-end backpropagation over greedy layer-wise training shrink: in relatively shallow and very wide networks, we even observed higher performance in models trained with greedy layer-wise training. Subsequent analysis of the representations formed by these networks shows that very wide greedy-trained networks exhibit more favorable representational geometry than do networks trained end-to-end with backpropagation. This work shows that width can compensate for restricted credit assignment and identifies differences in representational geometry as a potential mechanism for their improved performance.

Momentum

Related topics

Latest papers 96