Residual Learning

Follow this topic Ask about this topic

Momentum

17 papers in the last four weeks, up 143% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 103

All topics

Progressive Approximation in Deep Residual Networks: Theory and Validation
Apr 27, 2026Wei Wang, Xiao-Yong Wei, Qing LiEfficient Neural Network Inference Residual Learning
Department of Computing, the Hong Kong Polytechnic University.
Can an MLP Absorb Its Own Skip Connection Exactly?
Apr 26, 2026Antonij Mijoski, Marko KarbevskiMultilayer Perceptrons Residual Learning
IRMA, Universit´e de Strasbourg, France · Independent Researcher
Dissipative Latent Residual Physics-Informed Neural Networks for Modeling and Identification of Electromechanical Systems
Apr 20, 2026Youyuan Long, Gokhan Solak, Arash AjoudaniDynamical Systems Residual Learning
Human-Robot Interfaces and Interaction Lab, Istituto Italiano di Tecnologia, 16163 Genoa, Italy
Rethinking Cross-Dose PET Denoising: Mitigating Averaging Effects via Residual Noise Learning
Apr 18, 2026Yichao Liu, Zongru Shao, Yueyang Teng +1Residual Learning PET/CT Imaging
IWR, Heidelberg University, 69120, Heidelberg, Germany · Silicon Austria Labs, 4040, Linz, Austria · College of Medicine and Biological Information Engineering, Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, 110169, Shenyang, China +1
Beyond Logit Adjustment: A Residual Decomposition Framework for Long-Tailed Reranking
Apr 2, 2026Zhanliang Wang, Hongzhuo Chen, Quan Minh Nguyen +2Long-Tailed Classification Learning to Rank
University of Pennsylvania, Philadelphia, PA, USA · Children’s Hospital of Philadelphia, Philadelphia, PA, USA
Learn Structure, Adapt on the Fly: Multi-Scale Residual Learning and Online Adaptation for Aerial Manipulators
Mar 12, 2026Samaksh Ujjawal, Naveen Sudheer Nair, Shivansh Pratap Singh +3Aerial Robotics Test-Time Adaptation
1Robotics Research Center, IIIT Hyderabad, India. · Department of Computer Science, University of Manchester, UK. · 3Newcastle University, UK.
ANCRe: Adaptive Neural Connection Reassignment for Efficient Depth Scaling
Feb 9, 2026Yilang Zhang, Bingcong Li, Niao He +1Residual Learning Neural Network Optimization
University of Minnesota · ETH Zurich
SA-ResGS: Self-Augmented Residual 3D Gaussian Splatting for Next Best View Selection
Jan 6, 2026Kim Jun-Seong, Tae-Hyun Oh, Eduardo Pérez-Pellitero +13D Gaussian Splatting Next-Best-View Planning
1POSTECH · 2KAIST · 3Huawei Noah’s Ark Lab +1
Variability Aware Recursive Neural Network (VARNN): A Residual-Memory Model for Capturing Temporal Deviation in Sequence Regression Modeling
Oct 10, 2025Haroon Gharwi, Yue Dai, Kai ShuMemory-Augmented Neural Networks Residual Learning
Department of Computer Science, Illinois Institute of Technology, Chicago, IL, USA · Department of Computer Science, Emory University, Atlanta, GA, USA
KITINet: KInetic Theory Inspired Inter-Channel Information Exchange for Residual Networks
May 23, 2025Mingquan Feng, Yifan Fu, Tongcheng Zhang +3Feature Interaction Modeling Residual Learning
Shanghai Jiao Tong University
Branch Scaling Manifests as Implicit Architectural Regularization for Improving Generalization in Overparameterized ResNets
Mar 7, 2024Zixiong Yu, Guhan Chen, Jianfa Lai +2Neural Network Generalization Residual Learning
Huawei Large Model Data Technology Lab, Shenzhen · Tsinghua University, Beijing · Kyoto University, Kyoto
Deep Residual Networks Learn the Geodesic Curve in the Wasserstein Space
Feb 18, 2021Kuo Gai, Shihua ZhangResidual Learning Neural Network Training Dynamics
Shanghai Institute for Mathematics and Interdisciplinary Sciences (SIMIS) Shanghai 200433, China · Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing 100190, China · School of Mathematical Sciences2026 University of Chinese Academy of Sciences Beijing 100049, China
On the Residual Scaling of Looped Transformers: Stability and Transferability
Date pendingShaowen Wang, Bingrui Li, Ge Zhang +3Residual Learning Hyperparameter Transfer
Tsinghua University · ByteDance Seed · M-A-P

Apr 27, 2026, cs.LGJ/K move · Enter open · S save

Progressive Approximation in Deep Residual Networks: Theory and Validation

Wei Wang, Xiao-Yong Wei, Qing Li

Department of Computing, the Hong Kong Polytechnic University.

Efficient Neural Network Inference Residual Learning Universal Approximation Neural Network Approximation Theory Early-Exit Neural Networks

Open paper AskPDF arXiv

The Universal Approximation Theorem (UAT) guarantees universal function approximation but does not explain how residual models distribute approximation across layers. We reframe residual networks as a layer-wise approximation process that builds an approximation trajectory from input to target, and prove the existence of progressive trajectories where error decreases monotonically with depth. It reveals that residual networks can implement structured, step-by-step refinement rather than end-to-end (E2E) black-box mapping. Building on this, we propose Layer-wise Progressive Approximation (LPA), a theoretically grounded training principle that explicitly aligns each layer with its residual target to realize such trajectories. LPA is architecture-agnostic: we observe progressive behavior in residual FNNs, ResNets, and Transformers across tasks including complex surface fitting, image classification, and NLP with LLMs for generation and classification. Crucially, this enables ``train once, use

N

models": a single network yields useful predictions at every depth, supporting efficient shallow inference without retraining. Our work unifies approximation theory with practical deep learning, providing a new lens on representation learning and a flexible framework for multi-depth deployment. The source code will be released unpon acceptance at https://(open_upon_acceptance).

Momentum

Related topics

Latest papers 103