Residual Learning

Ask about this topic

Overview

Residual learning studies how models can build on an existing representation or prediction by adding learned updates, rather than learning the entire transformation from scratch. In deep networks, skip connections and newer approaches such as attention-based routing and multi-stream hyper-connections shape information and gradients across layers; in hybrid models, learned corrections refine physics-based, statistical, or frozen-model predictions. These methods are evaluated in settings including image classification, language modeling, diffusion, time-series forecasting, scientific simulation, and control. A central challenge is determining when residual pathways improve depth, stability, and generalization—and when their complexity, added compute, or poorly transferable corrections instead hurt performance.

Latest papers

CardsList

158 papers32 papers in the last four weeks, up 23% on the four weeks before. 0.2% of all new papers.

  1. Generative Refinement Learning for Temporal Interpolation in Scientific Machine Learning

    Oct 27, 2026Pu Ren, Michael W. Mahoney, N. Benjamin ErichsonScientific MLResidual Learning

    Lawrence Berkeley National Lab · International Computer Science Institute · University of California, Berkeley

  2. Kronecker-Structured Video Completion with Cross-Fitted Self-Supervised Residual Learning

    Oct 26, 2026Qi Qi, Yanfei Wang, Yanjun Cui +1Tensor DecompositionResidual Learning

    Mathematical Research Center, Shandong University, Jinan 250100, China · School of Mathematics, Shandong University, Jinan 250100, China · Dartmouth College, Hanover, NH 03755, USA +1

  3. Guarded Agent-in-the-Loop Residual Adaptation for Evolving Wind Power Regimes

    Oct 26, 2026Yuhang Dong, Min Xia, Jinghong Li +3Wind Power ForecastingNon-Stationary Time Series Forecasting

    Nanjing University of Information Science and Technology, Nanjing, China

  4. Sliding Networks: A Family of Residual Structures Preserving Feature Learning and Feature Diversity

    Oct 24, 2026Datong Zhou, Blake BordelonNeural Scaling LawsResidual Learning

    Laboratoire Jacques-Louis Lions (LJLL) & Laboratoire de Probabilités, Statistique et Modélisation (LPSM) · Oden Institute and Department of Neuroscience University of Texas at Austin

  5. Polynomial Residual Networks are Generically Identifiable

    Oct 24, 2026Isabel Dahlgren, Giovanni Luca Marchetti, Kathlén KohnParameter IdentifiabilityPolynomial Neural Networks

  6. Early learning in large residual networks

    Oct 22, 2026Vladimir Narovlansky, Boris HaninNeural Network Training DynamicsResidual Learning

    ORFE Department, Princeton University, Princeton, NJ 08544, USA · Racah Institute of Physics, The Hebrew University of Jerusalem, Jerusalem 91904, Israel

  7. Width and Depth Limits Commute During Training in Linear Residual Networks

    Oct 22, 2026Ruijia Zhang, Soufiane HayouNeural Network Training DynamicsDeep Linear Networks

    Department of Applied Mathematics and Statistics, Johns Hopkins University

  8. Local and Global Recursive Ridge Regression for Online Residual Correction of Frozen Multivariate Forecasters under Label Delay

    Oct 21, 2026Takumi Fujimoto, Hiroaki NishiMultivariate Time Series ForecastingOnline Time Series Forecasting

    School of Science for Open and Environmental Systems, Graduate School of Science and Technology, Keio University, Japan · Department of System Design, Faculty of Science and Technology, Keio University, Japan

  9. AGGR-Mamba: Mamba-based Network with Attention-Guided Gated Residual Refinement for Undersampled MRI Reconstruction

    Oct 20, 2026Yue Zhuo, Wenbin Deng, Lingjun Liu +2Accelerated MRI ReconstructionResidual Learning

    School of Mathematics and Statistics, Huizhou University, Huizhou, China · School of Computer Science and Engineering, Huizhou University, Huizhou, China

  10. Physics-anchored residual quantum neural networks for cloud-cover parameterization

    Oct 19, 2026Zhuo Wang, Ce Yu, Bin YangQuantum Neural NetworksQuantum Machine Learning

    Tianjin University

  11. Paloma: Phase-Conditioned Residual Modulation for Time Series Forecasting

    Oct 18, 2026Jingru Fei, Wei Fan, Kun Yi +2Time Series ForecastingResidual Learning

    Beijing Institute of Technology · University of Auckland · State Information Center +1

  12. Synergy-Aware Residual Flow Matching for Single-Cell Perturbation Prediction

    Oct 18, 2026Jingrui Wu, Wangyun DanSingle-Cell Perturbation ModelingFlow-Based Generative Modeling

    Department of Statistics, Rice University, Houston, TX, USA · Department of Biostatistics and Bioinformatics, Emory University, Atlanta, GA, USA

  13. Enhancing analytical photon transport in turbid media through physics-guided neural network corrections

    Oct 17, 2026Guojun Tang, Paul D. Teal, Alexander DoroninRadiative TransferNeural Surrogate Modeling

    Victoria University of Wellington, School of Engineering and Computer Science, Wellington, New Zealand

  14. On-Policy Residual Learning for Diffusion Language Models

    Oct 14, 2026Yifan Yu, Yuqing Jian, Jisen Li +9Diffusion Language ModelsDiffusion Language Model Inference

    Together AI · University of Illinois Urbana-Champaign · University of California, Berkeley +3

  15. Two-stage Residual Learning for Modelling Strength-Informed Optimisation of Thermal Systems

    Oct 14, 2026Talha Ansar, Waqar Muhammad Ashraf, Abdulelah S. Alshehri +1Surrogate ModelingSurrogate-Assisted Optimization

    Department of Mechanical Engineering, University of Engineering and Technology Lahore, Kala Shah Kaku, Pakistan · The Sargent Centre for Process Systems Engineering, Department of Chemical Engineering, University College London, London, UK · Department of Chemical Engineering, College of Engineering, King Saud University, Riyadh, Saudi Arabia +3

  16. Historical-Feature Residual Adaptation for Sequential Screening

    Oct 13, 2026Yixiang Hong, Yuchen ZhangSequential Decision MakingLearning to Rank

    The University of Texas Health Science Center at Houston Houston, Texas, USA

  17. Cepheid: Decoupling Scale from Structure for Cross-Family Motion Completion

    Oct 13, 2026Chih-Hai Su, Tsung-Wei Ke, Yu-Lun LiuResidual LearningMotion Representation Learning

    National Yang Ming Chiao Tung University · Department of Computer Science and Information Engineering, National Taiwan University

  18. Multi-Stream Residual Computation in Diffusion Language Models

    Oct 13, 2026Ekaterina Alimaskina, Suren Kazarian, Gleb Molodtsov +1Diffusion Language ModelsMasked Diffusion Models

    BRAIn Lab, Yandex Research

  19. On the Depth Transfer of Neural Optimizers via L-Smoothness

    Oct 9, 2026Maximilian Wong, Chenyu Xu, Jiajin Li +1Deep Learning OptimizationHyperparameter Transfer

    University of Chicago · Peking University · Sauder School of Business University of British Columbia +1

  20. Multi-Stream Attention Residuals

    Oct 9, 2026Xiaohan Qin, Xiangdong Zhang, Tuo Dai +7Attention Residual ConnectionsResidual Learning

    School of AI, Shanghai Jiao Tong University · Shanghai Innovation Institute · Dots Studio, Xiaohongshu Inc. +2

  21. LAIR-Net: Leaky Alignment-Impulse Residual Networks for Tabular Regression

    Oct 8, 2026Rahul Goswami, Aryan Bhambu, Bittu KarmakarTabular RegressionResidual Learning

    Indian Institute of Technology Guwahati · SAFIR, Sorbonne University Abu Dhabi · Indian Institute of Technology Bombay

  22. Conditional Residual Prediction: Improving Autoregressive Video Diffusion without a Bidirectional Teacher

    Oct 8, 2026Bowen Zheng, Zhiguang Liu, Jiarong Ou +2Causal Video GenerationVideo Diffusion Models

    The Chinese University of Hong Kong, Shenzhen · Tencent Hunyuan

  23. RISR: Residual-Informed Scientific Equation Discovery with Large Language Models

    Oct 8, 2026Haobo Li, Wenshuo Zhang, Wenxiao Zhao +6Symbolic RegressionLarge Language Model-Guided Scientific Discovery

    Shanghai AI Laboratory · The Hong Kong University of Science and Technology · University of California, Los Angeles

  24. CARE: A Lightweight Plug-in Gated Correction and Uncertainty-aware Module for Long-term Time Series Forecasting

    Oct 8, 2026Guo Cheng, Changlong Lv, Jingyi HouMultivariate Time Series ForecastingLong-Term Time Series Forecasting

    School of Artificial Intelligence, University of Science and Technology Beijing, Beijing 100083, China · Institute of Artificial Intelligence, University of Science and Technology Beijing, Beijing 100083, China · Key Laboratory of Intelligent Bionic Unmanned Systems, Ministry of Education, University of Science and Technology Beijing, Beijing 100083, China

  25. Temporal Residual Bottleneck for Robust Asynchronous Collaborative Perception

    Oct 7, 2026Melih Yazgan, Ahmed Abouelazm, J. Marius ZöllnerCooperative PerceptionAutonomous Driving Perception

    FZI Research Center for Information Technology · Karlsruhe Institute of Technology

  26. Residual Learning in Empirical Asset Pricing

    Oct 7, 2026Dexin Peng, Xiaoyu WangQuantitative FinanceResidual Learning

    FinTech Thrust, Hong Kong University of Science and Technology (Guangzhou)

  27. Should We Skip Diffusion?

    Oct 4, 2026Yiping Ji, James Martens, Simon LuceyDiffusion TransformerDiffusion-Based Image Generation

    Australian Institute for Machine Learning, Adelaide University · Independent Researcher

  28. The Neuro-Physical Inverter: A Modular Framework for Magnetotelluric Inversion Coupling Ensemble Conditioning with Residual Learning

    Oct 2, 2026Jae Deok Kim, Sai Ravela, Rob. L. EvansInverse ProblemsUncertainty Quantification

    MIT-WHOI Joint Program in Oceanography/Applied Ocean Science and Engineering, Cambridge, MA 02139 USA · Department of Earth, Atmospheric and Planetary Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139 USA · Department of Geology and Geophysics, Woods Hole Oceanographic Institution, Woods Hole, MA 02543 USA

  29. Signal Simplification Is Not Predictive Simplification: Diagnosing Residual Neural Forecasting in Short-Horizon Volatility

    Oct 2, 2026Bingqi Lian, Linfeng Cheng, Mei Lu +1Time Series ForecastingResidual Learning

    Science Academy, College of Computer, Mathematical, and Natural Sciences, University of Maryland, College Park, MD 20742, USA · Department of Electrical and Computer Engineering, University of Maryland, College Park, MD 20742, USA

  30. Do ResNets Route? Sparse Interaction Experts in Residual Networks

    Oct 2, 2026Liang Yan, Siying Chen, Kaijie Chen +3Residual LearningSparse Neural Networks

    Fudan University · University of Washington · Tongji University +2