Neural Network Training Dynamics

Latest papers 349

All topics
CardsList
  1. Algorithmic Information Dynamics of Learning: A Certified, Differentiable Complexity Controller for Grokking

    Aug 14, 2026Luan Ozelim, Abicumaran Uthamacumaran, Hector ZenilNeural Network Training DynamicsGrokking

  2. Neural Quadratic Forms: A Unified Minimal Model for Sudden Learning and Scaling Laws

    Aug 13, 2026Liu Ziyin, Yizhou Xu, Tomaso Poggio +1Dynamical SystemsNeural Network Training Dynamics

  3. TESLA: Taylor Expansion of Sinusoidal Learnable Activations

    Aug 12, 2026Daehwa Ko, Jaehyeon Kim, Seunghyun Ham +1Neural Network GeneralizationNeural Network Activation Functions

  4. Correlation flow governs learning at criticality

    Aug 8, 2026Andrea Combette, Nelly Pustelnik, Antoine VenailleNeural Network InitializationNeural Network Training Dynamics

  5. Tools to Explain Neural Networks for Power System Dynamics

    Aug 8, 2026Petros Ellinas, Johanna Vorwerk, Spyros ChatzivasileiadisNeural Surrogate ModelingNeural Network Interpretability

  6. Learning in Deep Networks under Dale's Constraint

    Aug 7, 2026Roy Abel, Shimon UllmanBackpropagationBiologically Plausible Learning

  7. Hidden Gauge Controls Feature Specialization in ReLU Networks

    Aug 7, 2026Tongxi WangReLU Neural NetworksNeural Network Training Dynamics

  8. Cascading Through the Hierarchy: Regularizer-Induced Feature Detection as Phase Transitions in Deep Linear Neural Networks

    Aug 6, 2026Björn Ladewig, Ibrahim Talha Ersoy, Karoline WiesnerStatistical Physics of LearningRepresentation Learning

  9. Optimal Training-Time Scaling in Gradual Adaptation

    Aug 5, 2026Zonghuan Xu, Krishna HarishDomain AdaptationScaling Laws

  10. Robustness Emerges Early in Training Dynamics, but Is Not Preserved

    Aug 5, 2026Jiangang Yang, Wenhui Shi, Lu Hu +2Distribution Shift RobustnessRepresentation Learning

  11. Divisive Normalization Shapes Low-Rank Slow Manifolds for Continuous Working Memory

    Aug 3, 2026Zhaotian Gu, Jie Su, Weiwei Wang +3Recurrent Neural NetworksNeural Network Training Dynamics

  12. Tunneling the Loss Landscape: Bypassing Memorization with Monte Carlo Parameter Swapping

    Aug 3, 2026Lai Shun Chan, Xiaotian Zhang, Yue Shang +2Neural Network GeneralizationMarkov Chain Monte Carlo

  13. Simplifying Neural Networks During Training

    Jul 30, 2026Lorenzo Sciandra, Samuele Fonio, Roberto EspositoNeural Network Training DynamicsNeural Network Compression

  14. Towards joint scaling laws with optimal batch size schedules

    Jul 30, 2026Jiaxiang Li, Zhiqi Bu, Shiyun XuDeep Learning OptimizationLanguage Model Scaling Laws

  15. Semantic Space Search Trajectory Networks

    Jul 27, 2026Julian Agudelo, Alberto Tonda, Gabriela Ochoa +3Neural Network Training Dynamics

  16. A Statistical Difference between Single-Layer Learning and Hierarchical Learning in Wide Neural Networks

    Jul 26, 2026Sumio WatanabeNeural Network GeneralizationHierarchical Representation Learning

  17. Mini-batch Noise Lowers Sharpness via Dominant-Subspace Fluctuations

    Jul 25, 2026Junho So, Dongwook ShinNeural Network Training DynamicsGradient Descent Dynamics

  18. Hyperball May Not Be a Free Lunch

    Jul 24, 2026Yihao Xiao, Jialong Sun, Zitian Gao +5Learning Rate SchedulingNeural Network Optimization

  19. A Defense of the Quadratic Model

    Jul 23, 2026Alexandru Meterez, Pranav Ajit Nair, Depen Morwani +3Language Model PretrainingEdge of Stability

  20. Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay

    Jul 23, 2026Xiaolong Li, Zhangchen Zhou, Zhi-Qin John XuWeight DecayNeural Network Optimization

  21. Conditioned Direct Feedback Alignment via Activity and Error Geometry

    Jul 20, 2026Houman Safaai, Varun Reddy, Bernardo L. SabatiniMultilayer PerceptronsNeural Network Training Dynamics

  22. Phasor Attention: Mean Root Square Normalization for Phase Manifold Preservation

    Jul 20, 2026Sungwoo Goo, Hwi-yeol Yun, Sangkeun JungGradient ClippingLayer Normalization

  23. Dropout and Random Gradient Masking Are Asymptotically Equivalent in Large ResNets

    Jul 18, 2026Javier Maass, Lénaïc ChizatNeural Network Training Dynamics