Neural Network Initialization

Latest papers 28

All topics
CardsList
  1. Ideal Paths for Approximating Logistic Gradient Descent Trajectories at Large Initialization

    Oct 2, 2026Junjie Xiao, Huiwen JiaLogistic RegressionNeural Network Initialization

  2. Dimension-Free Rank Lifting from Random Hyperplane Arrangements

    Sep 30, 2026Luca Becchetti, Matteo Russo, Ruben SkorupinskiNeural Network Initialization

  3. Procedural Core: A Compact Recurrent Initialization for Vision Transformers

    Sep 29, 2026Zachary Shinnick, Christian Internò, Hemanth Saratchandran +2Neural Network InitializationVision Transformer

  4. Copy the Same, Distill the Difference: Initializing Linear Vision Transformers

    Sep 28, 2026Huaiyuan Qin, Muli Yang, Gabriel James Goenawan +9Neural Network InitializationVision Transformer

  5. Structural priors for data-efficient language learning

    Sep 12, 2026Yana Veitsman, Jonas Mayer Martins, Jonathan Lautenschlager +1Language Model PretrainingNeural Network Initialization

  6. Critical initialization destabilizes higher input derivatives in wide scalar-input networks

    Sep 10, 2026Prashant Singh, Pranav SinghNeural Network Initialization

  7. Unveiling the Secret of AdaLN-Zero in Diffusion Transformer

    Aug 10, 2026Jie Zhu, Mingyu Ding, Boqiang Duan +2Diffusion TransformerNeural Network Initialization

  8. Correlation flow governs learning at criticality

    Aug 8, 2026Andrea Combette, Nelly Pustelnik, Antoine VenailleNeural Network InitializationNeural Network Training Dynamics

  9. LLM-Guided Initialization for Accelerated Hybrid Quantum-Classical Medical Image Classification

    Jul 29, 2026Riza Alaudin Syah, Irwan Alnarus Kautsar, Haza Nuzly Bin Abdull HamedNeural Network InitializationHybrid Quantum-Classical ML

  10. Calibrated Partial Resets: Preventing Policy Collapse in Continual Reinforcement Learning

    Jul 27, 2026Luc McCutcheon, Evangelos Chatzaroulas, Saber FallahNon-Stationary RLNeural Network Initialization

  11. Small Initialization Matters for Large Language Models

    Jun 16, 2026Liangkai Hang, Junjie Yao, Zhiyu Li +3Language Model PretrainingNeural Network Initialization

  12. Taylor-Calibrate: Principled Initialization for Hybrid Linear Attention Distillation

    Jun 15, 2026Zhongzhu Zhou, Qingyang Wu, Junxiong Wang +4Neural Network InitializationLinear Attention

  13. Robust Neural Tucker Factorization with Bias Correction and Adaptive Initialization

    Jun 15, 2026Yuchao Su, Yixin RanTensor CompletionNeural Network Initialization

  14. Jacobi-Anger Method for Deterministic Initialization in Implicit Neural Representation

    Jun 4, 2026Mohammed Alsakabi, Kejia Hu, John M. Dolan +1Implicit Neural RepresentationsNeural Network Initialization

  15. Do Deep Networks Forget Initialization? A Forgetting-Time View of Practical Inductive Bias

    May 27, 2026Mohua Das, Pierfrancesco Beneventano, Shibshankar Dey +2Neural Network GeneralizationNeural Network Initialization

  16. CAWI: Copula-Aligned Weight Initialization for Randomized Neural Networks

    May 12, 2026Mushir Akhtar, M. Tanveer, Mohd. ArshadNeural Network Initialization

  17. A Random-Matrix Criterion for Initializing Gated Recurrent Neural Networks

    May 11, 2026Tommaso Fioratti, Riccardo Marcaccioli, Francesco CasolaNeural Network InitializationRecurrent Neural Networks

  18. Criticality and Saturation in Orthogonal Neural Networks

    May 7, 2026Max Guillen, Jan E. GerkenNeural Network Initialization

  19. How Long Does Infinite Width Last? Signal Propagation in Long-Range Linear Recurrences

    May 6, 2026Mariia SeleznovaNeural Network InitializationRecurrent Neural Networks

  20. A Theory of How Pretraining Shapes Inductive Bias in Fine-Tuning

    Feb 23, 2026Nicolas Anguita, Francesco Locatello, Andrew M. Saxe +4Neural Network GeneralizationRepresentation Learning

  21. Favorability of Loss Landscape with Weight Decay Requires Both Large Overparametrization and Initialization

    May 28, 2025Etienne Boursier, Matthew Bowditch, Matthias Englert +1Weight DecayNeural Network Initialization