Neural Scaling Laws

Momentum

9 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 53

All topics
CardsList
  1. A Solvable Model of Adaptive Learning Rate Rescaling: Acceleration, Stability & Scaling

    Oct 5, 2026Itay Lavie, Clarissa Lauditi, Cengiz PehlevanEdge of StabilityNeural Scaling Laws

  2. A Unified Scaling Law for Time Series Foundation Models

    Oct 4, 2026Xilin Dai, Yiding Liu, Zewei Dong +2Time Series ForecastingNeural Scaling Laws

  3. Overcoming Kernel Redundancy for Scaling Logic Gate Networks

    Oct 1, 2026Sejin Park, Hongjae Lee, Changwoo Han +1Neural Scaling LawsConditional Computation

  4. Neural scaling laws and evolution of learnable activation functions of Kolmogorov-Arnold networks

    Oct 1, 2026Tilen Cadez, Sanghoon Lee, Kyoung-Min KimNeural Network Activation FunctionsKolmogorov-Arnold Networks

  5. Effective Does Not Mean Useful: Conditional Functional Substitutability for Redundancy and Scaling in Transformers

    Sep 30, 2026Jiaheng Chen, Jiaxing Li, Yucheng Xiao +4Neural Scaling Laws

  6. Neural Scaling Laws of Transformer Operator Network

    Sep 27, 2026Haoran Yan, Zhongjie Shi, Yuanzhe Xi +2Neural Network GeneralizationNeural Network Approximation Theory

  7. The Capability Manifold and ML Scaling Laws

    Sep 23, 2026Syed Ali Raza Zaidi, Maryam HafeezScaling LawsNeural Scaling Laws

  8. A Scaling Study for fMRI Foundation Models

    Sep 23, 2026Wenhao Ye, Xuanye Pan, Junfeng Xia +3Medical Imaging Foundation ModelsfMRI Foundation Models

  9. Symmetry without a manifold: intrinsic dimension on orbits

    Sep 15, 2026Chon-Fai Kam, Miloud Bessafi, Frédéric CadetIntrinsic DimensionalityNeural Scaling Laws

  10. Coupled Scaling: A Representational Accessibility Framework for Neural Scaling Laws

    Sep 3, 2026Jie WangNeural Representation GeometryNeural Scaling Laws

  11. Neural Quadratic Forms: A Unified Minimal Model for Sudden Learning and Scaling Laws

    Aug 13, 2026Liu Ziyin, Yizhou Xu, Tomaso Poggio +1Dynamical SystemsNeural Network Training Dynamics

  12. Skaling: Chinchilla's Exponents Meet Kaplan's Coupling

    Aug 7, 2026Mathurin Videau, Badr Youbi-Idrissi, David Lopez-Paz +1Language Model Scaling LawsNeural Scaling Laws

  13. Mechanisms of Width Scaling in Normalized Residual Networks: The Effective Alignment Dimension

    Jul 27, 2026Jinhao Zhang, Zeyu Liu, Zicheng Yan +4Neural Network GeneralizationResidual Learning

  14. Hyperparameter Transfer in Graph Neural Networks

    Jul 6, 2026Gage DeZoort, Boris HaninGraph Neural NetworksNeural Network Optimization

  15. How to Allocate Your Tokens? Scaling Laws with Training Steps and Batch Size

    Jul 1, 2026Fabian SchaippLanguage Model Scaling LawsScaling Laws

  16. A Stochastic--Geometric Theory of Scaling Laws in Grokking

    Jun 29, 2026Róisín Luo, Christian Gagné, Jonas Ngnawé +2Neural Network GeneralizationNeural Network Optimization

  17. How Width and Data Shape Generalization Scaling Laws in Quadratic Neural Networks

    Jun 26, 2026Julius Girardin, Emanuele Troiani, Yizhou Xu +3Neural Network GeneralizationScaling Laws

  18. Neural Scaling Universality: If Exponents Are Fixed, Time to Understand Coefficients

    Jun 23, 2026Yizhou Liu, Jeff GoreLanguage Model Scaling LawsNeural Scaling Laws

  19. Statistical Properties of Training & Generalization

    Jun 18, 2026Itay Lavie, Noam Levi, Yonatan KahnNeural Network GeneralizationScientific ML

  20. Towards Engineering Scaling Laws with Pretraining Data Composition

    Jun 18, 2026Jan-Lucas Uslu, Kevin Greif, Daniel Whiteson +1Synthetic Data PretrainingHigh-Energy Physics

  21. Explaining Data Mixing Scaling Laws

    Jun 6, 2026Rui Dai, Shuran ZhengData Mixture OptimizationNeural Scaling Laws

  22. Compute-Optimal Network Design for Echocardiography Myocardial Segmentation and Perfusion Quantification using Neural Scaling Laws

    Jun 4, 2026Clara Rodrigo González, Matthieu Toulemonde, Lasha Gvinianidze +5Ultrasound Image SegmentationMedical Image Analysis

  23. Neuron Populations Exhibit Divergent Selectivity with Scale

    Jun 2, 2026Amil Dravid, Yasaman Bahri, Alexei A. Efros +1Neural Network InterpretabilityNeural Encoding

  24. Structure and Scale in Simplicial Sequence Modelling

    May 31, 2026Matthew Farrugia-RobertsRepresentation GeometryTransformer

  25. Spectral Reach: Understanding Neural Scaling as Progress into the Spectral Tail

    May 29, 2026Konstantin Nikolaou, Jonas Scheunemann, Sven Krippendorf +2Representation LearningNeural Network Training Dynamics

  26. How Much Is a Dataset Worth? Scaling Laws, the Vendi Score, and Matrix Spectral Functions

    May 28, 2026Jeff A. Bilmes, Gantavya Bhatt, Arnav M. DasSubmodular OptimizationNeural Scaling Laws

  27. On the Optimizer Dependence of Neural Scaling Laws

    May 28, 2026Vansh Ramani, Shourya Vir JainNatural Gradient MethodsNeural Network Optimization

  28. Neural Scaling Laws for Jet Generation

    May 27, 2026Oz Amram, Darius A. Faroughy, Tjarko Gerdes +5High-Energy PhysicsAutoregressive Generation

  29. Pretrained Approximators for Low-Thrust Trajectory Cost and Reachability

    May 26, 2026Zhong Zhang, Giacomo Acciarini, Dario Izzo +2Neural Surrogate ModelingTrajectory Optimization

  30. Unified Neural Scaling Laws

    May 25, 2026Ethan Caballero, Priyank Jaini, David Krueger +1Language Model Scaling LawsScaling Laws

  31. Asymmetric Scaling Laws from Sparse Features

    May 22, 2026John Sous, Michael WinerDouble DescentActivation Sparsity

  32. A Boundary-Layer Mechanism for One-Third Scaling in Online Softmax Classification

    May 21, 2026Marcel Kühn, Yoon Thelge, Bernd RosenowStatistical Physics of LearningMulticlass Classification

  33. How Do Electrocardiogram Models Scale?

    May 17, 2026Jiawei Li, Fabio Bonassi, Ming Jin +4Self-Supervised Pre-TrainingElectrocardiography

  34. Scaling Laws and Tradeoffs in Recurrent Networks of Expressive Neurons

    May 12, 2026Aaron Spieler, Georg Martius, Anna LevinaRecurrent Neural NetworksNeural Scaling Laws

  35. How Much is Brain Data Worth for Machine Learning?

    May 10, 2026Lane Lewis, Zhixin Wang, David Schwab +1Neural Scaling LawsData-Efficient Learning

  36. Tokens-per-Parameter Coverage Is Critical for Robust LLM Scaling Law Extrapolation

    May 8, 2026Joshua Shay Kricheli, Alexander Lawrence Reid, Soumajyoti Sarkar +2Parameter EstimationLanguage Model Scaling Laws

  37. On the Invariance and Generality of Neural Scaling Laws

    May 8, 2026Xing Han, Liu Ziyin, Suchi Saria +1Language Model Scaling LawsCross-Domain Generalization

  38. How Big Should a Wireless Foundation Model Be?

    May 8, 2026Wei-Lun Cheng, Wanjiun LiaoWireless Foundation ModelsWireless Communications

  39. Deep Wave Network for Modeling Multi-Scale Physical Dynamics

    May 5, 2026Alexander I. Khrabry, Edward A. Startsev, Andrew T. Powis +1U-NetMulti-Scale Feature Fusion

  40. Prescriptive Scaling Laws for Data Constrained Training

    May 2, 2026Justin Lovelace, Christian Belardi, Srivatsa Kundurthy +2Weight DecayLanguage Model Scaling Laws

  41. A Limit Theory of Foundation Models: A Mathematical Approach to Understanding Emergent Intelligence and Scaling Laws

    Apr 27, 2026Jun Shu, Junxiong Jia, Deyu Meng +1Neural Scaling LawsAI Scaling Laws

  42. OmniMouse: Scaling properties of multi-modal, multi-task Brain Models on 150B Neural Tokens

    Apr 20, 2026Konstantin F. Willeke, Polina Turishcheva, Alex Gilbert +18Neural DecodingVisual Neuroscience

  43. How Much Data is Enough? The Zeta Law of Discoverability in Biomedical Data, featuring the enigmatic Riemann zeta function

    Apr 19, 2026Paul M. ThompsonNeural Scaling LawsData-Efficient Learning

  44. μμpscaling small models: Principled warm starts and hyperparameter transfer

    Feb 11, 2026Yuxin Ma, Nan Chen, Mateo Díaz +3Neural Network OptimizationHyperparameter Transfer

  45. Deriving Neural Scaling Laws from the statistics of natural language

    Feb 7, 2026Francesco Cagnetta, Allan Raventós, Surya Ganguli +1Language Model Scaling LawsNeural Scaling Laws

  46. From Zipf's Law to Neural Scaling through Heaps' Law and Hilberg's Hypothesis

    Dec 15, 2025Łukasz DębowskiNeural Scaling Laws

  47. Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Models

    Jun 16, 2025Zhenyu Liao, Michael W. MahoneyRandom Matrix TheoryNeural Network Generalization