Low-Data Regimes

Follow this topic Ask about this topic

Recent momentum

+0%

3 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

2 new papers

A weekly snapshot of new work published in Low-Data Regimes.

Period ending 2026-09-07

3 new papers

A weekly snapshot of new work published in Low-Data Regimes.

47 papers

Latest in Low-Data Regimes

Browse all topics →

An Exploratory Replica-Overlap Probe of the Grokking Transition
Sep 22, 2026A. C. Opus, J. Q. LuPhase Transitions Low-Data Regimes
High-Magnetization Sampling at Low Temperatures: Ising Models and Bayesian Sparse Linear Regression
Sep 8, 2026Syamantak Kumar, Purnamrita Sarkar, Kevin Tian +1Two-Dimensional Ising Model Markovian Sampling
Space Generative AI with Solar Energy Harvesting
Sep 1, 2026Jierui Zhang, Jianhao Huang, Zhanwei Wang +1Satellite Generative Artificial Intelligence
Recursive Criticality of AI Self-Improvement
Aug 31, 2026Mikhail BurtsevRecursive Self-Improvement Artificial Intelligence Research
TEMPO: Makespan-Aware Expert-Parallel Load Balancing Across Memory- and Compute-Bound Regimes
Aug 13, 2026Jie Li, Chenxin Jia, Jinliang Shen +5Load Balancing Multi-Gpu Systems
The Tamed Subgradient Unadjusted Langevin Algorithm beyond Convexity
Aug 6, 2026Iosif Lytras, Nikolaos Makras, Sotirios SabanisLangevin Dynamics Finite Time Analysis
On a joint simultaneous learning of relevant feature subsets and subspaces in regression-like problems
Jul 30, 2026Illia HorenkoChaos Low-Data Regimes
Variational meta-learning inference for low dimensional neural system identification
Jul 21, 2026Matteo Rufolo, Dario Piga, Marco ForgioneSystem Identification Variational Inference
Attractor Geometry Determines the Identifiability Limits of System Discovery
Jul 20, 2026Matteo Gallo, Fabio Anselmi, Paolo LazzariIdentifiability Symbolic Regression
Beyond the Edge of Chaos: Stability-Expressivity Transfer in Reservoir Forecasting
Jul 20, 2026Yao Du, Xingang WangReservoir Computing Chaos
Algebraic Representability as the Limiting Regime of Grokking: An Exactly Solvable Model with Holomorphic Activations
Jul 15, 2026Chon-Fai Kam, Xavier Cadet, Miloud Bessafi +1Grokking Two-Layer Neural Networks
Decentralized Gradient Descent: Bottleneck Regimes and Budget Complexity
Jul 13, 2026Nicolò MichelusiLow-Data Regimes Token Budgets
Grokking Is Conditional and Fragile: A Fully-Tractable, Multi-Seed Study at 12K Parameters
Jul 6, 2026Yoshiyuki OotaniGrokking Large Models
Broken Ergodicity and the Violation of the Fluctuation-Dissipation Theorem Lead to Generalization Beyond Overfitting in Machine Learning
Jul 5, 2026Chan Li, Nigel GoldenfeldMean-Field Limit Two-Layer Neural Networks
Is Variational Monte Carlo Robust? Sharp Moment Thresholds and Heavy-tailed Stochastic Optimization
Jun 24, 2026Philipp Grohs, Davide NobileVariational Monte Carlo
Topological Out-of-Domain Generalization in Dynamical Systems Reconstruction
Jun 22, 2026Georg Trede, Charlotte Ricarda Doll, Elias Weber +1Dynamical Systems Out-Of-Domain Generalization
The $(1 + 1)$ -EA in Dynamic Environments
Jun 11, 2026Georg Hasebe, Johannes Lengler, Raghu Raman RaviLow-Data Regimes Random
The Mathematics of AI Winters: The mathematical Taxonomy of Paradigm Fragility in AI Winter
Jun 10, 2026Miquel Noguer i Alonso, David Pacheco AznarArtificial Intelligence Research Artificial Intelligence Systems
Beyond Linear and Overcomplete Regimes: A Mean-Field Analysis of Bottleneck Autoencoders
Jun 5, 2026Santanu Das, Ramyak Bilas, Pascal Esser +1Masked Autoencoders Two-Layer Neural Networks
Data-Constrained Language Model Pretraining: Improved Regularization and Scaling Laws
Jun 5, 2026Zhiwei Xu, Shihao Wu, Hanseul Cho +2Large Language Model Pretraining Model Pretraining
A prism hierarchy of learning regimes in large linear autoencoders
Jun 3, 2026Eugene Golikov, Yaroslav Gusev, Dmitry YarotskyLow-Data Regimes Masked Autoencoders
Regime-Arrival Uncertainty in Generalization Bounds under Distribution Shift
Jun 1, 2026Prince PoudelDistribution Shifts Generalization Bounds
A Note on Stability for Orthogonalized Matrix Momentum with Client Sampling
Jun 1, 2026Da Chang, Qiankun Shi, Lvgang Zhang +2Finite-Sample Orthogonality
Fast Generalization after Interpolation via Critically Damped Momentum Optimization
Jun 1, 2026Luca Muscarnera, Silas Ruhrberg Estévez, Yuanzhang Xiao +1Gradient Descent Interpolation
Unveiling Multi-regime Patterns in SciML: Distinct Failure Modes and Regime-specific Optimization
May 27, 2026Yuxin Wang, Yuanzhe Hu, Xiaokun Zhong +7Low-Data Regimes Machine Learning
How Should LLMs Consume High-Quality Data? Optimal Data Scheduling via Quality-Aware Functional Scaling Laws
May 25, 2026Zhitao Zhu, Xili Wang, Shizhe Wu +2Large Models Large Language
LLMTabBench: Evaluating LLMs on Binary Tabular Classification From Zero to Few Shots
May 23, 2026Daria Grushina, Kseniia Kuvshinova, Alina Kostromina +3Tabular Learning Benchmark Datasets
Asymmetric Scaling Laws from Sparse Features
May 22, 2026John Sous, Michael WinerScaling Laws Over-Parameterized Neural Networks
Training-Free Fine-Grained Semantic Segmentations in Low Data Regimes: A FungiTastic Baseline
May 21, 2026Sebastian Cavada, Francesco Pelosin, Lapo FaggiFew-Shot Segmentation Dense Semantic Segmentation
LoRA vs. Full Fine-Tuning: A Theoretical Perspective
May 18, 2026Ali Zindari, Rotem Mulayoff, Sebastian U. StichLow-Rank Adaptation Finetuning
Feature Learning in Linear-Width Two-Layer Networks: Two vs. One Step of Gradient Descent
May 18, 2026Behrad Moniri, Hamed HassaniTwo-Layer Neural Networks Feature Learning
Genetic algorithm vs. gradient descent for training a neural network architecture dedicated to low data regimes in small medical datasets
May 13, 2026Amine Boukhari, Boglarka Ecsedi, Laszlo Papp +1Neural Architectures Gradient Descent
In-context learning to predict critical transitions in dynamical systems
May 12, 2026Yunus Sevinchan, Juan Nathaniel, Kai Ueltzhöffer +8Transitions Dynamical Systems
Quantum Transfer Learning Shows Improved Robustness in Low-Data Regimes
May 9, 2026Li-An Lo, Li-Yi Hsu, Hsien-Yi HsiehQuantum Machine Learning Transfer Learning
Learnability and Competition in High-Dimensional Multi-Component ICA
May 8, 2026Eser Ilke Genc, Samet Demir, Zafer DoganIndependent Component Analysis Low-Data Regimes
Central Limit Theorem for Two-Time-Scale Approximate Distributionally Robust RL
May 8, 2026Shengbo Wang, Zexi ZhangCentral Limit Theorem Model-Free Reinforcement Learning
Linear Response Estimators for Singular Statistical Models
May 8, 2026Chris Elliott, Daniel MurfetStatistical Models Estimators
Learning Discriminators for Resampling in the Ensemble Gaussian Mixture Filter through a Normalizing Flow Approach
May 1, 2026Zain Jabbar, Andrey A. PopovParticle Filters Normalizing Flows
Early Detection of Latent Microstructure Regimes in Limit Order Books
Apr 22, 2026Prakul Sunil Hiremath, Vruksha Arun HiremathLimit Order Book Stochastic Volatility
Late Fusion Neural Operators for Extrapolation Across Parameter Space in Partial Differential Equations
Apr 17, 2026Eva van Tegelen, Taniya Kapoor, George A. K. van Voorn +2Neural Operators Fourier Neural Operators
Can Generative Artificial Intelligence Survive Data Contamination? Theoretical Guarantees under Contaminated Recursive Training
Feb 17, 2026Kevin Wang, Hongqian Niu, Didong LiGenerative Artificial Intelligence Model Training
Agile Reinforcement Learning through Separable Neural Architecture and Applications
Jan 30, 2026Rajib Mostakim, Reza T. Batley, Sourav SahaDeep Reinforcement Learning Low-Data Regimes
Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Models
Jun 16, 2025Zhenyu Liao, Michael W. MahoneyHigh-Dimensional Data Largest Eigenvalue
Temporal Variational Implicit Neural Representations
Jun 2, 2025Batuhan Koyuncu, Rachael DeVries, Ole Winther +1Multivariate Time Series Forecasting Time Series
Universality of Benign Overfitting in Binary Linear Classification
Jan 17, 2025Ichiro Hashimoto, Stanislav Volgushev, Piotr ZwiernikOverfitting Binary Classification
Adversarial dynamical systems characterize when data-driven learning succeeds or fails
Jul 8, 2024Matthew J. Colbrook, Igor Mezić, Alexei StepanenkoKoopman Operator Low-Data Regimes
A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning
Sep 6, 2021Yehuda Dar, Vidya Muthukumar, Richard G. BaraniukOver-Parameterized Neural Networks Machine Learning

Sep 22, 2026 · cs.LGJ/K move · Enter open · S save

An Exploratory Replica-Overlap Probe of the Grokking Transition

A. C. Opus, J. Q. Lu

We trained 64 independently seeded networks in four configurations, continuing each to sustained convergence or a 40,000-epoch ceiling. We then asked whether an RSB-inspired distribution of pairwise weight overlaps changes across the grokking transition. It is the alignment step, not the overlap statistic, that determines what this registered probe can report. The registered implementation permutes hidden units without the corresponding bias and head-internal permutations and therefore does not preserve the network function. Every q_wt value computed through this alignment inherits the defect; q_fn does not, because it is computed from predictions of the unpermuted models. The numerical-precision requirement also failed, and an audit found protocol deviations. Consequently, the pre-registered rule gives no verdict: registered outcome UNDETERMINED (reason code C0_INSTRUMENT_INVALID). These data provide neither a confirmatory null nor a validated reading of the Parisi order parameter. Only frac40 cleared the 12/16 checkpoint-completeness requirement. For this configuration, a post-hoc criterion applied to the same data gave a Hartigan-dip interval containing zero (95% CI for Delta dip = [-0.017, 0.034]), whereas the overlap standard deviation increased by a factor of about 5.6. A post-hoc calibration assigns the dip test zero power at the simulated separations; the interval is therefore uninformative, not evidence of no change. The standard-deviation ratio is the only statistic here with power at the observed effect. Ensemble loss was near-flat only under the pre-specified 1% threshold. Finally, grokking rates of 0/16, 11/16 and 16/16 remain descriptive because train fraction is confounded with split identity.

Phase Transitions Low-Data Regimes Transitions

Open paper Ask about it PDF