Overfitting

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 48

All topics

Dataset Identity, Not Novelty: The Source of an Inflated OOD Detection Gain
Oct 1, 2026Donghoon Lee, Shinjin KangOut-Of-Distribution Detection Out-Of-Distribution
Hongik University
Misalignment of Low-Loss Regions Causes Grokking
Sep 30, 2026Yongding Tian, Zaid Al-Ars, Maksim Kitsak +1Grokking Overfitting
Computer Engineering Lab Delft University of Technology Delft, NL · HDL TypeTech Delft, NL · Network and Architecture Service Delft University of Technology Delft, NL +1
Forking: Sudden Overfitting Under Replay
Sep 30, 2026Shanbin Yu, Shaoyang Guo, Haoran Zhao +2Overfitting Catastrophic Forgetting
MetaCircle (元环智能) · Tsinghua University · Peking University +1
Benign Overfitting for General Norms and Distributions
Sep 27, 2026Daniel Barzilai, Ohad ShamirOverfitting Spectral Norm
Weizmann Institute of Science · University of Toronto & Vector Institute
Double Descent and Malign Overfitting in Diffusion Models
Sep 22, 2026Raphaël Urfin, Tony Bonnaire, Giulio Biroli +1Overfitting Overparameterization
Laboratoire de Physique de l’École normale supérieure, ENS, Université PSL, CNRS, Sorbonne Université, Université Paris Cité, F-75005 Paris, France · Université Paris-Saclay, CNRS, Institut d’Astrophysique Spatiale, 91405 Orsay, France · Department of Computing Sciences, Bocconi University, Milano, Italy
MaskVLA: Visual Masking Against Trajectory Overfitting of Vision-Language-Action Model
Sep 20, 2026Yuxuan Jiang, Jiaying Huang, Ge Wang +9Diffusion-Based Vision-Language-Actions Overfitting
FNii-Shenzhen, The Chinese University of Hong Kong, Shenzhen, China · School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, China · Ising AI +2
Structured Features Overfit Where Random Features Grok
Sep 14, 2026Chon-Fai Kam, Miloud Bessafi, Frederic CadetOverparameterization Overfitting
University Paris City & University of Reunion, Paris, France; Dipartimento di Fisica e Chimica · Emilio Segr`e, Universit`a degli Studi di Palermo, Italy · EnergyLab, University of Reunion, Saint-Denis, France +2
SynthSentry: Detecting Synthetic Data Contamination in Language Model Training Data
Sep 14, 2026Praveen Kumar Myakala, Ravichandra Namburi, Sowmya Keragodu Jayaramu +1Synthetic Data Training Data
Independent AI Researcher Melissa, USA · Independent Researcher Frisco, USA · Independent Researcher Austin, USA
Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data
Sep 10, 2026Atindra Jha, Margaret Li, Jure Leskovec +2Mixture-Of-Experts Overfitting
Stanford University · Paul G. Allen School of Computer Science, University of Washington
Overfitting Mitigation via Singular Value Decomposition in Minimum Bayes Risk Decoding
Sep 1, 2026Riza Setiawan Soetedjo, Yusuke Sakai, Hidetaka Kamigaito +2Singular Value Decomposition Low-Rank Structure
Nara Institute of Science and Technology (NAIST) · The University of Tokyo
High-dimensional ridgeless least squares interpolation under spiked covariance structures
Aug 7, 2026Zhijun Liu, Dandan JiangKernel Ridge Regression Covariance
College of Sciences, Northeastern University, China. · School of Mathematics and Statistics, Xi’an Jiao Tong University, China.
Rethinking Self-Evolution: A Constrained Exploration-Exploitation Process for Mitigating Skill Overfitting
Jul 29, 2026Hongqiang Lin, Chao Liu, Xiaofan Bai +4Skill Evolution Skills
Zhejiang University · Alibaba Group
MxGPS: Multiplex Graph Transformers for a Power Grid Foundation Model
Jul 15, 2026Charilaos Papaioannou, Ioannis Tsantilas, Dimitris Giannakakos +9Optimal Power Flow Multiplex Graph Transformers
Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
Jul 6, 2026Lucas PintoSabotage Model Auditing
Independent Researcher
Broken Ergodicity and the Violation of the Fluctuation-Dissipation Theorem Lead to Generalization Beyond Overfitting in Machine Learning
Jul 5, 2026Chan Li, Nigel GoldenfeldFluctuation-Dissipation Theorem Mean-Field Limit
Department of Physics, University of California San Diego, 9500 Gilman Drive, La Jolla, California 92093, USA. · Halıcıoğlu Data Science Institute, University of California San Diego, 9500 Gilman Drive, La Jolla, California 92093, USA
Benign Overfitting Does Not Occur in Diffusion Models
Jul 2, 2026Tyler Farghly, Benjamin Dupuis, Alain Durmus +1Overfitting Score-Based Diffusion Model
1INRIA - École Normale Supérieure - PSL Research University - CNRS, France · 2 École Polytechnique - CMAP - IP Paris, France
Demystifying Training-Time Augmentation for Data-Constrained Language Model Pretraining
Jun 15, 2026Michael K. Chen, Xikun Zhang, Fan Bai +2Large Language Model Pretraining Data Augmentation
1UC San Diego · 2RMIT University · 3Bloomberg AI
What Fits (Into Few Tokens) Doesn't Overfit: Compression and Generalization in ML Research Agents
Jun 9, 2026Martin Andres Bertran, Aaron Roth, Zhiwei Steven WuMle-Bench Lite Overfitting
Amazon Responsible AI · University of Pennsylvania · Carnegie Mellon University
A Theoretical Analysis of Memory and Overfitting Phenomena in Stochastic Interpolation Models
Jun 7, 2026Yunchen Li, Shaohui Lin, Zhou YuStochastic Interpolants Overfitting
East China Normal University
Assessing True Generalisability of Audio-Visual Speech Recognisers
Jun 5, 2026Zhaofeng Lin, Stavros Petridis, Maja Pantic +1Automatic Speech Recognition Evaluation Overfitting
Trinity College Dublin, Ireland · Imperial College London, UK
SORA: Free Second-Order Attacks in Fast Adversarial Training
May 30, 2026Mazdak Teymourian, Ramtin Moslemi, Farzan Rahmani +1Adversarial Training Attention-Guided Training
Department of Computer Engineering, Sharif University of Technology, Tehran, Iran.
Resolving Endpoint Underfitting in Diffusion Bridges via Noise Alignment
May 27, 2026Yurong Gao, Zicheng Zhang, Congying Han +2Diffusion Bridges Image Restoration
University of Chinese Academy of Sciences
A Principled Self-Referenced Early Stopping Approach for Deep Image Prior
May 24, 2026Chaoyan Huang, Cheng-Han Huang, Ismail R. Alkhouri +1Visual Priors Bayesian Inverse Problems
Department of Computational Mathematics, Science, & Engineering, Michigan State University · Department of Electrical Engineering and Computer Science, University of Michigan · X Computational Physics Division, Los Alamos National Laboratory +2
Test-Time Training Undermines Safety Guardrails
May 21, 2026Simone Antonelli, Sadegh Akhondzadeh, Aleksandar BojchevskiInference-Time Defense Test-Time Training
CISPA Helmholtz Center for Information Security · University of Cologne
Beyond Temperature: Hyperfitting as a Late-Stage Geometric Expansion
May 21, 2026Meimingwei Li, Yuanhao Ding, Esteban Garces Arias +1Overfitting
Department of Statistics, LMU Munich · School of Computer and Information Engineering, Henan University · Munich Center for Machine Learning (MCML)
Toward Understanding Adversarial Distillation: Why Robust Teachers Fail
May 21, 2026Hongsin Lee, Hye Won ChungTeacher-Student Distillation Dataset Distillation
School of Electrical Engineering, KAIST, Daejeon, Korea.
TextReg: Mitigating Prompt Distributional Overfitting via Regularized Text-Space Optimization
May 20, 2026Lucheng Fu, Ye Yu, Yiyang Wang +4Overfitting Text-Driven
Georgia Institute of Technology · University of Illinois Urbana-Champaign
Reconciling Contradictory Views on the Effectiveness of SFT in LLMs: An Interaction Perspective
May 18, 2026Junpeng Zhang, Lei Cheng, Guoxi Zhang +3Large Language Models Fail Overfitting
Shanghai Jiao Tong University · Beijing Institute for General Artificial Intelligence · UniDT +1
Detecting overfitting in Neural Networks during long-horizon grokking using Random Matrix Theory
May 12, 2026Hari K. Prakash, Charles H MartinOverfitting Neural Network Training
University of California San Diego Data Science and Engineering · Calculation Consulting San Francisco, CA, USA
FedOUI: OUI-Guided Client Weighting for Federated Aggregation
May 12, 2026Alberto Fernández-Hernández, Jose I. Mestre, Cristian Pérez-Corral +3Federated Learning Fedavg
Universitat Politècnica de València, Valencia, Spain · Universitat Jaume I, Castelló de la Plana, Spain · Openchip & Software Technologies S.L., Spain
OUI as a Structural Observable: Towards an Activation-Centric View of Neural Network Training
May 12, 2026Alberto Fernández-Hernández, Jose I. Mestre, Cristian Pérez-Corral +3Deep Network Neural Network
Universitat Politècnica de València, Valencia, Spain · Universitat Jaume I, Castelló de la Plana, Spain · Openchip & Software Technologies S.L., Spain
Data-driven transport modelling without overfit
May 9, 2026Peter Vanya, Katarína Šimková, Rastislav FarkašTransport Data-Driven
Sev.en Global Investments, V Celnici 4, 11000 Prague, Czechia · Vrije Universiteit Brussels, Brussels, Belgium · Universit´e Libre de Bruxelles, Brussels, Belgium +1
When Does $\ell_2$ -Boosting Overfit Benignly? High-Dimensional Risk Asymptotics and the $\ell_1$ Implicit Bias
May 7, 2026Ye Su, Jian Li, Yong LiuOverfitting Empirical Risk Minimization
Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences Shenzhen 518055, China · School of Artificial Intelligence Beijing Normal University Beijing, 100875, China · Gaoling School of Artificial Intelligence Renmin University of China Beijing 100872, China
Understanding diffusion models requires rethinking (again) generalization
May 7, 2026Pierre Marion, Yu-Han WuOverfitting Diffusion Models
Inria, École Normale Supérieure, PSL Research University · LPSM, Sorbonne University & Google DeepMind
MER-DG: Modality-Entropy Regularization for Multimodal Domain Generalization
May 3, 2026Yavuz Yarici, Ghassan AlRegibMultimodal Domain Generalization Cross-Modal
1OLIVES at the Center for Signal and Information Processing CSIP, School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA.
Unveiling the Backdoor Mechanism Hidden Behind Catastrophic Overfitting in Fast Adversarial Training
Apr 27, 2026Mengnan Zhao, Lihe Zhang, Tianhang Zheng +2Adversarial Training Overfitting
School of Computer Science and Technology, Anhui University, Hefei 230601, China · School of Information and Communication Engineering, Dalian University of Technology (DUT), Dalian 116024, China · School of Computer Science and Technology, Zhejiang University, Hangzhou 310058, China +1
Mitigating Error Amplification in Fast Adversarial Training
Apr 27, 2026Mengnan Zhao, Lihe Zhang, Bo Wang +3Adversarial Training Attention-Guided Training
Anhui University, Anhui, China · Dalian University of Technology, Liaoning, China · Zhejiang University, Zhejiang, China +1
Learning Curves and Benign Overfitting of Spectral Algorithms in Large Dimensions
Apr 25, 2026Weihao Lu, Qian Lin, Yingcun Xia +1Kernel Ridge Regression Spectral Representation Method
Department of Statistics and Data Science, National University of Singapore. · Department of Statistics and Data Science, Tsinghua University.
Exploiting LLM-as-a-Judge Disposition on Free Text Legal QA via Prompt Optimization
Apr 22, 2026Mohamed Hesham Elganayni, Runsheng Chen, Sebastian Nagl +1Llm-As-A-Judge Prompt Optimization
Technical University of Munich Germany
Benign Overfitting in Adversarial Training for Vision Transformers
Apr 21, 2026Jiaming Zhang, Meng Ding, Shaopeng Fu +2Adversarial Training Self-Supervised Vision Transformers
Division of CEMSE, King Abdullah University of Science and Technology · School of Statistics, Renmin University of China · Department of Computer Science and Engineering, State University of New York at Buffalo +1
Superficial Success vs. Internal Breakdown: An Empirical Study of Generalization in Adaptive Multi-Agent Systems
Apr 21, 2026Namyoung So, Seokgyu Jang, Taeuk KimModel-Based Multi-Agent Systems Overfitting
Department of Computer Science, Hanyang University, Seoul, Republic of Korea
Test-Time Perturbation Learning with Delayed Feedback for Vision-Language-Action Models
Apr 20, 2026Zehua Zang, Xi Wang, Fuchun Sun +4Diffusion-Based Vision-Language-Actions Test-Time Adaptation
Institute of Software, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · Tsinghua University, Beijing, China +2
Grokking of Diffusion Models: Case Study on Modular Addition
Apr 20, 2026Joon Hyeok Kim, Yong-Hyun Park, Mattis Dalsætra Østby +1Diffusion Models Grokking
Department of Computer and Information Science University of Pennsylvania Philadelphia, PA 19104, USA
Incoherent Deformation, Not Capacity: Diagnosing and Mitigating Overfitting in Dynamic Gaussian Splatting
Apr 17, 2026Ahmad DrobyGaussian Splatting Gaussian Primitives
Independent Researcher
Spectral Overfitting in Noisy Linear Probing of Pretrained Representations
Mar 2, 2026Zice Wang, Zhenyu ZhangNoisy Labels Overfitting
School of Future Technology, Northeastern University, Shenyang, China.
Training-free LLM Verification via Recycling Few-shot Examples
Jun 8, 2025Dongseok Lee, Jimyung Hong, Dongyoung Kim +1Training-Free Overfitting
Yonsei University · KAIST
Benign Overfitting with Quantum Kernels
Mar 21, 2025Joachim Tomasi, Sandrine Anthoine, Hachem KadriQuantum Kernels Overfitting
Aix-Marseille Univ, CNRS, LIS, Marseille, France. · Aix Marseille Univ, CNRS, I2M, Marseille, France.
Universality of Benign Overfitting in Binary Linear Classification
Jan 17, 2025Ichiro Hashimoto, Stanislav Volgushev, Piotr ZwiernikOverfitting Empirical Risk Minimization
Department of Statistical Sciences, University of Toronto

Oct 1, 2026, cs.LGJ/K move · Enter open · S save

Dataset Identity, Not Novelty: The Source of an Inflated OOD Detection Gain

Donghoon Lee, Shinjin Kang

Hongik University

Out-Of-Distribution Detection Out-Of-Distribution Imagenet Overfitting

Open paper AskPDF arXiv

A post-hoc out-of-distribution (OOD) detector reads the activations of a trained classifier and returns a score. It fits that score on in-distribution data, and the benchmarks that evaluate it supply a second piece of OOD data for the fitting itself. Some detectors tune a constant on it. Others fit a direction in feature space or train a flexible combiner and report the number that it reaches as the gain that is still available. Every such fit is validated on held-out samples of the same OOD dataset. That check rules out memorizing individual images. It says nothing about a fit that has instead learned which dataset it is looking at, and a direction that recognizes one OOD dataset rather than novelty passes it perfectly. The detector that a practitioner installs meets OOD data from a source that nobody fitted it on, so the difference decides what the reported number is worth. We measure it by holding out the whole OOD dataset rather than a sample of it, and we call that gap the inflation. We read it across a range of combiners on ImageNet and CIFAR-100 backbones. Most of the gain that the usual protocol reports turns out to be dataset identity rather than novelty. The size of the fit does not move what survives, so the effect is not ordinary overfitting. The share depends instead on whether the input exposes class identity, and two controls that vary that property alone separate the inflation on every backbone of both benchmarks. A closed form accounts for the effect and computes it from the fitting rows, so a practitioner can tell which fits will inflate without running the hold-out protocol. One of these fits survives, namely the single constant that the field already picks on a designated validation dataset. Anything above it reports a gain that the hold-out protocol does not return, and on one benchmark what survives falls while what is reported climbs.

Momentum

Related topics

Latest papers 48