Overview

Implicit bias is the tendency of a model and its training procedure to select some solutions or representations over others, even without an explicit regularizer; these preferences shape generalization, extrapolation, and sometimes fairness. Researchers derive these preferences from gradient descent, gradient flow, and stochastic optimization in settings such as separable classification, deep linear and ReLU networks, matrix factorization, and attention, often finding max-margin, sparse, or low-rank solutions. They vary initialization, depth, parameterization, optimizer, step size, and noise, and test consequences in tasks including length generalization, shortcut learning, and demographic representations in language and vision-language models. Current work is moving beyond asymptotic results in simplified models toward finite-time dynamics and realistic architectures, but the selected bias can shift during training and improve one notion of generalization while worsening another; fair outputs can also conceal biased internal representations.

Latest papers

CardsList

36 papers7 papers in the last four weeks, up 75% on the four weeks before. 0.0% of all new papers.

  1. Classification with Large-Step Gradient Descent: Unifying Self-Stabilization and Convergence Rates

    Nov 12, 2026Salma Tarmoun, Lachlan Ewen MacDonald, Konstantinos Emmanouilidis +2Large-Step-Size Gradient DescentGradient Descent Dynamics

  2. Implicit Bias in State Space Models and Linear Autoregressive Training

    Nov 6, 2026Gal Katzhendler, Gilad Yehudai, Gal VardiImplicit BiasState Space Models

  3. Implicit Bias of (Stochastic) Gradient Descent on Deep Linear Diagonal Networks

    Oct 29, 2026Wiebke Bartolomaeus, Holger RauhutImplicit BiasImplicit Regularization

  4. RAGing Against Misogyny: Improving Implicit Misogyny Explanations

    Oct 26, 2026Ivana Crescenzi, Arianna Muti, Maarten Sap +2Implicit BiasSocial Bias in Language Models

  5. Mapping the Solution-Set Structure of Grokking on Latin Squares

    Oct 26, 2026Zihan Gu, Lulu Zhao, Tianyi Ma +4GrokkingNeural Network Generalization

  6. A Minimal ReLU Classifier on Non-Linearly Separable Data: Convergence and Implicit Bias at and beyond the Edge of Stability

    Oct 16, 2026Quan Nguyen, Robin Yadav, Param Mody +1Neural Network OptimizationReLU Neural Networks

  7. From Geometry to Generalization: Why Row Normalization Can Beat Adam and Muon

    Oct 8, 2026Jihwan Kim, Dogyoon Song, Chulhee YunAdaptive Gradient MethodsClassification

  8. Learning the identity: a case study of how SGD selects among functional decompositions

    Sep 30, 2026Andy Arditi, Weian Xie, David Bau +1Deep Linear NetworksImplicit Bias

  9. On the Two Faces of Adam in Separable Linear Classification

    Sep 27, 2026Chen Fan, Csaba SzepesváriImplicit BiasClassification

  10. High-Dimensional Learning Dynamics of Attention-Indexed Models

    Sep 3, 2026Yizhou Xu, Margarita Sagitova, Lenka Zdeborová +1Implicit BiasAttention Mechanisms

  11. Non-asymptotic implicit bias of logistic regression at early-stage gradient descent dynamics

    Aug 5, 2026Han BaoLogistic RegressionImplicit Bias

  12. Variable Selection in the Context of AI Fairness

    Aug 4, 2026Ivan Luciano Danesi, Chiara Frigerio, Fabio Maccaferri +2Algorithmic FairnessGroup Fairness

  13. From Minds to Models: The Intersection of Psychology and LLM Behaviours

    Jul 30, 2026Oliver Guidetti, Reza RyanSocial Bias in Language ModelsImplicit Bias

  14. Relative Positions Generalize, Absolute Positions Memorize: An Implicit-Bias Account of Length Generalization in Attention

    Jul 21, 2026Subham Singh, Ashutosh Mishra, Subha RautSoftmax AttentionRotary Positional Embeddings

  15. Gradient Flow Dynamics and Implicit Bias of Diagonal Linear Networks under Infinitesimal Initialization

    Jul 14, 2026Jiajie Zhao, Jianxing Wang, Junjie Yang +2Deep Linear NetworksImplicit Bias

  16. The Anatomy of Implicit Bias: Information Allocation in Neural Network Training

    Jul 8, 2026Zhang Gongyue, Wang Zhiyong, Liu Donghan +3Neural Network OptimizationImplicit Bias

  17. Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse

    Jul 3, 2026Shuang Liang, Tom Jacobs, Guido MontúfarShallow Neural NetworksReLU Neural Networks

  18. Fisher-Geometric Sharpness and the Implicit Bias of SGD toward Flat Minima

    Jun 18, 2026Md Sakir Ahmed, Kumaresh Sarmah, Hemen DuttaInformation GeometryImplicit Bias

  19. Vision-Language Models Suppress Female Representations Under Ambiguous Input

    May 29, 2026Arnau Marin-Llobet, Simon Henniger, Mahzarin R. BanajiVLM EvaluationVLM Interpretability

  20. The Implicit Bias of Depth: From Neural Collapse to Softmax Codes

    May 21, 2026Connall Garrod, Jonathan P. Keating, Christos ThrampoulidisDeep Linear NetworksImplicit Bias

  21. Estimating Implicit Regularization in Deep Learning

    May 6, 2026Joseph H. Rudoler, Kevin Tan, Giles Hooker +1Neural Network OptimizationImplicit Bias

  22. Towards The Implicit Bias on Multiclass Separable Data Under Norm Constraints

    Mar 24, 2026Shengping Xie, Zekun Wu, Quan Chen +1Implicit BiasGradient Descent

  23. Dual Space Preconditioning for Gradient Descent in the Overparameterized Regime

    Mar 11, 2026Reza Ghane, Danil Akhtiamov, Babak HassibiGradient DescentImplicit Bias

  24. Minor First, Major Last: A Depth-Induced Implicit Bias of Sharpness-Aware Minimization

    Mar 9, 2026Chaewon Moon, Dongkuk Si, Chulhee YunSharpness-Aware MinimizationImplicit Bias