Spectral Bias

Ask about this topic

Overview

Spectral bias describes how neural networks often fit smooth, low-frequency structure before fine detail, which can limit performance on oscillatory signals, sharp boundaries, and complex physical fields. Researchers study this behavior in implicit neural representations, physics-informed neural networks, neural operators, and other models, using tools such as Fourier analysis and neural tangent kernels. Mitigation methods include Fourier and wavelet features, SIREN-style periodic activations, frequency-aware losses, local or multibranch architectures, and changes to initialization or optimization. A central open question is when these interventions genuinely improve high-frequency fidelity and generalization, since benefits vary across tasks and added frequency capacity can also complicate training or distort important structure.

Latest papers

CardsList

37 papers6 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

  1. Task-aligned spectral inductive biases in parameterized quantum circuits

    Oct 16, 2026Felix Frohnert, Adrián Pérez-Salinas, Sebastian Lange +3Quantum Machine LearningSpectral Bias

    ⟨aQaL⟩Applied Quantum Algorithms, Universiteit Leiden, Leiden, The Netherlands · Dahlem Center for Complex Quantum Systems, Freie Universität Berlin, Germany · Institute for Theoretical Physics, ETH Zürich, Switzerland +2

  2. Why Shallow Neural Networks Learn Sinusoidal Components First: An Orthogonal Polynomial Dynamics Framework

    Oct 15, 2026Yanyang XiaoSpectral BiasNeural Network Training Dynamics

    Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences

  3. Spatial-Frequency-Aware Implicit Neural Representation of Multidimensional Signals via MLP-KAN Fusion

    Oct 8, 2026Wen Yan, Ligen Shi, Jun Qiu +3Implicit Neural RepresentationsSpectral Bias

    Institute of Computational Imaging, Beijing Information Science and Technology University, Beijing 102206, China · College of Computer Science (College of Software), Inner Mongolia University, Hohhot 010021, China · Research Center for Spatiotemporal Intelligence, Inner Mongolia University, Hohhot 010021, China

  4. Optimizer Geometry Sets the Pace: Spectral Learning Dynamics in Matrix Factorization

    Oct 3, 2026Mahalakshmi Sabanayagam, Simon LuceyGradient Descent DynamicsSpectral Bias

    Australian Institute for Machine Learning, Adelaide University

  5. Operator-informed initialization for Fourier features physics-informed neural networks

    Oct 2, 2026Juan Molina, Paris Perdikaris, Mircea Petrache +2Physics-Informed MLNeural Network Initialization

    National Center for Artificial Intelligence, Santiago, Chile · Department of Mechanical Engineering and Applied Mechanics, University of Pennsylvania, Philadelphia, PA, USA · Department of Mathematics, Pontificia Universidad Católica de Chile, Santiago, Chile +3

  6. Contrastive Attention Mitigates Spectral Bias in Spiking Transformers

    Oct 1, 2026Xiaoli Liu, Malu Zhang, Yang YangSpiking TransformersSelf-Attention

    School of Computer Science and Engineering, University of Electronic Science and Technology of China

  7. Spectral Reversal: Counteracting Singular Value Bias for Graph Prompting

    Sep 26, 2026Hanxu Yang, Yuhuan Zhao, Xiaodong He +1Graph Neural NetworksSpectral Bias

    School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China

  8. Balancing Frequencies and Pixels in Flow Matching

    Sep 2, 2026Lucas Degeorge, Paul Couairon, Arijit Ghosh +3Image GenerationFlow Matching

    LIX, École Polytechnique, CNRS, IP Paris, France · AMIAD · LIGM, École Nationale des Ponts et Chaussées, IP Paris, UGE, CNRS, France +1

  9. When Does Frequency Decomposition Benefit Physics-Informed Neural Networks? A Preliminary Ablation Study

    Aug 24, 2026Shubham RaiNeural PDE SolversPhysics-Informed ML

  10. More Experts, Worse Dynamics: Inverse Scaling and Spectral Bias in Mixture-of-Experts State-Space Models

    Aug 22, 2026Chandresh PandeyMixture-of-Experts ModelsState Space Models

    Computer Science and Engineering, Delhi Technological University, India

  11. Sustaining Plasticity via Learnable Wavelet Activations in Continual Learning

    Aug 13, 2026Zeyang Zhang, Tieliang Gong, Junyan Lu +1Loss of PlasticityContinual Learning

    School of Computer Science and Technology, Xi’an Jiaotong University and Institute of Multimedia Knowledge Fusion and Engineering, Xi’an 710049, China

  12. Alternating Levenberg-Marquardt Training of Physics-Informed Neural Networks with Fourier-Enhanced Features

    Aug 6, 2026Yulun Wu, Matthieu Barreau, Miguel Aguiar +1Neural PDE SolversSpectral Bias

    Division of Decision and Control Systems, School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, 100 44, Sweden

  13. SPARC-Net: A Spectral, Causality-Aware, and Hard-Constrained Physics-Informed Architecture for Stiff and Shock-Dominated Partial Differential Equations

    Jul 13, 2026Divyavardhan Singh, Dimple Sonone, Hammad Mohammad +1PDE SolvingSpectral Bias

    Dept. of Electronics Engg. SVNIT Surat, India

  14. Frequency Shift Physics-Informed Extreme Learning Machine for Solving High-Frequency Partial Differential Equations

    Jul 2, 2026Xiong Xiong, Ruonan Zhai, Zheng Zeng +3PDE SolvingFrequency-Domain Feature Learning

    School of Mathematics and Statistics, Northwestern Polytechnical University, Xi’an, 710072, China · MIIT Key Laboratory of Dynamics and Control of Complex Systems, Northwestern Polytechnical University, Xi’an, 710072, China · College of Science, Shijiazhuang University, Shijiazhuang, 050035, China +2

  15. ScaLe-INR: Scale and Learn Implicit Neural Representations

    Jun 26, 2026Buwaneka Epakanda, Athulya Ratnayake, Pandula Thennakoon +4Implicit Neural RepresentationsFrequency-Domain Feature Learning

  16. Calibrated Harmonic Overlaid Implicit Neural Representations for Multi-Dimensional Data

    Jun 25, 2026Honghang Chen, Xiujun Zhang, Xiaoli Sun +1Implicit Neural RepresentationsSpectral Bias

    Shenzhen University, Shenzhen, China · Shenzhen Polytechnic University, Shenzhen, China · Southern Illinois University Carbondale, USA

  17. Spectral Gating via Damped Oscillations for Adaptive Implicit Neural Representations

    Jun 22, 2026Alex Costanzino, Pierluigi Zama Ramirez, Giuseppe Lisanti +1Implicit Neural RepresentationsFrequency-Domain Feature Learning

    CVLab, University of Bologna · Ca’ Foscari University of Venice

  18. The Fractal Neural Operator: Overcoming Spectral Bias in Chaotic Attractors via Prime-Harmonic Weierstrass Encodings

    Jun 22, 2026Kanishk AwadhiyaSpectral BiasChaotic Dynamical Systems

    Independent Researcher

  19. RepNN: Tackling spectral bias in deep neural networks for regression and PDE problems via parameter reparameterization

    Jun 15, 2026Yong Wang, Tao Zhou, Xuhui MengPDE SolvingSpectral Bias

    Institute of Interdisciplinary Research for Mathematics and Applied Science, School of Mathematics and Statistics, Huazhong University of Science and Technology, Wuhan 430074, China · Institute of Computational Mathematics, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China

  20. SirenFNO: Efficient and Full Frequency Learning of Fourier Neural Operators

    Jun 9, 2026Pengqing Shi, Jie Yin, Stephen Tierney +1Fourier Neural OperatorSpectral Bias

    The University of Sydney, Australia

  21. Correcting Neural Operator Spectral Bias via Diffusion Posterior Sampling with Sparse Observations

    Jun 2, 2026Niccolò Perrone, Fanny Lehmann, Stefania Fresca +1PDE Surrogate ModelingSpectral Bias

    Université Paris-Saclay, CentraleSupélec, CNRS, ENS Paris-Saclay · Laboratoire de Mécanique Paris-Saclay UMR 9026 · 8-10 rue Joliot Curie, 91190 Gif-sur-Yvette, France +5

  22. Colored Noise Diffusion Sampling

    May 28, 2026Hadar Davidson, Noam Issachar, Sagie BenaimDiffusion SamplingSpectral Bias

    The Hebrew University of Jerusalem

  23. Aperiodic and Low-Frequency Spectral Bias in Reconstruction based EEG Foundation Models

    May 26, 2026Aditya Kommineni, Emily Zhou, Kleanthis Avramidis +7Self-Supervised Time Series Representation LearningEEG Decoding

    Ming Hsieh Department of Electrical and Computer Engineering1 University of Southern California · Thomas Lord Department of Computer Science2[cs.LG] University of Southern California · Department of Mathematical Sciences3 Aalborg University

  24. Iterative Refinement Neural Operators are Learned Fixed-Point Solvers: A Principled Approach to Spectral Bias Mitigation

    May 21, 2026Xiaotian Liu, Shuyuan Shang, Xiaopeng Wang +2Neural PDE SolversSpectral Bias

    1Dartmouth College · 2CUHK Shenzhen · 3Lawrence Berkeley National Lab

  25. A Fourier perspective on the learning dynamics of neural networks: from sample complexities to mechanistic insights

    May 16, 2026Fabiola Ricci, Claudia Merger, Sebastian GoldtSample ComplexityFrequency-Domain Feature Learning

    International School of Advanced Studies (SISSA), Trieste, Italy

  26. Frequency Bias and OOD Generalization in Neural Operators under a Variable-Coefficient Wave Equation

    May 13, 2026Runlong Xie, An LuoFourier Neural OperatorPDE Solving

    Independent Researcher · School of Statistics, University of Minnesota, MN, USA.

  27. Spectral Energy Centroid: a Metric for Improving Performance and Analyzing Spectral Bias in Implicit Neural Representations

    May 12, 2026Tomasz Dądela, Adam Kania, Maciej Rut +1Implicit Neural RepresentationsSpectral Bias

    Jagiellonian University IDEAS Kraków, Poland

  28. Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit

    May 8, 2026Konstantin Riedl, Konstantinos Spiliopoulos, Justin SirignanoOverparameterized Neural NetworkNeural Network Optimization

    Mathematical Institute University of Oxford · Department of Mathematics & Statistics Boston University

  29. Diagnosing Spectral Ceilings in Equivariant Neural Force Fields

    May 8, 2026Hyunmog KimMachine Learning Interatomic PotentialsSpectral Bias

    Independent Researcher.

  30. Topological Neural Tangent Kernel

    May 1, 2026Sanjukta KrishnagopalSpectral BiasHigher-Order GNNs

    Department of Computer Science, University of California Santa Barbara