Fisher Information

Follow this topic Ask about this topic

Fisher information measures how much observable data or predictions change when parameters change, giving a geometric view of sensitivity and uncertainty. In machine learning, researchers use Fisher matrices and approximations such as diagonal or Kronecker-factored forms for continual-learning methods like elastic weight consolidation, model compression, fine-tuning, robustness analysis, and data selection. Related work studies quantum Fisher information, where measurement choices affect how much information about a quantum state is accessible, as well as uses in sampling and causal discovery. A central challenge is making these measures computationally tractable while understanding when their estimates reflect meaningful importance: results can depend on the data, model, readout, and whether sensitivity is measured locally or across a training trajectory.

Latest papers

22 papers6 papers in the last four weeks, up 100% on the four weeks before. 0.0% of all new papers.

FisherEdit: Fisher Information Guided Knowledge Editing for Large Language Models
Oct 19, 2026Yizhou Yang, Jiahong Fu, Zhiwu Wang +6Knowledge Editing Language Model Editing
Xi’an Jiaotong University · Zhejiang University · City University of Hong Kong +2
Fisher information and the geometry of memorization in neural networks
Oct 12, 2026Daniel T. Bernstein, Chase Goddard, Luca Di Carlo +2Neural Network Memorization Neural Network Generalization
Princeton University1; Lewis-Sigler Institute · Princeton University2; Joseph Henry Laboratories · University of Sydney3; School of Physics +1
Local Fisher Information Enables Sparse Causal Discovery
Oct 3, 2026Byeongguk Kang, Donghyeon Lee, Euijong Song +1Causal Discovery Fisher Information
Interdisciplinary Program in Artificial Intelligence Seoul National University, Korea · Department of Statistics Seoul National University, Korea · Institute for Data Innovation in Science Seoul National University, Korea
Fisher-Guided Submodular Data Selection for Continual Pre-Training of Large Language Models
Oct 1, 2026Zhenghao Zhao, Gaowen Liu, Zhiling Lan +1Continued Pretraining Catastrophic Forgetting
University of Illinois Chicago · Cisco Research
Predictive Geometry of Hidden Trajectories in Transformers
Sep 29, 2026Timur Mudarisov, Mikhail Burtsev, Tatiana Petrova +1Decoder-Only Language Models Representation Geometry in Language Models
University of Luxembourg, Luxembourg
Understanding LLM Parameter Update Sparsity through the Lens of Fisher
Sep 28, 2026Yufan Zhang, Sagnik Mukherjee, Hao PengNeural Network Training Dynamics Sparse Fine-Tuning
University of Illinois Urbana-Champaign
Fisher Simplicity in Kolmogorov-Arnold Networks and Multilayer Perceptrons
Sep 26, 2026Ami Tavory, Meir FederNeural Network Interpretability Kolmogorov-Arnold Networks
Meta Platforms Tel-Aviv, Israel · School of Electrical and Computer Engineering Tel-Aviv University Tel-Aviv, Israel
Certification cost of quantum models: measurement correlation, not parameter count
Sep 13, 2026Pavel Sulimov, Claude LehmannFisher Information Variational Quantum Circuits
Zurich University of Applied Sciences (ZHAW), Winterthur, Switzerland · University of Zurich, Zurich, Switzerland
Scalable Kronecker-Fisher Approximation: Efficient Hessian Analysis for Billion-Parameter Language Models Compression
Sep 2, 2026Viacheslav Yusupov, Daria Cherniuk, Evgeny FrolovLLM Compression Fisher Information
1HSE University · 2AXXX
Task-Resolved Fisher Spectroscopy for Quantum Reservoir Computing
Aug 26, 2026Yang PengReservoir Computing Quantum Reservoir Computing
Department of Physics and Astronomy, California State University, Northridge, California 91330, USA · Institute of Quantum Information and Matter and Department of Physics, California Institute of Technology, Pasadena, California 91125, USA
Loss-Parameterized Fisher Width Along Learning Trajectories
Aug 21, 2026Vu Khac KyFisher Information Neural Network Training Dynamics
Department of Mathematics, FPT University, Vietnam
Readout-Rank Laws for Isotropic Quantum Tangents
Aug 7, 2026Marwan Ait HaddouInformation Geometry Variational Quantum Circuits
Independent Researcher, Morocco
Rethinking Quantum Continual Learning with Quantum Fisher Information
Jul 17, 2026Yu-Chao Hsu, Yu-Cheng Lin, Tai-Yue Li +2Quantum Machine Learning Variational Quantum Circuits
School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea · Department of Electrophysics, National Yang Ming Chiao Tung University, Hsinchu 300092, Taiwan. · National Center for High-Performance Computing , National Institutes of Applied Research (NIAR), Hsinchu 300092, Taiwan
Fisher Rank Inflation: A Spectral Signature of Memorization under Label Noise
Jul 14, 2026Satwik Bathula, Anand A. JoshiNeural Network Memorization Noisy-Label Learning
Department of Electrical and Computer Engineering, University of Southern California
Spectral Perturbation of the Empirical Fisher Information Matrix under Weight Quantization
Jun 26, 2026Rahid Zahid Alekberli, Hikmat KarimovInformation Geometry Weight-Only Quantization
Institute of Defense Technologies and Cybersecurity, Azerbaijan Technical University, Baku, Azerbaijan
Input-Dependent Fisher Information for Local Sensitivity Analysis of Medical Image Classifiers
Jun 15, 2026Sourya Sengupta. Mark A. AnastasioNeural Network Interpretability Medical Image Classification
Department of Electrical and Computer Engineering, University of Illinois Urbana–Champaign, Urbana, IL 61801 USA
Fisher-Guided Progressive Parameter Selection for Adaptive Fine-Tuning
Jun 8, 2026Ghodsiyeh Rostami, Po-Han Chen, Mahdi S. HosseiniFine-Tuning Vision Foundation Model Adaptation
Concordia University · Concordia University Mila - Québec AI Institute
Measuring Model Robustness via Fisher Information: Spectral Bounds, Theoretical Guarantees, and Practical Algorithms
Jun 3, 2026Chong Zhang, Xiang Li, Jia Wang +3Neural Network Robustness Adversarial Robustness
1Xi’an Jiaotong-Liverpool University · 2The Chinese University of Hong Kong.
Complexity of Non-Log-Concave Sampling in Fisher Information
May 15, 2026Sinho Chewi, Andre WibisonoOracle Complexity Log-Concave Sampling
Yale University, Department of Statistics and Data Science. · Yale University, Department of Computer Science.
Beyond Activation Alignment: The Geometry of Neural Sensitivity
May 4, 2026Amirhossein Yavari, Farnaz Zamani EsfahlaniNeural Representation Geometry Neural Decoding
Stephenson School of Biomedical Engineering, University of Oklahoma · Data Science and Analytics Institute, University of Oklahoma
Elastic Weight Consolidation Done Right for Continual Learning
Mar 19, 2026Xuan Liu, Xiaobin ChangRegularization-Based Continual Learning Continual Learning
School of Artificial Intelligence, Sun Yat-sen University, China · Key Laboratory of Intelligent Assessment Technology for Sustainable Tourism, Ministry of Culture and Tourism, Sun Yat-sen University · Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, China
Personalized Federated Learning with Residual Fisher Information for Medical Image Segmentation
Mar 16, 2026Meilu Zhu, Yuxing Li, Zhiwei Wang +1Federated Learning Heterogeneous Federated Learning
Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, China

Oct 19, 2026J/K move · Enter open · S save

FisherEdit: Fisher Information Guided Knowledge Editing for Large Language Models

Yizhou Yang, Jiahong Fu, Zhiwu Wang, Kailong Fan+5

Xi’an Jiaotong University · Zhejiang University · City University of Hong Kong+2

Knowledge Editing Language Model Editing Continual Learning for LLMs Fisher Information

Open paper AskPDF arXiv

Sequential knowledge editing in Large Language Models (LLMs) often leads to a degradation of general capabilities due to cumulative parameter interference. In this paper, we revisit prominent projection-based methods like AlphaEdit and reveal a surprising mismatch: their robustness is primarily driven by update regularization rather than the intended null-space projection. Based on this insight, we propose FisherEdit, a theoretically grounded method built on Fisher geometry, which replaces coarse activation-space projections with importance-aware regularization in parameter space. It penalizes changes to sensitive parameters more strongly while retaining flexibility for new facts, and admits an efficient row-wise closed-form solution. Experiments on Llama-3, Qwen2.5, and Olmo-3 across multiple benchmarks demonstrate that FisherEdit achieves state-of-the-art performance while better preserving model stability and downstream task accuracy.