Bias Variance Decomposition

Bias-variance decomposition is a fundamental concept in machine learning that analyzes a model's prediction error by separating it into bias (systematic error) and variance (error due to model sensitivity to training data). Current research focuses on extending this decomposition to diverse settings, including ensemble methods, imbalanced datasets, and generative models, often employing neural networks and boosting algorithms. These investigations aim to improve model performance and uncertainty estimation, leading to more reliable and robust predictions across various applications, from healthcare to natural language processing. A deeper understanding of the bias-variance trade-off is crucial for designing better algorithms and interpreting model behavior.

Papers

October 19, 2024

Theoretical Aspects of Bias and Diversity in Minimum Bayes Risk Decoding
Hidetaka Kamigaito, Hiroyuki Deguchi, Yusuke Sakai, Katsuhiko Hayashi, Taro Watanabe
Text Generation Language Generation Absolute Stance Bias Diversity Awareness Minimum Bayes Risk Bias Variance Decomposition Theoretical Consideration

June 10, 2024

Data-Driven Switchback Experiments: Theoretical Tradeoffs and Empirical Bayes Designs
Ruoxuan Xiong, Alex Chin, Sean J. Taylor
Optimal Trade Empirical Bayes Spillover Effect Bias Variance Decomposition

February 9, 2024

Towards a Systematic Approach to Design New Ensemble Learning Algorithms
João Mendes-Moreira, Tiago Mendes-Neves
Ensemble Learning Systematic Approach Ensemble Algorithm Bias Variance Decomposition

February 6, 2024

A Bias-Variance Decomposition for Ensembles over Multiple Synthetic Datasets
Ossi Räisä, Antti Honkela
Synthetic Data Diverse Ensemble Downstream Forecasting Task Bias Variance Decomposition

November 20, 2023

Estimation of entropy-regularized optimal transport maps between non-compactly supported measures
Matthew Werenski, James M. Murphy, Shuchin Aeron
Estimation Task New Measure Bias Variance Decomposition Entropy Regularized Optimal Transport Subgaussian Assumption

October 28, 2023

Rethinking Semi-Supervised Imbalanced Node Classification from Bias-Variance Decomposition
Divin Yan, Gengchen Wei, Chen Yang, Shengzhong Zhang, Zengfeng Huang
Class Imbalance Node Classification Graph Structured Data Data Imbalance Imbalanced Datasets Imbalanced Node Classification Bias Variance Decomposition

October 9, 2023

A Bias-Variance-Covariance Decomposition of Kernel Scores for Generative Models
Sebastian G. Gruber, Florian Buettner
Large Language Model Generative Model Language Generation Bias Variance Decomposition

May 29, 2023

On the Variance, Admissibility, and Stability of Empirical Risk Minimization
Gil Kur, Eli Putterman, Alexander Rakhlin
Core Stability Empirical Risk Minimization Variance Information Square Loss Bias Variance Decomposition

February 17, 2023

The Unbearable Weight of Massive Privilege: Revisiting Bias-Variance Trade-Offs in the Context of Fair Prediction
Falaah Arif Khan, Julia Stoyanovich
Context Information Gaussian Mixture Model Bias Variance FAir Classifier Conditional Model Bias Variance Decomposition Fair Prediction Weight Monitoring Privileged Feature

January 10, 2023

A Unified Theory of Diversity in Ensemble Learning
Danny Wood, Tingting Mu, Andrew Webb, Henry Reeve, Mikel Luján, Gavin Brown
Ensemble Learning Diversity Awareness Unified Model Ensemble Diversity Bias Variance Decomposition

October 21, 2022

Uncertainty Estimates of Predictions via a General Bias-Variance Decomposition
Sebastian G. Gruber, Florian Buettner
Human Prediction Uncertainty Estimation Model Uncertainty Bregman Information Bias Variance Decomposition Confidence Region

July 11, 2022

Multi-Study Boosting: Theoretical Considerations for Merging vs. Ensembling
Cathy Shyr, Pragya Sur, Giovanni Parmigiani, Prasad Patil
Research Reproducibility Model Ensembling Bias Variance Decomposition Theoretical Consideration

June 21, 2022

Ensembling over Classifiers: a Bias-Variance Perspective
Neha Gupta, Jamie Smith, Ben Adlam, Zelda Mariet
Simple Classifier Bootstrapping End to End Ensemble Classifier Bias Variance Bias Variance Decomposition Bias Reduction

May 19, 2022

Diverse Weight Averaging for Out-of-Distribution Generalization
Alexandre Ramé, Matthieu Kirchmeyer, Thibaud Rahier, Alain Rakotomamonjy, Patrick Gallinari, Matthieu Cord
Distribution Generalization WEight AVERaging Bias Variance Decomposition Multi Domain Benchmark

April 26, 2022

Bias-Variance Decompositions for Margin Losses
Danny Wood, Tingting Mu, Gavin Brown
Convex Loss Additive Model Square Loss Margin Loss Bias Variance Decomposition

March 10, 2022

Bias-variance decomposition of overparameterized regression with random linear features
Jason W. Rocks, Pankaj Mehta
Novel Regression Random Feature Random Matrix Bias Variance Bias Variance Decomposition

December 13, 2021

Addressing Bias in Active Learning with Depth Uncertainty Networks... or Not
Chelsea Murray, James U. Allingham, Javier Antorán, José Miguel Hernández-Lobato
Active Learning Generalization Error Depth Network Bias Correction Bias Variance Decomposition Addressing Bias