Covariance Conditioning

Covariance conditioning focuses on accurately modeling and leveraging the covariance structure of data, aiming to improve the performance and robustness of various machine learning and statistical methods. Current research emphasizes developing efficient algorithms for covariance estimation in high-dimensional settings, often incorporating deep learning architectures like neural networks and Gaussian processes, and exploring applications in diverse fields such as class incremental learning, diffusion models, and time series forecasting. Improved covariance modeling leads to more accurate uncertainty quantification, enhanced model generalization, and better performance in tasks ranging from image classification to probabilistic traffic prediction.

Papers

October 29, 2024

Joint Estimation of Conditional Mean and Covariance for Unbalanced Panels
Damir Filipovic, Paul Schneider
Covariance Matrix Relevant Covariates Covariance Conditioning Portfolio Selection Joint Estimation Panel Discussion Asset Pricing

October 18, 2024

WeSpeR: Population spectrum retrieval and spectral density estimation of weighted sample covariance
Benoit Oriol
App to App Retrieval Covariance Conditioning Na\"ive Population Spectral Energy Distribution Spectral Estimator Sample Covariance Matrix

October 17, 2024

Generalization for Least Squares Regression With Simple Spiked Covariances
Jiping Li, Rishi Sonthalia
Strong Generalization Generalization Error Least Square Generalization Property Random Matrix Spike Protein Covariance Conditioning Spiked Matrix

October 15, 2024

Free Hunch: Denoiser Covariance Estimation for Diffusion Models Without Extra Costs
Severi Rissanen, Markus Heinonen, Arno Solin
Diffusion Model Hidden CoST Covariance Conditioning Covariance Estimation Generative Trajectory

October 12, 2024

Identification of Non-causal Graphical Models
Junyao You, Mattia Zorzi
Person Identification Autoregressive Model Covariance Conditioning Autoregressive Graph

October 8, 2024

Simple Relative Deviation Bounds for Covariance and Gram Matrices
Daniel Barzilai, Ohad Shamir
Covariance Conditioning Smallest Eigenvalue Gram Matrix

October 2, 2024

Induced Covariance for Causal Discovery in Linear Sparse Structures
Saeed Mohseni-Sehdeh, Walid Saad
Causal Discovery Causal Model Causal Relationship Cause Effect Sparse Structure Covariance Conditioning Causal Discovery Algorithm Sparse Causal

September 26, 2024

Task-recency bias strikes back: Adapting covariances in Exemplar-Free Class Incremental Learning
Grzegorz Rypeść, Sebastian Cygert, Tomasz Trzciński, Bartłomiej Twardowski
Feature Extractor Covariance Conditioning Event Collapse Exemplar Free Class Incremental Learning Dimensional Collapse Recency Bias

June 16, 2024

Improving Probabilistic Diffusion Models With Optimal Covariance Matching
Zijing Ou, Mingtian Zhang, Andi Zhang, Tim Z. Xiao, Yingzhen Li, David Barber
Diffusion Model Diffusion Probabilistic Model Covariance Conditioning Covariance Approximation

May 23, 2024

Iterative Methods for Full-Scale Gaussian Process Approximations for Large Spatial Data
Tim Gyger, Reinhard Furrer, Fabio Sigrist
Gaussian Process Adaptive Preconditioner Covariance Conditioning Conjugate Gradient Iterative Method Probabilistic Regression

April 4, 2024

Conditioning of Banach Space Valued Gaussian Random Variables: An Approximation Approach Based on Martingales
Ingo Steinwart
Gaussian Process Conditional Distribution Covariance Conditioning Consistent Conditioning Gaussian Data Martingale Difference Banach Space Approximation Method

March 13, 2024

Self-Supervised Learning for Covariance Estimation
Tzvi Diskin, Ami Wiesel
Deep Learning Self Supervised Learning Foundation Model Covariance Conditioning Covariance Estimation Adaptive Detection

February 14, 2024

Synergistic eigenanalysis of covariance and Hessian matrices for enhanced binary classification
Agus Hartoyo, Jan Argasiński, Aleksandra Trenk, Kinga Przybylska, Anna Błasiak, Alessandro Crimi
Hessian Matrix Covariance Matrix Discriminant Analysis Covariance Conditioning Improved Classification Class Separability

February 3, 2024

Improving Diffusion Models for Inverse Problems Using Optimal Posterior Covariance
Xinyu Peng, Ziyang Zheng, Wenrui Dai, Nuoqian Xiao, Chenglin Li, Junni Zou, Hongkai Xiong
Diffusion Model Inverse Problem Covariance Conditioning Posterior Covariance

February 1, 2024

Multivariate Probabilistic Time Series Forecasting with Correlated Errors
Vincent Zhihao Zheng, Lijun Sun
Covariance Conditioning Underlying Causal Probabilistic Time Series Forecasting Correlated Noise Covariance Structure

October 29, 2023

TIC-TAC: A Framework for Improved Covariance Estimation in Deep Heteroscedastic Regression
Megh Shukla, Mathieu Salzmann, Alexandre Alahi
New Framework Covariance Conditioning Covariance Estimation Negative Log Likelihood Heteroscedastic Regression Task Correlation Gradient Covariance

August 17, 2023

Hitting the High-Dimensional Notes: An ODE for SGD learning dynamics on GLMs and multi-index models
Elizabeth Collins-Woodfin, Courtney Paquette, Elliot Paquette, Inbar Seroussi
High Dimensional Stochastic Gradient Descent Covariance Conditioning Multi Index Model Note Level

June 23, 2023

Two derivations of Principal Component Analysis on datasets of distributions
Vlad Niculae
Data Set Principal Component Analysis Covariance Conditioning Mathematical Derivation Probabilistic Principal Component Analysis

June 17, 2023

Neural Fast Full-Rank Spatial Covariance Analysis for Blind Source Separation
Yoshiaki Bando, Yoshiki Masuyama, Aditya Arie Nugraha, Kazuyoshi Yoshii
Source Separation Covariance Conditioning Blind Source Separation Separation Model Mixture Signal

May 30, 2023

Probabilistic computation and uncertainty quantification with emerging covariance
Hengyuan Ma, Yang Qi, Li Zhang, Wenlian Lu, Jianfeng Feng
Uncertainty Quantification Bayesian Perspective Covariance Conditioning Bayesian Computation Stochastic Network Probabilistic Representation

Covariance Conditioning

Papers

Joint Estimation of Conditional Mean and Covariance for Unbalanced Panels

WeSpeR: Population spectrum retrieval and spectral density estimation of weighted sample covariance

Generalization for Least Squares Regression With Simple Spiked Covariances

Free Hunch: Denoiser Covariance Estimation for Diffusion Models Without Extra Costs

Identification of Non-causal Graphical Models

Simple Relative Deviation Bounds for Covariance and Gram Matrices

Induced Covariance for Causal Discovery in Linear Sparse Structures

Task-recency bias strikes back: Adapting covariances in Exemplar-Free Class Incremental Learning

Improving Probabilistic Diffusion Models With Optimal Covariance Matching

Iterative Methods for Full-Scale Gaussian Process Approximations for Large Spatial Data

Conditioning of Banach Space Valued Gaussian Random Variables: An Approximation Approach Based on Martingales

Self-Supervised Learning for Covariance Estimation

Synergistic eigenanalysis of covariance and Hessian matrices for enhanced binary classification

Improving Diffusion Models for Inverse Problems Using Optimal Posterior Covariance

Multivariate Probabilistic Time Series Forecasting with Correlated Errors

TIC-TAC: A Framework for Improved Covariance Estimation in Deep Heteroscedastic Regression

Hitting the High-Dimensional Notes: An ODE for SGD learning dynamics on GLMs and multi-index models

Two derivations of Principal Component Analysis on datasets of distributions

Neural Fast Full-Rank Spatial Covariance Analysis for Blind Source Separation

Probabilistic computation and uncertainty quantification with emerging covariance