Mean Embeddings

Mean embeddings represent probability distributions as vectors in a reproducing kernel Hilbert space, enabling the comparison and manipulation of distributions using linear algebra techniques. Current research focuses on improving the scalability and expressiveness of these embeddings, particularly through neural network integration and the development of robust estimators, often within the context of conditional distributions. This approach finds applications in diverse fields, including causal inference, reinforcement learning, and safety verification, by providing efficient methods for handling uncertainty and learning from limited data. The resulting advancements offer improved accuracy and scalability for various machine learning tasks involving probabilistic modeling.

Papers

September 17, 2024

Norm of Mean Contextualized Embeddings Determines their Variance
Hiroaki Yamagiwa, Hidetoshi Shimodaira
Jina Embeddings Layer Normalization Contextual Embeddings Token Embeddings Variance Information Rapid Norm Growth Mean Embeddings

March 16, 2024

Neural-Kernel Conditional Mean Embeddings
Eiki Shimizu, Kenji Fukumizu, Dino Sejdinovic
Distributional Reinforcement Learning Hyper Parameter Kernel Based Conditional Density Kernel Mean Embeddings Mean Embeddings

March 15, 2024

Data-Driven Distributionally Robust Safety Verification Using Barrier Certificates and Conditional Mean Embeddings
Oliver Schön, Zhengang Zhong, Sadegh Soudjani
Gaussian Process Provable Guarantee Efficient Verification Transition Kernel Barrier Certificate Mean Embeddings

May 14, 2023

Conditional mean embeddings and optimal feature selection via positive definite kernels
Palle E. T. Jorgensen, Myung-Sin Song, James Tian
Feature Selection Positive Definite Kernel Mean Embeddings

December 9, 2022

Doubly Robust Kernel Statistics for Testing Distributional Treatment Effects
Jake Fawkes, Robert Hu, Robin J. Evans, Dino Sejdinovic
Causal Inference Treatment Effect Doubly Robust Mean Embeddings

November 2, 2022

Bayesian Counterfactual Mean Embeddings and Off-Policy Evaluation
Diego Martinez-Taboada, Dino Sejdinovic
Policy Evaluation Counterfactual Estimation Counterfactual Distribution Mean Embeddings

August 2, 2022

Optimal Rates for Regularized Conditional Mean Embedding Learning
Zhu Li, Dimitri Meunier, Mattes Mollenhauer, Arthur Gretton
LeArning Abstract Learning Rate Kernel Hilbert Space Kernel Ridge Regression Optimal Rate Mean Embeddings Hilbert Schmidt Conditional Mean