Hierarchical Mixture

Hierarchical mixtures model complex data by combining simpler models in a layered structure, aiming to improve accuracy and efficiency in tasks like classification and anomaly detection. Current research emphasizes developing sophisticated gating mechanisms beyond softmax, exploring diverse expert model types (e.g., Gaussian processes, Generalized Dirichlet classifiers, normalizing flows), and employing variational inference for efficient learning. These advancements enhance the representation power and scalability of hierarchical mixtures, leading to improved performance in various applications including robotics, image processing, and natural language processing.

Papers

October 29, 2024

Hierarchical mixtures of Unigram models for short text clustering: the role of Beta-Liouville priors
Massimo Bilancia, Samuele Magro
Integral Role Hyper Parameter Prior Distribution Multinomial Logistic Beta Distribution Gaussian Prior Hierarchical Mixture Multinomial Mixture Short Text Clustering

October 25, 2024

October 3, 2024

On Expert Estimation in Hierarchical Mixture of Experts: Beyond Softmax Gating Functions
Huy Nguyen, Xing Han, Carl William Harris, Suchi Saria, Nhat Ho
Foundation Model Estimation Task Expert Knowledge Softmax Gating Hierarchical Mixture

May 2, 2024

Hierarchical mixture of discriminative Generalized Dirichlet classifiers
Elvis Togban, Djemel Ziou
Compositional Data Hierarchical Mixture Dirichlet Process Mixture Model Ordered Matrix Dirichlet Discriminative Classifier

March 20, 2024

Hierarchical Gaussian Mixture Normalizing Flow Modeling for Unified Anomaly Detection
Xincheng Yao, Ruoqi Li, Zefeng Qian, Lu Wang, Chongyang Zhang
Anomaly Detection Hierarchical Mixture

February 23, 2024

Co-Supervised Learning: Improving Weak-to-Strong Generalization with Hierarchical Mixture of Experts
Yuejiang Liu, Alexandre Alahi
Expert Knowledge Student Model Level Supervision Weak to Strong Co Training Strong Model Co Supervision Hierarchical Mixture

October 25, 2023

Probabilistic Integral Circuits
Gennaro Gala, Cassio de Campos, Robert Peharz, Antonio Vergari, Erik Quaeghebeur
Latent Variable Probabilistic Circuit Continuous Model Hierarchical Mixture

February 9, 2023

Gaussian Process-Gated Hierarchical Mixtures of Experts
Yuhao Liu, Marzieh Ajirak, Petar Djuric
Gaussian Process Variational Inference Expert Knowledge RSD Difference of Gaussian Gating Mechanism Hierarchical Mixture

November 2, 2022

Variational Hierarchical Mixtures for Probabilistic Learning of Inverse Dynamics
Hany Abdulsamad, Peter Nickl, Pascal Klink, Jan Peters
Variational Inference Local Regression Probabilistic Learning Bayesian Kernel Probabilistic Regression Hierarchical Mixture

June 10, 2022

Hierarchical mixtures of Gaussians for combined dimensionality reduction and clustering
Sacha Sokoloski, Philipp Berens
Dimensionality Reduction Data Clustering RSD Difference of Gaussian High Dimensional Data Hierarchical Mixture

December 9, 2021

Learning Transferable Motor Skills with Hierarchical Latent Mixture Policies
Dushyant Rao, Fereshteh Sadeghi, Leonard Hasenclever, Markus Wulfmeier, Martina Zambelli, Giulia Vezzani, Dhruva Tirumala, Yusuf Aytar, Josh Merel, Nicolas Heess, Raia Hadsell
Latent Variable Based Imitation Motor Skill Hierarchical Mixture

Hierarchical Mixture

Papers

Hierarchical mixtures of Unigram models for short text clustering: the role of Beta-Liouville priors

Unified Cross-Modal Image Synthesis with Hierarchical Mixture of Product-of-Experts

Hierarchical Mixture of Experts: Generalizable Learning for High-Level Synthesis

On Expert Estimation in Hierarchical Mixture of Experts: Beyond Softmax Gating Functions

Hierarchical mixture of discriminative Generalized Dirichlet classifiers

Hierarchical Gaussian Mixture Normalizing Flow Modeling for Unified Anomaly Detection

Co-Supervised Learning: Improving Weak-to-Strong Generalization with Hierarchical Mixture of Experts

Probabilistic Integral Circuits

Gaussian Process-Gated Hierarchical Mixtures of Experts

Variational Hierarchical Mixtures for Probabilistic Learning of Inverse Dynamics

Hierarchical mixtures of Gaussians for combined dimensionality reduction and clustering

Learning Transferable Motor Skills with Hierarchical Latent Mixture Policies