FastManly: An EM-Gradient Algorithm for Manly Mixture Models

Katharine M. Clark; Paul D. McNicholas

stat.MLOct 1, 2024

FastManly: An EM-Gradient Algorithm for Manly Mixture Models

Authors: Katharine M. Clark, Paul D. McNicholas

Organizations: Department of Mathematics & Statistics, Trent University, Ontario, Canada. · Department of Mathematics & Statistics, McMaster University, Ontario, Canada.

Abstract

A faster implementation of mixtures of Manly transformations is proposed. This method, called FastManly, uses Newton's method for optimization in an EM gradient algorithm instead of Nelder-Mead in a traditional EM. A gradient and full Hessian are derived. Simulations show improved performance with noticeable speedups.

Figures & tables

	$p$
$n$	3	6	12	24	48
300	0.02 (-3.78) [23.04]	0.00 (-0.69) [8.72]	-0.00 (-0.81) [212.35]	0.13 (-2.52) [323.00]	6.28 (-1.35) [2993.53]
600	0.09 (-0.13) [28.07]	0.00 (-0.36) [144.84]	0.00 (-0.23) [1.56]	0.00 (-0.65) [22.55]	0.03 (-1.46) [52.71]
1200	0.03 (-4.33) [10.14]	0.00 (-0.01) [20.16]	0.00 (-0.16) [0.09]	0.00 (-0.31) [374.30]	0.04 (-0.51) [35.41]
2400	0.12 (-0.84) [34.50]	0.01 (0.00) [632.12]	0.00 (-0.01) [0.02]	0.00 (-0.01) [13.11]	0.13 (-0.00) [143.51]
4800	0.16 (-8.67) [161.79]	0.01 (0.00) [1326.01]	0.00 (0.00) [0.01]	0.00 (-0.01) [0.05]	0.23 (0.01) [194.33]

Table 1 : Summary of median, (minimum), and [maximum] differences between FastManly’s and ManlyMix’s log-likelihoods for each scenario. Differences are taken FastManly–ManlyMix.

Explore similar work

Local linear convergence of gradient methods for overparameterized Gaussian mixtures
May 29, 2026Jingxing Wang, Vasileios Charisopoulos, Maryam FazelGaussian Mixture Models Overparameterization
Electrical & Computer Engineering, University of Washington, Seattle, WA · National Institute for Theory and Mathematics in Biology, Chicago, IL · Amazon, Inc.
Differentiable Expectation-Maximisation and Applications to Gaussian Mixture Model Optimal Transport
Sep 2, 2025Samuel Boïté, Eloi Tanguy, Julie Delon +2Expectation-Maximization Wasserstein Distance
Université Paris Cité, CNRS, MAP5, F-75006 Paris, France · Centre Borelli, CNRS and ENS Paris-Saclay, F-91190 Gif-sur-Yvette, France · CMAP, CNRS, Ecole Polytechnique, Institut Polytechnique de Paris
FastMix: Fast Data Mixture Optimization via Gradient Descent
Jun 12, 2026Haoru Tan, Sitong Wu, Yanfeng Chen +5Pretraining Efficient Sequence Mixers
University of Hong Kong · 2Hunyuan LLM, Tencent · 3Chinese University of Hong Kong

May 29, 2026, cs.LGJ/K move · Enter open · S save

Local linear convergence of gradient methods for overparameterized Gaussian mixtures

Jingxing Wang, Vasileios Charisopoulos, Maryam Fazel

Electrical & Computer Engineering, University of Washington, Seattle, WA · National Institute for Theory and Mathematics in Biology, Chicago, IL · Amazon, Inc.

Gaussian Mixture Models Overparameterization Gradient Descent Expectation-Maximization Convergence Gradient Component+1

Open paper AskPDF arXiv

We study the problem of learning Gaussian mixture models under overparameterization. Prior work has shown that while overparameterization is essential for avoiding spurious local optima and enables global recovery of the ground-truth model using the gradient-EM (expectation-maximization) algorithm, it can dramatically slow down the local rate of convergence. Under certain assumptions on the mixture weights, we show that a standard divergence measure minimized by statistical learning procedures possesses a manifold of slow growth on which the well-known Polyak stepsize reduces the loss geometrically, and design a gradient-based method that converges to minimizers at a locally linear rate. Additionally, we show that our method converges to nearly optimal solutions -- up to a natural misspecification threshold -- for mixtures with arbitrary weights. At a high level, the method alternates between several "short" gradient descent steps that approach the manifold and "long" Polyak steps that contract the distance to minimizers. Our results suggest that slow convergence is not an intrinsic challenge of overparameterization, but can be overcome by exploiting the favorable structure of the loss landscape.

Abstract

Explore similar work

Local linear convergence of gradient methods for overparameterized Gaussian mixtures

Differentiable Expectation-Maximisation and Applications to Gaussian Mixture Model Optimal Transport

FastMix: Fast Data Mixture Optimization via Gradient Descent