libhmm: A Modern C++20 Library for Hidden Markov Models with Correct MLE Emission M-Steps

Gary Wolfman

cs.MSMay 28, 2026

libhmm: A Modern C++20 Library for Hidden Markov Models with Correct MLE Emission M-Steps

Authors: Gary Wolfman

Organizations: Independent Researcher

Abstract

We describe libhmm, a C++20 library for Hidden Markov Model parameter estimation, sequence decoding, and model selection. libhmm addresses two gaps in existing software: the absence of a well-maintained, zero-dependency C++ HMM library suitable for embedding in production systems, and the widespread use of method-of-moments (MOM) approximations in the emission distribution M-step of the Baum-Welch algorithm. The library implements correct maximum likelihood estimators for sixteen scalar emission distributions, including an ECME algorithm for the location-scale Student-t distribution, Newton-Raphson maximization for Gamma, Beta, Weibull, and Negative Binomial distributions, and the von Mises distribution for circular data. All forward-backward and Viterbi calculations operate in full log-space. SIMD acceleration is provided for AVX-512, AVX2, SSE2, and ARM NEON via compile-time dispatch with scalar fallback. Version 4 adds multivariate observation support via the BasicHmm<Obs> template, with three multivariate emission families (diagonal Gaussian, full-covariance Gaussian, and independent components) each with correct weighted MLE M-steps. Python bindings are available via the companion package pylibhmm. We compare libhmm against established C and C++ HMM libraries and against published R reference packages on seven real-data benchmarks, and discuss the architectural tradeoffs made in the design.

Explore similar work

Sep 15, 2026cs.LG

High-Performance Tensor Formulation of the Viterbi Algorithm for Hidden Semi-Markov Models

Hidden Semi-Markov Models (HSMMs) are fundamental probabilistic models widely adopted across diverse domains, from computational biology to finance and signal processing. The Viterbi algorithm decodes the most likely state sequence given an HSMM and can be applied iteratively for ab initio model learning. However, existing Viterbi implementations remain sequential, and GPU-accelerated solutions are entirely absent, making HSMM decoding impractical for large-scale workloads. We present a tensor-based formulation of the Viterbi algorithm for HSMMs, restructuring the inner loops into tensor operations that naturally map onto SIMD units and massively parallel architectures. Building on this formulation, we provide optimized implementations spanning single- and multi-core CPUs, and, for the first time, GPU. Experimental evaluation demonstrates speedups of up to 14x on a single core, over 200x with multi-core, and over 570x on GPU over the state-of-the-art sequential baseline, establishing a new performance baseline for large-scale HSMM decoding.

Lorenzo Piarulli, Elia Belli, Daniele De Sensi

Hidden Markov Models Tensor Programs Expectation-Maximization Tensor

Ask PDF

Sep 16, 2024stat.ML

The EM-algorithm and the Method of Moments in Softmax Mixture Models

Softmax Mixture Models (SMMs) are discrete

K

-component mixture models for the probabilities of selecting one of

p

candidate feature vectors

X_1,\ldots,X_p\in\mathbb{R}^L

in heterogeneous populations and are widely used in econometrics and scientific applications. Related softmax mixture mechanisms also appear in modern LLM architectures. We provide a theoretical and methodological study of SMMs, focusing on the Expectation-Maximization (EM) algorithm and the Method of Moments (MoM). We show that EM recovers the mixture atoms at the parametric rate, up to logarithmic factors, after

\mathcal{O}(\log N)

iterations, provided atom separation is at least of order

\log K

. This improves on separation conditions in existing analyses of EM for high-dimensional Gaussian mixtures. We also develop MoM procedures for parameter and subspace estimation. Although MoM parameter estimates converge more slowly than EM and can deteriorate with

K

, they provide provable warm starts for EM and are useful for small

K

. For general

K

, we estimate the atom subspace via MoM and recommend running EM from multiple random initializations within this subspace. Finally, as

p\to\infty

, we show that SMMs approximate mixtures of exponential tilts of the feature distribution, yielding asymptotic identifiability.

Xin Bing, Florentina Bunea, Jonathan Niles-Weed +1

Mixture Models Expectation-Maximization Diffie-Hellman

Ask PDF

May 20, 2026cs.LG

Symbolic Density Estimation for Discrete Distributions

Discrete probability laws underpin statistical modeling, yet the catalog of interpretable distributions has expanded only gradually through centuries of case-by-case mathematical derivations. We introduce symbolic density estimation (SDE), an unsupervised framework that automatically recovers closed-form probability mass functions by composing elementary analytic operations within a structured search space. Our method integrates domain-specific structural priors with evolutionary search and a validity-aware inference stage, and it extends to richer distribution families such as zero inflation and finite mixtures. To support systematic evaluation and future research, we contribute a benchmark dataset spanning a broad collection of commonly used discrete distributions. The proposed algorithm recovers all benchmark families with accurate parameter estimates. A real data application shows that it identifies concise and interpretable mixture models that improve goodness-of-fit over standard models.

Ziwen Liu, Meng Li

Density Ratio Estimation

Ask PDF