MM-algorithms for traditional and convex NMF with Tweedie and Negative Binomial cost functions and empirical evaluation

Elisabeth Sommer James; Asger Hobolth; Marta Pelizzola

cs.LGMar 10, 2026

MM-algorithms for traditional and convex NMF with Tweedie and Negative Binomial cost functions and empirical evaluation

Authors: Elisabeth Sommer James, Asger Hobolth, Marta Pelizzola

Organizations: Department of Mathematics, Aarhus University, Denmark

Abstract

Non-negative matrix factorisation (NMF) is a widely used tool for unsupervised learning and feature extraction, with applications ranging from genomics to text analysis and signal processing. Standard formulations of NMF are typically derived under Gaussian or Poisson noise assumptions, which may be inadequate for data exhibiting overdispersion or other complex mean-variance relationships. In this paper, we develop a unified framework for both traditional and convex NMF under a broad class of distributional assumptions, including Negative Binomial and Tweedie models, where the connection between the Tweedie and the $β$ -divergence is also highlighted. Using a Majorize-Minimisation approach, we derive multiplicative update rules for all considered models, and novel updates for convex NMF with Poisson and Negative Binomial cost functions. We provide a unified implementation of all considered models, including the first implementations of several convex NMF models. Empirical evaluations on mutational and word count data demonstrate that the choice of noise model critically affects model fit and feature recovery, and that convex NMF can provide an efficient and robust alternative to traditional NMF in scenarios where the number of classes is large. The code for our proposed updates is available in the R package nmfgenr and can be found at https://github.com/MartaPelizzola/nmfgenr.

Figures & tables

Distribution	Traditional		Convex
Distribution	Algorithm	Implementation	Algorithm	Implementation
Normal	✓ 1	✓ 4	✓ 5	☑
Poisson	✓ 1	✓ 4	✓ 2	☑
Tweedie	✓ 2	✓ 2	✓ 2	☑
Negative Binomial	✓ 3	✓ 3	☑	☑

Table 1: Overview of traditional and convex non-negative matrix factorisation models under different distributional assumptions. For each distribution, existing estimation procedure and implementation are indicated with ✓; new contributions introduced in this paper are indicated with ☑ . References for existing algorithms and implementation are provided as footnotes.

Symbol	Distribution
$\mathcal{N}$	Normal distribution
Po	Poisson distribution
TW p	Tweedie distribution with power $p$
NB α	Negative Binomial distribution with dispersion $\alpha$

Table 2: Abbreviations for distributions specifying the cost functions.

Figure 1: Unit deviances and the corresponding Tweedie distributions with parameters

(\mu,\sigma^{2})=(50,1)

for

p\in\{0,1\}

p\in\{1.5,2\}

, and

p\in\{2.5,3\}

	Normal distribution	Poisson distribution

Update	NMF/ $\mathcal{T}$ / $\mathcal{N}$	NMF/ $\mathcal{T}$ /Po
$H^{(t+1)}$	$H^{t}\cdot\left(\frac{W^{T}V}{W^{T}WH}\right)$	$H^{t}\cdot\left(\frac{W^{T}\left[\frac{V}{WH}\right]}{W^{T}\ J_{N,M}}\right)$
$W^{(t+1)}$	$W^{t}\cdot\left(\frac{VH^{T}}{WHH^{T}}\right)$	$W^{t}\cdot\left(\frac{\left[\frac{V}{(WH)}\right]H^{T}}{J_{N,M}\ H^{T}}\right)$

	NMF/ $\mathcal{C}$ / $\mathcal{N}$	NMF/ $\mathcal{C}$ /Po

Table 4: Updates for NMF and convex NMF with the Normal and Poisson distributions.

J_{M,N}

denotes an

M\times N

matrix of ones.

Figure 2: Run time analysis per iteration. The run time in seconds is plotted against the number of data points

N

for the different methods and distributional assumptions and number of dimensions M. Results are averaged over 100 iterations. The rank of the factorisation is K = 5 as estimated in Section 3.1 , and the values of the dispersion parameter

\alpha

are as given in 4 , being 45.21 and 36.44 for traditional and convex NMF respectively.

Figure 3: Comparison of BIC values and mean-variance relationships across the 8 NMF models, applied to the mutational count data of liver cancer patients.

Figure 4: Residual analysis of NMF/ $\mathcal{T}$ (on the left) and NMF/ $\mathcal{C}$ (on the right) under the Normal, Poisson, Tweedie, and Negative Binomial distributions applied to the mutational count data of liver cancer patients. In each panel, the coloured lines are 2 standard deviations of the assumed model.

Figure 5: Quality of estimated signatures for mutational count data of liver cancer patients. Each method is compared to NMF/ $\mathcal{T}$ /NB 45.21 /5 (left) and to the COSMIC signatures (right). We show the quality of the estimated signatures measured by cosine similarity for all methods.

Figure 6: Comparison of BIC values and mean-variance relationships across the 8 NMF models on the word count data of 500 documents.

Figure 7: Residual analysis of NMF/ $\mathcal{T}$ (on the left) and NMF/ $\mathcal{C}$ (on the right) under the Normal, Poisson, Tweedie, and Negative Binomial distributions applied to the word count data of 500 documents. In each panel, the coloured lines denote twice the standard deviation of the assumed model. The residual plots are downsampled from 3177000 to 100000 points.

Figure 8: Quality of estimated features for text count data of newsgroups posts. Each method is compared to NMF/ $\mathcal{C}$ /TW 1.026 /7. We show the quality of the estimated features measured by cosine similarity for all methods.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Fitting of power parameter for datasets of newsgroups word counts and mutational counts from liver cancer patients.

Figure 10: Profile likelihood function of

\alpha

applied to datasets of mutational counts from liver cancer patients and newsgroups word counts.

Figure 11: Top words for the topical domains estimated by NMF/ $\mathcal{C}$ /TW 1.02 /7 and their corresponding weight in the respective topic.

Figure 12: Weights (D matrix) for NMF/ $\mathcal{C}$ /TW 1.02 /7. The weight of all topics for each document are represented by the different colours of the horizontal bars. Each bar correspond to a document. Documents are ordered by their original labels.

Non-Negative Matrix Factorization

Non-negative matrix factorization (NMF) decomposes nonnegative data into additive, interpretable latent components for analysis and discovery.

See all 33 papers

An Efficient Newton Algorithm for Nonnegative Matrix Factorization with the Kullback-Leibler Divergence
Jul 15, 2026Damien Lesens, Jérémy E. Cohen, Bora UçarNon-Negative Matrix Factorization Low-Rank Matrix Decomposition
ENS de Lyon · Univ. Lyon, INSA-Lyon, UCBL, UJM, CNRS, Inserm, CREATIS UMR 5220, U1294, F-69621, Lyon, France. · CNRS and LIP (UMR5668, Université de Lyon - ENS de Lyon - UCBL - CNRS - Inria), 46 allée d'Italie, Lyon, France.
A Provably-Correct and Robust Convex Model for Smooth Separable NMF
Nov 10, 2025Junjun Pan, Valentin Leplat, Michael Ng +1Non-Negative Matrix Factorization Hyperspectral Unmixing
Department of Mathematics, Hong Kong Baptist University, Hong Kong. · Institute of Data Science and Artificial Intelligence, Faculty of Computer Science and Engineering, Innopolis University. · Department of Mathematics and Operational Research, Université de Mons, Rue de Houdain 9, 7000 Mons, Belgium.
A New Family of Poisson Non-negative Matrix Factorization Methods Using the Shifted Log Link
Jan 9, 2026Eric Weine, Peter Carbonetto, Rafael A. Irizarry +1Non-Negative Matrix Factorization Generalized Linear Models
Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA · Department of Human Genetics, University of Chicago, Chicago, IL, USA · Department of Data Science, Dana-Farber Cancer Institute, Boston, MA, USA +1

Jul 15, 2026J/K move · Enter open · S save

An Efficient Newton Algorithm for Nonnegative Matrix Factorization with the Kullback-Leibler Divergence

Damien Lesens, Jérémy E. Cohen, Bora Uçar

ENS de Lyon · Univ. Lyon, INSA-Lyon, UCBL, UJM, CNRS, Inserm, CREATIS UMR 5220, U1294, F-69621, Lyon, France. · CNRS and LIP (UMR5668, Université de Lyon - ENS de Lyon - UCBL - CNRS - Inria), 46 allée d'Italie, Lyon, France.

Non-Negative Matrix Factorization Low-Rank Matrix Decomposition Second-Order Optimization

Open paper AskPDF arXiv

Nonnegative Matrix Factorization (NMF) is a fundamental tool in unsupervised learning, which approximates a nonnegative matrix by the product of two low-rank nonnegative factors. The Kullback-Leibler (KL) divergence is best suited to measure the data to model discrepancy when the decomposed data sample follows a Poisson distribution, which is the case for count datasets such as term-document matrices or images. Most KL-NMF algorithms in the literature minimize a separable majorant of the loss to find their next iterate. We argue that this method has reached its limits and propose to use instead the second-order Taylor expansion of the loss, leading to a Newton-type method. We minimize this non-separable surrogate by proposing a generalization of the well-known HALS algorithm. This yields an efficient KL-NMF algorithm which provably converges and which competes favorably with state-of-the-art algorithms on a large variety of datasets.

Abstract

Non-Negative Matrix Factorization

Explore similar work

An Efficient Newton Algorithm for Nonnegative Matrix Factorization with the Kullback-Leibler Divergence

A Provably-Correct and Robust Convex Model for Smooth Separable NMF

A New Family of Poisson Non-negative Matrix Factorization Methods Using the Shifted Log Link