eess.SPDec 19, 2025

Alternating Direction Method of Multipliers for Nonlinear Matrix Decompositions

Authors: Atharva Awari, Nicolas Gillis, Arnaud Vandaele

Organizations: University of Mons

Abstract

We present an algorithm based on the alternating direction method of multipliers (ADMM) for solving nonlinear matrix decompositions (NMD). Given an input matrix X∈Rm×nX \in \mathbb{R}^{m \times n} and a factorization rank r≪min⁡(m,n)r \ll \min(m, n), NMD seeks matrices W∈Rm×rW \in \mathbb{R}^{m \times r} and H∈Rr×nH \in \mathbb{R}^{r \times n} such that X≈f(WH)X \approx f(WH), where ff is an element-wise nonlinear function. We evaluate our method on several representative nonlinear models: the rectified linear unit activation f(x)=max⁡(0,x)f(x) = \max(0, x), suitable for nonnegative sparse data approximation, the component-wise square f(x)=x2f(x) = x^2, applicable to probabilistic circuit representation, and the MinMax transform f(x)=min⁡(b,max⁡(a,x))f(x) = \min(b, \max(a, x)), relevant for recommender systems. The proposed framework flexibly supports diverse loss functions, including least squares, ℓ1\ell_1 norm, and the Kullback-Leibler divergence, and can be readily extended to other nonlinearities and metrics. We illustrate the applicability, efficiency, and adaptability of the approach on real-world datasets, highlighting its potential for a broad range of applications.

Explore similar work

CardsList
  1. Nonnegative Matrix Factorization in the Component-Wise L1 Norm for Sparse Data

    Mar 31, 2026Giovanni Seraghiti, Kévin Dubrulle, Arnaud Vandaele +1Non-Negative Matrix FactorizationMatrix Factorization

  2. Separable Nonnegative Matrix Factorization Using Powered Ratio-of-Norms Regularization

    Aug 28, 2026Matthew McCarver, Jing QinNon-Negative Matrix FactorizationNonconvex Optimization

  3. Over-fitting Control in Deep Nonlinear Matrix Completion via Gradual Nonsmooth Regularization

    Mar 15, 2024Sajad Faramarzi, Farzan Haddadi, Sajjad Amini +2Matrix CompletionLatent-Space Regularization