Alternating Direction Method of Multipliers for Nonlinear Matrix Decompositions
Organizations: University of Mons
Abstract
We present an algorithm based on the alternating direction method of multipliers (ADMM) for solving nonlinear matrix decompositions (NMD). Given an input matrix and a factorization rank , NMD seeks matrices and such that , where is an element-wise nonlinear function. We evaluate our method on several representative nonlinear models: the rectified linear unit activation , suitable for nonnegative sparse data approximation, the component-wise square , applicable to probabilistic circuit representation, and the MinMax transform , relevant for recommender systems. The proposed framework flexibly supports diverse loss functions, including least squares, norm, and the Kullback-Leibler divergence, and can be readily extended to other nonlinearities and metrics. We illustrate the applicability, efficiency, and adaptability of the approach on real-world datasets, highlighting its potential for a broad range of applications.
Matrix Factorization
Matrix factorization represents data or model weights as products of lower-dimensional matrices for compression, prediction, and latent-structure discovery.
See all 60 papers