Tensor Decomposition

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

Jul 6Week of Sep 21

Latest papers 32

Sep 30, 2026stat.ME

Heteroskedastic Canonical Polyadic Tensor Decomposition

When minimizing the squared-error loss, the popular CP decomposition can be interpreted as parameter inference in a Gaussian model with a low-rank mean tensor and constant variance across the tensor entries. We introduce heteroskedastic-CP (HCP), which models entrywise variability with a non-constant, low-rank precision tensor, and develop an alternating block-coordinate ascent method to recover both the low-rank mean and precision tensors from noisy observations. Our procedure is computationally competitive, with the same leading-order factor-update complexity as CP-ALS. We demonstrate HCP on synthetic experiments and an EEG application.

Kyle Ritscher, Carlos Llosa-Vite

Computer Science Research Institute, Sandia National Laboratories, Albuquerque, NM · Department of Statistics & Data Science, University of California, Los Angeles, CA

Tensor Decomposition Low-Rank Structure Decomposition Online Conformal Prediction

Sep 23, 2026math.NA

Tensor Decomposition of Transformer Key-Value Caches: Spectral Structure and Format Comparison

The key-value (KV) cache of autoregressive transformers can be viewed as a fourth-order tensor spanning attention heads, tokens, features, and grouped layers. We measure the singular-value spectra of all four mode unfoldings on Mistral-7B-v0.3 and LLaMA-2-13B and compare four standard tensor decompositions: Tucker, CP, tensor train, and t-SVD, at matched storage. The spectra partition the four axes into two classes. The token and feature modes carry low-rank structure, particularly for keys. The head and layer modes are nearly full-rank and resist compression at any practical error level. Among the four decompositions, Tucker achieves the lowest reconstruction error at every compression ratio from

2\times

to

5\times

, because it can leave the full-rank modes untouched. Comparisons with two-dimensional unfolding baselines show that the preferred representation differs between keys and values: 2D methods achieve lower key error, while four-way Tucker achieves lower value error at matched storage. A mode-pinning theorem certifies the full-rank preservation from the measured spectra alone. Two further spectral properties affect the compressible modes without touching the full-rank ones: values reach a higher error floor than keys at every ratio, and post-RoPE keys lose

41\%

-

64\%

of their pre-RoPE compressibility on both models.

Rahul Krishnan, Volker Schulz

Universität Trier · Fachbereich IV, Mathematik, Universität Trier

Key-Value Cache Compression Tensor Decomposition Autoregressive Transformers Transformer Architectures Spectral Comparison

Sep 21, 2026cs.LG

Topological Signal Processing With Unoriented Operators

Topological signal processing (TSP) processes signals on simplicial complexes with oriented boundary operators, which is the natural choice for flow signals or when the topological invariants play a role for the task at hand. However, many higher-order signals carry no orientation, and applying oriented operators to them is not well-defined since it introduces an arbitrary choice of simplex orientation. We study an unoriented TSP (UTSP) framework that replaces oriented boundaries with unoriented incidence matrices. First, we show that unoriented incidence and Laplacian matrices between arbitrary simplicial levels admit graph-like spectral properties. Second, since dropping orientation removes the Hodge decomposition, we introduce an unoriented counterpart, termed interaction-order decomposition, which quantifies how much of a higher-order signal is explained by aggregating lower-order signals. Third, we use this decomposition to derive regularizers for signal reconstruction that penalize each interaction order separately. Experiments on real-world data show that the order-aware regularizers outperform oriented baselines, with the largest gains when the signal energy is unevenly distributed across orders.

Andrea Cavallo, Varun Sarathchandran, Geert Leus +1

Delft University of Technology, Delft, Netherlands

Graph Signal Processing Tensor Decomposition Topology Regularization Logical Connectives

Sep 14, 2026cs.CV

Pre-Trained Low-Rank Tensor Decomposition for Multi-Dimensional Image Recovery

Recently, tensor decompositions are prevalent for multi-dimensional image representation, which learn the instance-specific structure of each image from scratch. However, tensor decompositions neglect the common structure across different images, leading to limited semantic modeling capability, high computational cost, and a large number of learnable parameters. To address this challenge, we suggest the first pre-trained low-rank tensor decomposition (PLTD) framework, which organically integrates the pre-trained large vision model into the classical tensor decomposition framework. Beyond the shallow and untrained deep tensor decomposition, the suggested PLTD achieves an unprecedented balance among higher recovery fidelity, fewer learnable parameters, and smaller carbon footprint. Specifically, PLTD factorizes the target tensor into a latent tensor and a learnable transform that maps the latent tensor back to the original data domain. The latent tensor consists of two indispensable and complementary terms, i.e., a fixed pre-trained latent tensor and a learnable low-rank latent tensor. The fixed pre-trained latent tensor is distilled from a pre-trained large vision model (i.e., DINOv3) to capture the common structure of the target tensor, while the learnable low-rank latent tensor characterizes the instance-specific structure of the target tensor. To examine the potential of PLTD, we develop the corresponding multi-dimensional image recovery model and theoretically justify the advantages of this framework. Additionally, we discuss the connections between PLTD and classical tensor decomposition frameworks. Extensive experiments on multi-dimensional image recovery demonstrate that PLTD consistently achieves superior performance compared with state-of-the-art methods.

Bing-Zhang Fu, Zhi-Long Han, Ting-Zhu Huang +2

School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu 611731, China · School of Mathematics and Statistics and Ministry of Education Key Lab of Intelligent Networks and Network Security, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China · Pazhou Laboratory (Huangpu), Guangzhou, Guangdong 510335, China

Tensor Decomposition Low-Rank Structure Image Restoration Multimodal Pretraining Self-Supervised Vision Transformers Pretraining+1

Sep 11, 2026cs.LG

Semi-Tensor Product-Based Multi-Term Randomized T-SVD and Its Visual Applications

Tensor singular value decomposition (T-SVD), which is built upon the tensor-tensor product (t-product), has emerged as a powerful tool for processing high-dimensional visual data such as color images and videos. However, the standard t-product imposes strict dimensional compatibility constraints. Although extensions based on the semi-tensor product (STP) relax this restriction, their single-term formulations still suffer from limited approximation accuracy. Moreover, these deterministic methods incur high computational costs when processing large-scale tensor data. To address these issues, this paper introduces a novel semi-tensor product for third-order tensors under the t-product framework induced by arbitrary invertible linear transforms. The resulting tensor semi-tensor product breaks the rigid dimension matching requirement of the standard t-product, while retaining the closed-form property of T-SVD. Based on this construction, we develop a multi-term semi-tensor product singular value decomposition (MSTP-SVD), which integrates multiple orthogonal decomposition terms to significantly improve low-rank approximation accuracy compared with single-term schemes. To reduce the computational cost of multi-term modeling, we incorporate randomized projection and power iteration techniques into the MSTP-SVD framework, yielding an accelerated multi-term randomized semi-tensor product SVD (MRSTP-SVD) algorithm that achieves a balance between reconstruction accuracy and computational efficiency. Experiments on image and video compression and completion tasks demonstrate the effectiveness of the proposed method.

Xingchen Xiao (School of Mathematics and Statistics, Southwest University, Chongqing +13

School of Mathematics and Statistics, Southwest University, Chongqing, 400715, China

Tensor Decomposition Singular Value Decomposition Video Compression

Sep 10, 2026stat.ML

Identifiability of Nonnegative Tensor Decompositions via Positive Scattering

Identifiability of tensor decompositions is often established through linear-algebraic conditions on the factor families. For nonnegative decompositions, however, positivity provides additional information that is not captured by dimension and independence alone: nonnegative terms cannot cancel, and their supports constrain competing decompositions. We introduce a positive scattering term that quantifies this additional source of identifiability and combine it with the dimension budget underlying the Lovitz--Petrov generalization of Kruskal's theorem. For every subset of components, we obtain two sufficient conditions: a threshold of

2|S|-2

guarantees minimality and nonnegative rank, while the stronger threshold

2|S|-1

guarantees uniqueness among nonnegative decompositions of the same length. The key result is a positive splitting inequality for irreducible exchanges of nonnegative rank-one tensors, which combines the dimension constraint with support-induced geometric rigidity. Although the scattering term is defined through an optimization over intermediate factor spaces, we show that its mode costs are exactly

0

,

1

, or

+\infty

, yielding an exact activation characterization in terms of graph connectivity. The resulting criterion can strictly certify sparse nonnegative tensor decompositions beyond the reach of Kruskal and Lovitz--Petrov conditions, including examples for which those conditions fail even after reshaping. In the matrix case, the two criteria reduce respectively to full-rank factorization and two-sided separability.

Haoming Wang, Ming Yuan

Columbia University

Tensor Decomposition Identifiability Decomposition Separability

Sep 8, 2026cs.LG

Constraint-Aware Discrete Black-Box Optimization Using Tensor Decomposition

Discrete black-box optimization is often addressed using approaches such as Sequential Model-Based Optimization (SMBO), which aims to improve sample efficiency by fitting surrogate models that approximate a costly objective function over a discrete search space. In many real-world problems, the set of feasible inputs is often given by logical constraints known in advance. However, existing surrogate modeling techniques generally fail to capture the symbolic rules governing feasibility in discrete input spaces. In this paper, we propose a surrogate modeling approach based on tensor decomposition that captures the structure of discrete search spaces while directly integrating feasibility information. To implement this approach, we formulate surrogate model training as a constrained polynomial optimization problem and solve a relaxed formulation using a differentiable penalty term derived from T-norms. Our experiments on both synthetic and real-world benchmarks, including a pressure vessel design task, demonstrate that the proposed method improves sample efficiency by effectively guiding the search away from infeasible regions.

Keisuke Onoue, Ryosuke Kojima

Nara Institute of Science and Technology, Nara, Japan · Graduate School of Medicine, Kyoto University, Kyoto, Japan · RIKEN Center for Biosystems Dynamics Research, Japan

Black-Box Optimization Tensor Decomposition

Sep 8, 2026cs.CV

Hyperspectral Anomaly Detection via Group Sparse Low-Rank Tensor Factorization With Automatic Anomaly Grouping

Low-rank tensor modeling has become an effective tool for hyperspectral anomaly detection. However, existing methods still suffer from high computational cost and limited flexibility in characterizing spatially structured anomalies. To address these issues, this paper proposes a hyperspectral anomaly detection method based on group sparse low-rank tensor factorization with automatic anomaly grouping (GSAA). Specifically, the low tubal rank background is characterized by imposing group sparsity on tensor factors, which provides an efficient alternative to direct tensor rank regularization. For anomaly modeling, a latent grouping map is introduced to build an automatic anomaly grouping penalty, allowing anomaly groups to be adaptively inferred from the data rather than predefined at the pixel level. To further exploit complementary spectral and spatial information, GSAA is applied in both domains, and the resulting detection maps are fused to form a spectral--spatial version of GSAA, termed GSAA-SS. An efficient linearized alternating direction method of multipliers algorithm with convergence guarantee is developed to solve the resulting model. Experimental results on five real hyperspectral datasets demonstrate that the proposed method achieves superior detection performance and competitive computational efficiency compared with several state-of-the-art methods.

Quan Yu, Yu-Hong Dai, Xiongjun Zhang

School of Mathematics and Statistics, Central China Normal University, Wuhan 430079, China, and also with the Key Laboratory of Nonlinear Analysis and Applications (Ministry of Education), Central China Normal University · State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China, and also with the School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China

Hyperspectral Image Graph Anomaly Detection Tensor Decomposition Low-Rank Structure Sparsity

Aug 31, 2026cs.LG

Tensor Methods for Language Models: From Token Representation to Training, Adaptation, Inference, Compression, and Interpretability

Large language models (LLMs) are built from structured high-dimensional objects such as token representations, weights, adaptation updates, caches, and activations, whose multilinear structure is underexploited by the conventional matrix-centric view. Tensor decompositions and tensor networks provide a principled algebraic language for this structure, yet the literature often treats them as isolated compression mechanisms. This survey organizes tensor methods for LLMs through two complementary views: a seven-stage lifecycle taxonomy covering tokenization, embeddings, pre-training, adaptation, compression, inference, and interpretability, and a component view covering embeddings, attention, and feed-forward networks. We provide unified notation and theoretical foundations, analyze tensorization strategies for individual Transformer components, and compare methods at each lifecycle stage while making differences in evaluation protocols and model scales explicit. We further connect tensor methods to neighboring efficiency techniques and probabilistic tensor networks. Finally, we synthesize open challenges and introduce

ρ_{\rm gap}

, a metric for the compression-realization gap between theoretical memory reduction and measured system-level speedup. By treating tensorization as a common structural principle, the survey provides a structured entry point to tensorized language models and clarifies when parameter savings can plausibly translate into memory efficiency, computational efficiency, or interpretability. The GitHub page dedicated to this paper is accessible at \href{https://github.com/ma-tt-a/awesome-tensor-methods-for-llms}{this https URL}.

Matvei Tarasov, Salman Ahmadi-Asl, Andre L. F. de Almeida +1

Innopolis University, Innopolis 420500, Russia · Department of Teleinformatics Engineering, Federal University of Cear´a, Fortaleza 60455-760, Brazil · Systems Research Institute of Polish Academy of Science and Warsaw University of Technology, Poland

Large Language Model Compression Tensor Networks Tensor Decomposition Data Compression Methods Interpretability Peak Memory

Aug 13, 2026cs.LG

Knowledge-guided Pattern Discovery via Coupled Tensor Factorizations

In order to understand complex systems such as the human metabolome or human brain, different sensing technologies are used, generating complex data. These datasets are often multiway, i.e., with more than two axes of variation such as a subjects by metabolites by time array. While tensor factorizations have successfully revealed interpretable patterns from such complex data, they have so far been mainly data-driven. On the other hand, there is more to data -- there are computational models (of these systems), which are rich sources of prior information. In this paper, we introduce a knowledge-guided approach that brings together data and computational models by jointly analyzing real data and simulated data (generated using a computational model) using coupled tensor factorizations with linear coupling. Our experiments on real metabolomics measurements demonstrate that guiding the analysis of such noisy data with simulated data improves the pattern discovery performance while also revealing potential discrepancies between data and computational models.

Gaute Johannessen, Geert Roelof van der Ploeg, Evrim Acar

Faculty of Mathematics and Natural Sciences, University of Oslo, Oslo · Department of Data Science and Knowledge Discovery, Simula Metropolitan Center for Digital Engineering, Oslo

Tensor Decomposition Metabolic Discovery

Aug 11, 2026cs.LG

Accelerated Learning of High Dimensional Functions with a Tensor-Featured Training Network

In this work we present a method to accelerate the optimization of learning high dimensional functions using deep neural network (DNN). This optimization procedure introduces contextual features into the first layer of a DNN. The parameters of DNN are optimized via standard gradient descent while keeping the input-feature basis fixed. After optimization of the DNN parameters, the feature layer is provided a chance to update and change before DNN optimization resumes. The feature layer has two types of functions: those that can be evaluated quickly in a matrix-free way on the domain (i.e. rank-1 features) and more complex features that must first be decomposed using tensor network (TN) decomposition strategies (tensor features). In particular, we study the effect of adding features which distill pretrained DNN into TNs using a discretize and decompose strategy. To efficiently decompose high-dimensional functions constructed from discretized DNN, we leverage a randomized tensor decomposition strategy. Using randomization, we are able to reduce the storage cost of decomposing high dimensional functions by at least 8 orders of magnitude. Using this approach, we are able to efficiently train models between 5 and 40 dimensions.

Karl Pierce, Yuehaw Khoo, Haizhao Yang

† The University of Maryland, College Park, College Park MD 20742 USA · ∗The University of Chicago, Chicago, IL 60637 USA

Tensor Decomposition High-Dimensional Tensor Networks Neural Network Optimization

Aug 4, 2026cs.LG

MINT: Tensor Decomposition on Stacked Recurrence Matrices for Time Series Data Mining

Recurrence plots are a time series data mining primitive applied to a variety of domains (e.g. star light curves, sound waveforms, CCT telemetry). This work proposes tensorized self-similarity matrices as a primitive for univariate time series datasets (

N\times n

) of

N

time series of length

n

with a subsequence window of length

m

, and whose tensor-based nature is naturally extensible to multivariate datasets. The proposed method to compute this primitive computes dot plots of size

N \times (n-m+1) \times (n-m+ 1)

from these datasets, where the subsequent tensor is mined using tensor decomposition methods to mine for co-clustered patterns. We demonstrate our results in mass rapid transit, electricity demand, wind turbine, and car traffic data, finding the MINT pipeline effectively co-clusters cross-sensor patterns in highly regular datasets containing motifs at regular intervals.

Kaamil Kaka, Audrey Der, Evangelos E. Papalexakis +2

University of California, Riverside, Department of Computer Science · Neuralix AI

Tensor Decomposition Time Series Recurrence Membership Inference

Aug 3, 2026cs.LG

Smooth Reparameterizations of Functions on Simplicial Product Spaces: Applications to Probabilistic Tensor Decomposition and Functional Data Registration

We consider optimization problems defined on product spaces of simplices. Examples of this class of problems include learning low-rank discrete multivariate probability distributions via simplex constrained tensor decomposition and performing functional data registration under the Square Root Velocity Function (SRVF) representation. In this work, we demonstrate the feasibility of replacing the product simplex with a smooth, elementwise strictly convex reparameterization, resulting in an unconstrained optimization problem on a manifold. We show that performing such a reparameterization results in the second order Karush-Kuhn-Tucker (KKT) points on the smooth manifold being mapped to the weak second order KKT points on the product simplex. This leads to a Riemannian Gradient Descent (RGD) algorithm for solving the reparameterized problem, which outperforms Projected Gradient Descent (PGD), and provides a more faithful representation of the original function shapes while performing curve registration.

Shashwat Kumar, Arafat Rahman, Anuj Srivastava +1

Department of Biomedical Engineering, Johns Hopkins University, USA · Department of Systems and Information Engineering, University of Virginia, USA · Department of Applied Mathematics and Statistics, Johns Hopkins University, USA +1

Reparameterization Simplex Image Registration Tensor Decomposition

Jul 28, 2026cs.LG

Breaking the Periodicity Assumption: Robust Tensorial Multi-View Clustering via Graph-Spectral Low-Rank Learning

Tensorial multi-view clustering (TMC) has achieved strong performance due to its ability to capture high-order correlations across multiple views. Most existing t-SVD-based TMC frameworks apply the Fast Fourier Transform (FFT) along the sample mode to impose frequency-domain low-rank constraints. However, we reveal that this widely adopted design critically relies on an implicit ``periodicity assumption'' induced by the sample arrangement. When samples are ordered by class, neighboring indices tend to be semantically similar, creating artificial local continuity along the sample mode and a favorable spectral structure for FFT-based low-rank regularization. Once this ordering is removed by random permutation, existing t-SVD-based TMC methods suffer severe performance degradation. This strong sensitivity to class ordering conflicts with the permutation-invariant nature of clustering and indicates that part of the reported performance may be attributed to a privileged sample arrangement rather than genuine high-order structure modeling. In this paper, we systematically investigate this phenomenon and its underlying algebraic and spectral mechanisms. To address this fundamental flaw, we further propose a graph-spectral low-rank tensor learning framework based on the Graph Fourier Transform (GFT), which replaces the fixed Fourier basis along the sample mode with a data-driven graph spectral basis, thereby capturing the intrinsic manifold structure without relying on a particular sample ordering. Moreover, we develop an anchor-based variant to address large-scale datasets efficiently. Extensive experiments on various benchmarks validate our findings and demonstrate the competitive or superior performance of the proposed methods compared with state-of-the-art TMC approaches.

Jintian Ji, Xingsu Li, Songhe Feng

Computer Science and Engineering The Hong Kong University of Science and Technology Hong Kong, China · School of Computer and Information Technology Beijing Jiaotong University, Beijing, China

Tensor Decomposition Clustering Low-Rank Structure High-Order Correlations Multi-View Periodic+1

Jul 17, 2026cs.LG

(MPO) $^2$ : Multivariate Polynomial Optimization based on Matrix Product Operators

Central to machine learning and signal processing is the ability to perform universal function approximation and learn complex input-output relationships from limited numbers of observations. Multivariate polynomial models offer a natural way to express such relationships through multiplicative feature interactions, but their coefficient tensors grow exponentially in size with the polynomial degree. Existing tensorized polynomial models reduce this cost, yet canonical polyadic decompositions have rank-limited expressivity, and tensor train formulations are feature order dependent. We introduce Multivariate Polynomial Optimization based on Matrix Product Operators (MPO)

^2

, a framework that combines learned MPO feature embeddings with compact polynomial weight tensors. This yields feature order independent polynomial representations that can incorporate structured operators such as projections, convolutions, and masks for weight tensor symmetries. Across regression and classification benchmarks, (MPO)

^2

improves over existing tensor decomposition based polynomial models and provides a flexible alternative for efficient polynomial function approximation.

Niccolò Ciolli, Anders Vestergaard Nørskov, Michael Kastoryano +2

Tensor Decomposition Matrix Multiplication Equivariant Learning Multilayer Perceptrons

Jul 8, 2026stat.ML

Tensorized algorithms and scalable filtering methods for hidden Markov and factorial hidden Markov models

A common method for the representation and analysis of time-series data is the hidden Markov model (HMM), where each observation is associated with a hidden state that evolves over time. However, many real-world systems are influenced by multiple independent factors, which are more naturally represented by factorial hidden Markov models (fHMM), where several hidden Markov chains jointly generate the observed data. Although an fHMM provides a richer and more realistic representation of many real-world systems, it can be reformulated as an equivalent HMM, but with a significantly larger state-space, leading to a severe increase in computational cost. In particular, the forward filtering algorithm, which is central to evaluation, decoding, and estimation tasks, becomes prohibitively expensive even for small systems. This work focuses on developing scalable methods for time-series analysis using tensor algebra to exploit the multidimensional structure of fHMM directly, without constructing intermediate HMM representations. Our novel filtering approach significantly improves computational performance and enables the efficient analysis of large systems and datasets, extending the scope of fHMM and providing a practical framework for data intensive applications.

Roxana Barrios, Ioannis Sgouralis

Department of Mathematics, University of Tennessee, Knoxville, TN

Hidden Markov Models Tensor Decomposition Bayesian Filtering Markov Hidden States

Jun 30, 2026math.NA

Online TT-ALS for Streaming Tensor Decomposition with Incremental Orthogonalization

Tensor Train (TT) decomposition is a powerful technique for analyzing high-dimensional data. Existing algorithms for computing TT decompositions can be categorized into two main types: conventional batch-based approaches and recursive online methods. In the context of streaming data, batch methods typically achieve higher reconstruction accuracy but often suffer from memory exhaustion, while online methods provide greater computational efficiency. In this work, we introduce Online TT-ALS (Alternating Least Squares), an algorithm that sequentially enforces orthogonality constraints. This approach allows for efficient and exact updates of the core tensor while maintaining high reconstruction accuracy. Theoretically, we prove that enforcing these orthogonal gauge constraints guarantees monotonic decrease of the local objective function and temporal smoothness. Computationally, our deterministic single-sweep update reduces the rank dependence from quadratic to linear, achieving an overall complexity of

\mathcal{O}(I^{n-1} r)

. Experimental results demonstrate that the proposed method outperforms existing online techniques not only in terms of mathematical approximation accuracy but also in human perception-based video quality metrics. Furthermore, compared to recent deep learning-based paradigms, our algebraic approach achieves speedups of several orders of magnitude. Consequently, our method exhibits high computational efficiency and is suitable for low-latency real-time processing applications.

Hiroki Takeda, Yuto Miyatake, Daisuke Furihata

Department of Pure and Applied Mathematics, The University of Osaka, Japan · D3 Center, The University of Osaka, Japan

Tensor Decomposition Reconstruction Fidelity Decomposition

Jun 23, 2026cs.LG

Low-Cost High-Order Singular Value Decomposition for Tensor-Based Reconstruction from Sparse Sensor Measurements: Urban Flow and Air-Quality Applications

Urban flow and air-quality simulations generate high-dimensional datasets describing velocity and pollutant transport across multiple spatial, temporal, and physical-variable dimensions. Reconstructing these fields from sparse sensor measurements is a fundamental challenge in environmental monitoring, digital twins, forecasting, and data assimilation. Existing low-cost reconstruction approaches are commonly based on matrix decompositions, which require multidimensional datasets to be flattened into two-dimensional snapshot matrices, thereby discarding important structural information. This work introduces the low-cost High-Order Singular Value Decomposition (lcHOSVD), a novel tensor-based sparse-sensing reconstruction framework for high-dimensional environmental fields. To the authors' knowledge, this is the first methodology that combines sparse sensing and HOSVD for field reconstruction. Unlike matrix-based approaches, lcHOSVD preserves the natural tensor structure of the data, enabling the exploitation of correlations across spatial, temporal, and physical-variable dimensions while substantially reducing the computational requirements of conventional HOSVD. The methodology is applied to urban flow and air-quality datasets, where three-dimensional velocity and pollutant concentration fields are reconstructed using only 1-4% of the available spatial locations. While lcSVD provides larger computational speed-ups, lcHOSVD consistently achieves lower reconstruction errors in configurations characterized by strong multidimensional coupling and heterogeneous dynamics across dimensions. Additional sensor-anisotropy analyses demonstrate that the tensor formulation is significantly more robust to uneven sensor distributions, a common situation in practical environmental monitoring networks.

Arindam Sengupta, Paul Jeanney, Ricardo Vinuesa +2

ETSI Aeronáutica y del Espacio, Universidad Politécnica de Madrid, Plaza Cardenal Cisneros, 3, Madrid, 28040, Spain · Ove Arup, C. de Alfonso XI, 12, Retiro, Madrid, 28014, Spain · Department of Aerospace Engineering, University of Michigan, Ann Arbor, MI 48109, United States

Tensor Decomposition Singular Value Decomposition Spatiotemporal Fields Air Pollution Velocity Field

Jun 22, 2026q-bio.GN

Privacy-preserving federated tensor decomposition of single-cell immune data: recovering multicellular programs across institutions

Tensor decomposition of donor

\times

cell-type

\times

gene single-cell data recovers \emph{multicellular programs}: coordinated axes of inter-individual transcriptional variation that span cell types and stratify disease. Yet immune single-cell atlases are increasingly multi-institution, multi-ancestry, and governed, so patient cells often cannot be pooled. We present a federated estimator: each site computes a local program subspace, and a coordinator merges these by stacked SVD under federated global-mean centering, provably equivalent (up to truncation) to the centralised decomposition. This centering makes the merge robust to site-label confounding (program AUC

0.957

vs.\

0.861

for naive per-site centering). Only program subspaces leave a site, and aggregation is compatible with secure aggregation. On a 261-donor systemic lupus erythematosus atlas it recovers the canonical interferon program (ISG enrichment AUC

0.998

; case--control separation

0.958

; bootstrap

Δ\text{AUC}=-0.000

, 95% CI

[-0.004,+0.012]

vs.\ centralised), across institution-scale and multi-ancestry partitions, and across three \emph{real} COVID-19 sites (subspace correlation

0.989

). It recovers the program when \emph{no site observes all cell types} (correlation

1.000

, exact by construction), which fixed-feature federated PCA cannot. On an interstitial-lung-disease atlas the recovered program predicts disease better than the best single cell type (AUC

0.96

vs.\

0.91

; gap 95% CI excludes zero) and the advantage survives federation; a liver cohort is consistent (

p=0.005

). Membership-inference shows secure aggregation cuts attack AUC from

0.91

to

0.61

. The method enables cross-institution, cross-ancestry recovery of multicellular immune programs without sharing cells.

Axel Faes, Stephanie M. van den Berg, Maryam Amir Haeri

Data Science Institute, Hasselt University, Belgium · University of Twente, The Netherlands · Learning, Data Analytics and Technology, University of Twente, The Netherlands

Single-Cell Rna Tensor Decomposition Decomposition Institutions

Jun 12, 2026cs.LG

Separable Neural Architectures as Physical World Models: from Mathematical Theory to Applications

This work introduces the Separable Neural Architecture (SNA), a function representational class combining neural approximation with tensor decomposition. The SNA decouples localized coordinate functions (atoms) from global interactions governed by a sparse, low-rank interaction object. This architecture possesses a compact and smooth inductive bias well-suited for solving partial differential equations (PDEs). When viewed as a Galerkin trial space under the variational SNA (VSNA) framework, the formulation satisfies classical variational guarantees under Lax-Milgram: well-posedness, quasi-optimality, convergence, and stability. In high-dimensional spatiotemporal--parametric PDEs, the VSNA mitigates the curse of dimensionality by scaling algebraically rather than exponentially. Exploiting an entirely factorized, tensor-native alternating least squares (ALS) optimization framework reduces this cost to linear in dimension. The VSNA is validated across elliptic, hyperbolic, and parabolic systems, demonstrating close alignment with predicted algebraic and spectral scaling rates. We showcase the SNA as a "solve once, query anywhere" physical world model via two engineering case studies: a 7D parametric manufacturing simulation and an experimental thermal-to-property inversion pipeline for Inconel 718. The VSNA executes a 1,000,000-query Monte Carlo sweep in 102s on a standard laptop CPU, yielding a 150,000x speedup over a full-grid finite element baseline hosted on an NVIDIA A100 GPU. It further enables real-time generative inverse-mode reconstructions under 100ms. These results demonstrate that the SNA serves as a compact mathematical substrate for continuous parameter manifolds to enable real-time inversion, optimization loops, and rapid uncertainty propagation.

Reza T Batley, Andrew Kichline, Sourav Saha

Kevin T. Crofton Department of Aerospace and Ocean Engineering, Virginia Polytechnic Institute and State University, 1600 Innovation Drive, Blacksburg, 24060, VA, United States

Neural Partial Differential Equation Solvers Partial Differential Equations Deep Learning Architectures Finite Element Method Tensor Decomposition Deep Unfolding Networks+2

Jun 9, 2026cs.LG

SirenFNO: Efficient and Full Frequency Learning of Fourier Neural Operators

Fourier neural operators (FNOs) are effective and efficient surrogates for approximating solutions of PDEs and generalize across discretizations. However, owing to the reliance on frequency truncation to maintain learning efficiency of FNOs, empirical studies suggest that FNOs exhibit spectral bias toward low-frequency information, which may hinder the learning capability especially for certain PDEs with strong high-frequency oscillations. To address this limitation, we propose SirenFNO, a novel framework that leverages sinusoidal representation networks (SIRENs) to learn implicit neural representations and performs mode-wise kernel parameterization. Our SIREN parameterization learns a full-grid spectrum with a constant and discretization-independent parameter count, thereby eliminating the need for frequency truncation. We further extend SirenFNO with functional tensor decompositions to enhance parameter and learning efficiency. Empirical results show that our SirenFNO consistently outperforms FNO with approximately

4

to

15

times parameter reductions with preserved discretization invariance, and our functional decomposition variants obtain performance improvements with a maximum of

73

times fewer parameters across multiple PDE benchmarks.

Pengqing Shi, Jie Yin, Stephen Tierney +1

The University of Sydney, Australia

Fourier Neural Operators Neural Operators Discretization Frequency Tensor Decomposition

Jun 3, 2026cs.LG

Hyperparameter Learning for Latent Factorization of Tensors for Representation Learning to Large-scale Dynamic Weighted Directed Network

Large-scale dynamic weighted directed networks (DWDNs) are widely used to model time-varying interactions among nodes. Latent factorization of tensors (LFT) extracts target knowledge from DWDNs via low-rank embedding. However, similar to many machine learning models, the performance of LFT heavily depends on the selection of hyperparameters. In practice, these parameters are often tuned manually or through grid search, which requires significant computational resources and human effort. Motivated by this challenge, this paper proposes an automated hyperparameter optimization framework based on Differential Evolution (DE) for LFT (DE-LFT). The proposed method integrates DE into the training process of the LFT model to automatically learn optimal regularization parameters

λ_1

,

λ_2

and

λ_3

. As a result, the model can adaptively search the hyperparameter space and improve prediction accuracy. Experimental results on four real-world datasets demonstrate that the proposed approach achieves lower MAE and RMSE compared with manually tuned baselines while reducing the need for extensive parameter tuning.

Yaqian Zhan, Jialan He, Tianzhu Chen

College of Computer and Information Science Southwest University Chongqing, China

General Grid Search Framework Tensor Decomposition Network Science Representation Learning Exploratory Factor Analysis

Jun 2, 2026cs.LG

Rethinking the Role of Tensor Decompositions in Post-Training LLM Compression

Post-training compression is essential for deploying large language models (LLMs) under tight resource constraints. Tensor decompositions have emerged as a promising direction, offering compact parameterizations well suited to Transformer weight structures. However, existing studies evaluate these methods in narrow settings, leaving unclear whether tensorization is effective at large-scale deployment. We systematically evaluate tensor compression across dense and MoE architectures, establishing performance trade-offs grounded in both empirical analysis and theoretical analysis. We identify a fundamental mismatch between the shared subspaces assumed by tensor decompositions and the heterogeneous representations learned by modern LLMs, thereby delineating their practical limits and clarifying their viable role in large-scale deployment. The code is available at https://github.com/brain-lab-research/TT-LLM.

Artur Zagitov, Alexander Miasnikov, Maxim Krutikov +5

BRAIn Lab, Moscow, Russia

Large Language Model Compression Tensor Decomposition Transformer Architectures Post-Training Role

Jun 2, 2026cs.LG

Bayesian Tensor Decomposition with Diffusion Model Prior

Low-rank tensor decomposition (TD) is usually effective on clean, fully observed data, but it often degrades under severe missingness or noise. Low-rankness is itself a useful but limited structural prior, and additional handcrafted priors (e.g., sparsity or smoothness) still fall short of capturing the rich statistics of real-world data. To compensate for this weak inductive bias under heavy corruption, one would like to inject a learned, data-driven prior; however, the state-of-the-art diffusion models are not readily compatible with current TD and tractable posterior inference. To address these challenges, we introduce DiffBCP, a hybrid-prior Bayesian CP decomposition framework that couples a cumulative shrinkage process prior over the CP factors for automatic rank selection with an off-the-shelf pre-trained diffusion model as an implicit data prior on the reconstructed tensor. To make posterior inference tractable despite the coupling among the likelihood, low-rank constraint, and diffusion prior, we develop a split Gibbs sampler: CP factors admit conjugate updates, while the diffusion block is sampled via low-rank-guided denoising. A noise-adaptive coupling schedule further reduces sensitivity to hand-tuned annealing. Experiments on image inpainting and denoising, including high-resolution out-of-distribution images, show consistent gains over Bayesian, nonlinear, and plug-and-play TD baselines.

Zerui Tao, Qibin Zhao

RIKEN Center for Advanced Intelligence Project (AIP), Tokyo, Japan.

Tensor Decomposition Diffusion Priors Low-Rank Structure Bayesian Posterior Decomposition+1

May 28, 2026cs.LG

Open Problem: Separating Geometric and Algorithmic Compression via Cayley-Table Completion

Modern statistical learning theory and deep learning characterize generalization primarily in terms of continuous capacity control (e.g., norm-based regularization, margin maximization, low-rank bias). While highly successful in continuous domains, deep learning consistently fails to extrapolate exact algorithmic or discrete algebraic rules, reflecting a missing inductive bias toward algorithmic complexity minimization. We propose the Cayley-table completion as the canonical testbed for this missing bias, serving as the discrete algebraic counterpart to matrix completion. Just as matrix factorization combined with weight decay yields an implicit geometric bias toward low linear rank, recent results demonstrate that operator-valued tensor factorizations paired with a flatness prior yield an implicit algorithmic bias toward exact discrete associativity. We pose the open problem of establishing formal exact recovery bounds for Cayley-table completion, and challenge the community to generalize continuous flatness priors to autonomously discover broader discrete algorithmic axioms without combinatorial search.

Dongsung Huh

Tensor Completion Optimal Sample Complexity Exact Open Problems Tensor Decomposition Completion+1

May 27, 2026cs.CV

Transfer learning RGB models to hyperspectral images with trainable tensor decompositions

Transfer learning makes it possible to use large vision networks on a variety of domains, by specializing their models' general filters to new tasks. However, these networks assume the input images to have 3 input channels, making them incompatible with multi- or hyperspectral images. Current approaches that mitigate this incompatibility sacrifice information in either the image, or the model. This work proposes a novel approach that preserves the image and spatial information present in the model by using partially trainable tensor decompositions. We create such decompositions of pretrained convolutional filters, separating the filters into spatial and spectral components. The spectral components are then replaced with trainable components of higher channel dimensionality. This creates hyperspectral filters that can specialize to new datasets, while retaining the spatial patterns of the original filter. Experiments on a variety of hyperspectral datasets show that our approach is more accurate and robust than other hyperspectral transfer learning methods.

Mariette Schönfeld, Laurens Devos, Wannes Meert +1

KU Leuven, Dept. of Computer Science, B-3000 Leuven, Belgium · Leuven.AI - KU Leuven Institute for AI, B-3000 Leuven, Belgium

Hyperspectral Image Transfer Learning Two-Level Spatial-Frequency Fusion Strategy Tensor Decomposition Convolutional Red-Green-Blue+1

May 13, 2026cs.LG

A Hybrid Tucker-LSTM Tensor Network Model for SOC Prediction in Electric Vehicles

Accurate state of charge estimation is critical for the success of electric vehicle battery management strategies, but it is well known that conventional estimators suffer from two fundamental shortcomings: cumulative errors that grow over time and reliance on simplified battery models that do not reflect real world dynamics. Therefore, this paper presents a novel hybrid approach combining Tucker tensor decomposition with LSTM networks, using full - lifecycle EV field data for SOC prediction. The inputs are charge status, mileage, voltage, current, cell differentials, and temporal features. Tucker decomposition is skillfully used to reduce dimensionality while maintaining the temporal structure, hence allowing a direct, fair comparison with standard LSTM. The result is unequivocal: Tucker - LSTM outperforms the baseline on all metrics, with MSE dropping 70.5% (from 21.07 to 6.22 ), MAE improving 48.7% (from 3.37% to 1.73%), RMSE falling from 4.59% to 2.49%, and

R^2

rising from 0.918 to 0.976. Since the experimental results demonstrably demonstrate that tensor decomposition compresses high-dimensional battery data very well without loss of predictive fidelity, this paper naturally opens up a new direction for tensor-based analytics in electric vehicle battery management.

Han Wang, Ying Wang, Bing Wang

College of Computer and Information Science Southwest University Chongqing, China · School of Culture Tourism Chongqing Vocational College of Culture and Arts Chongqing, China · Digital Intelligence Center China Automotive Engineering Research Institute Co., Ltd. Chongqing, China

Electric Vehicles Lithium-Ion Batteries Tensor Decomposition

May 11, 2026cs.LG

Robust Basis Spline Decoupling for the Compression of Transformer Models

Decoupling is a powerful modeling paradigm for representing multivariate functions as compositions of linear transformations and univariate nonlinear functions. A single-layer decoupling can be viewed as a fully connected neural network with a single hidden layer and flexible activation functions, providing a direct link with neural networks. Because of this, the use of decoupling methods has gained increasing attention in neural network domains, particularly compression, since it enables structured approximations with reduced parameter complexity. Existing tensor-based decoupling methods typically rely on polynomial or piecewise-linear parameterizations of the internal nonlinear functions, which can suffer from numerical instability or limited expressiveness. In this work, we introduce a B-spline-based decoupling framework that generalizes these existing approaches. By exploiting the local support and flexible smoothness control of B-splines, the proposed formulation yields a more numerically stable and expressive representation. We derive a constrained coupled matrix-tensor factorization and propose a robust alternating least-squares algorithm, called R-CMTF-BSD, incorporating normalization and Tikhonov regularization. The proposed method is validated through experiments on synthetic data and transformer model compression. Results on the Vision and Swin Transformer architectures demonstrate that B-spline decoupling enables substantial parameter reduction while maintaining competitive accuracy, making the R-CMTF-BSD algorithm a promising tool for structured neural network compression.

Joppe De Jonghe, Van Tien Pham, Mariya Ishteva

aNUMA, Department of Computer Science, KU Leuven, Kleinhoefstraat 4, Geel, 2440, Antwerpen, België · bUniversité Paris-Saclay, CEA, List, Palaiseau, F-91120, France

Tensor Decomposition Transformer Architectures Splines Data Compression Methods

May 6, 2026cs.LG

A Biased Nonnegative Block Term Tensor Decomposition Model for Dynamic QoS Prediction

With the rapid development of cloud computing and Web services, Quality of Service (QoS) has become a key criterion for service selection and recommendation. Tensor latent feature analysis provides an effective way to model multidimensional QoS data, and most existing QoS prediction methods are mainly based on Canonical Polyadic (CP) decomposition or Tucker decomposition. However, constrained by their inherent structural properties, these methods cannot accurately capture the complex and dynamic dependencies in user-service interactions, which limits their prediction performance. To address this issue, this paper proposes a dynamic QoS prediction framework based on the Biased Nonnegative Block Term Tensor Decomposition Model, termed BNBT. Specifically, the proposed framework is developed from three aspects: (1) block term tensor decomposition is employed to enhance the representation capability of latent feature learning; (2) linear bias terms are incorporated to further improve prediction accuracy; and (3) a tensor-oriented single-element-dependent nonnegative multiplicative update algorithm, called SLF-NMUT, is designed for efficient parameter estimation. Extensive experiments on real-world QoS datasets demonstrate that the proposed BNBT framework consistently outperforms several state-of-the-art QoS prediction methods in terms of prediction accuracy.

Wenjing Liu, Yujia Lei, Qu Wang

College of Computer and Information Science Southwest University Chongqing, China

Tensor Decomposition Future Latent Representations

Apr 29, 2026cs.AI

Toward Personalized Digital Twins for Cognitive Decline Assessment: A Multimodal, Uncertainty-Aware Framework

Cognitive decline is highly heterogeneous across individuals, which complicates prognosis, trial design, and treatment planning. We present the Personalized Cognitive Decline Assessment Digital Twin (PCD-DT), a multimodal and uncertainty-aware framework for modeling patient-specific disease trajectories from sparse, noisy, and irregular longitudinal data. The framework combines three methodological components: (1) latent state-space models for individualized temporal dynamics, (2) multimodal fusion for clinical, biomarker, and imaging features, and (3) uncertainty-aware validation and adaptive updating for robust digital twin operation. We also outline how conditional generative models can support data augmentation and stress testing for underrepresented progression patterns. As a preliminary feasibility study, we analyze longitudinal TADPOLE trajectories and show clear separation between cognitively normal and Alzheimer's disease cohorts in ADAS13, ventricle volume, and hippocampal volume over five years. We further conduct a multimodal next-visit prediction ablation using an LSTM sequence model on 3{,}003 visit-pair sequences derived from TADPOLE, where the combined cognitive plus MRI configuration achieves the lowest standardized RMSE for both ADAS13 (0.4419) and ventricle volume (0.5842), outperforming a Last Observation Carried Forward baseline. A Bayesian tensor modeling component for high-dimensional imaging fusion is also discussed. These results support the feasibility of the proposed architecture while also highlighting the need for stronger uncertainty calibration and longer-horizon predictive evaluation. The PCD-DT framework provides a principled starting point for personalized in silico modeling in neurodegenerative disease. This work positions PCD-DT as a foundational step toward clinically deployable, uncertainty-aware digital twin systems.

Bulent Soykan, Gulsah Hancerliogullari Koksalmis, Hsin-Hsiung Huang +1

Department of Mechanical, Industrial & Manufac. Eng. The University of Toledo Toledo, OH, USA · Department of Industrial Eng. and Mngt. Systems University of Central Florida Orlando, FL, USA · Department of Statistics and Data Science University of Central Florida Orlando, FL, USA +1

Mild Cognitive Impairment Patient Trajectories Digital Twins Multimodal Neuroimaging Tensor Decomposition

Apr 16, 2026stat.ML

Unsupervised feature selection using Bayesian Tucker decomposition

In this paper, we proposed Bayesian Tucker decomposition (BTuD) in which residual is supposed to obey Gaussian distribution analogous to linear regression. Although we have proposed an algorithm to perform the proposed BTuD, the conventional higher-order orthogonal iteration can generate Tucker decomposition consistent with the present implementation. Using the proposed BTuD, we can perform unsupervised feature selection successfully applied to various synthetic datasets, global coupled maps with randomized coupling strength, and gene expression profiles. Thus we can conclude that our newly proposed unsupervised feature selection method is promising. In addition to this, BTuD based unsupervised FE is expected to coincide with TD based unsupervised FE that were previously proposed and successfully applied to a wide range of problems.

Y-h. Taguchi, Yoh-ichi Mototake

Department of Physics, Chuo University, Tokyo, 112-8551, Japan. · Graduate School of Social Data Science, Hitotsubashi University, Tokyo, 186-8601, Japan.

Feature Selection Tensor Decomposition Unsupervised Decomposition

Jun 30, 2025math.OC

Flow-Through Tensors: A Unified Computational Graph Architecture for Multi-Layer Transportation Network Optimization

Modern transportation network modeling increasingly involves the integration of diverse methodologies including sensor-based forecasting, reinforcement learning, classical flow optimization, and demand modeling that have traditionally been developed in isolation. This paper introduces Flow Through Tensors (FTT), a unified computational graph architecture that connects origin destination flows, path probabilities, and link travel times as interconnected tensors. Our framework makes three key contributions: first, it establishes a consistent mathematical structure that enables gradient-based optimization across previously separate modeling elements; second, it supports multidimensional analysis of traffic patterns over time, space, and user groups with precise quantification of system efficiency; third, it implements tensor decomposition techniques that maintain computational tractability for large scale applications. These innovations collectively enable real time control strategies, efficient coordination between multiple transportation modes and operators, and rigorous enforcement of physical network constraints. The FTT framework bridges the gap between theoretical transportation models and practical deployment needs, providing a foundation for next generation integrated mobility systems.

Xuesong Zhou, Taehooie Kim, Mostafa Ameli +3

Accurate Traffic Forecasting Origin Destination Flows Tensor Decomposition Data-Driven Forecasting