With the rapid development of modern scientific research, tensor data have become increasingly prevalent across a wide range of fields. How to effectively characterize the relationship between tensor predictors and a response while preserving the intrinsic structure of tensor data has therefore become an important problem in high-dimensional statistical analysis. Existing tensor regression methods mainly focus on conditional means or conditional quantiles. When the response distribution is asymmetric, heavy-tailed, or contaminated by outlying observations, the conditional modal provides a robust and interpretable alternative characterization. In this paper, we propose a partial modal tensor regression (PMTR) method. The proposed method constructs the partial modal tensor covariance, which incorporates information on the conditional modal to guide tensor dimension reduction, and subsequently establishes a modal tensor regression model in the reduced tensor space. This strategy alleviates the dimensionality burden caused by direct vectorization while preserving the intrinsic structural information of tensor predictors. Under suitable regularity conditions, we establish the consistency and asymptotic properties of the proposed estimators and employ a data-driven dimension-selection procedure to determine the effective dimension along each tensor mode.
Figure 1: Average estimated coefficient tensors for lower-rank coefficient structures under different error distributions and sample sizes. The four structures are square-shaped (rank 1), cross-shaped (rank 2), T-shaped (rank 2), and frame-shaped (rank 2). The results are averaged over 100 Monte Carlo replications. LogN and Cont. denote the log-normal and contaminated normal distributions, respectively.
Figure 2: Average estimated coefficient tensors for higher-rank coefficient structures under different error distributions and sample sizes. The three structures are disk-shaped (rank 6), triangle-shaped (rank 9), and bi-circle-shaped (rank 10). The results are averaged over 100 Monte Carlo replications.
Figure 3: Average estimated coefficient tensors for the triangle-shaped structure under different error distributions and noise level. The true coefficient tensor has rank 9 and the sample size is n=200 . The results are averaged over 100 Monte Carlo replications.
Figure 4: Boxplots of absolute Frobenius estimation errors under different simulation settings. Panel (a) compares different coefficient structures with sample size n=200 and Gaussian errors, without additional noise scaling. Panel (b) examines the effect of sample size for the bi-circle-shaped coefficient structure under Gaussian errors, without additional noise scaling. Panel (c) compares different error distributions with sample size n=200 and the bi-circle-shaped coefficient structure, without additional noise scaling. Panel (d) examines the effect of noise level with sample size n=200 , the bi-circle-shaped coefficient structure, and Gaussian errors.
Figure 5: Empirical distribution of the daily average PM 2.5 concentrations.
Figure 6: Heat-map visualization of the estimated PMTR coefficient matrix thresholded at the cutoff values 5, 8, 10, and 15. Coefficients with absolute values smaller than the corresponding cutoff are set to zero. The columns correspond to SO 2 , NO 2 , CO, O 3 , TEMP, PRES, DEWP, and WSPM, and the rows correspond to the 24 hours of a day.
Matrix-valued time series arise in a wide range of applications, such as spatio-temporal data from medical imaging and geophysics. Existing methods are mainly designed for static settings and lack adaptability to streaming and time-varying environments. Adaptive filtering techniques have also been largely limited to data with scalar or vector values, leaving adaptive forecasting for matrix-valued time series inadequately understood. To bridge these gaps, we develop an adaptive tensor regression framework that includes Matrix-on-Matrix (MoM) and Tensor-on-Matrix (ToM) formulations for streaming matrix-valued prediction. The two formulations differ in whether to directly model matrix-valued outputs or to exploit temporal structure via higher-order tensor representations. For the proposed tensor regression framework, we develop stochastic gradient descent (SGD) algorithms for online learning. We show that stacking multiple responses across time into higher-order tensors improves performance; in particular, the ToM achieves lower steady-state error and stronger denoising capability than MoM, motivating our focus on the ToM model. We further characterize the tracking behavior of SGD under time-varying dynamics. From a statistical perspective, we establish fixed-time recovery guarantees for ToM under general low-dimensional structures, including sparsity, low-rankness, and their joint sparselow-rank models.
Zhen Qin, Yang Chen
Michigan Institute for Computational Discovery and Engineering, Department of Statistics, Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109 USA · Department of Statistics, University of Michigan, Ann Arbor, MI 48109 USA
Tensor-valued data arise naturally in multidimensional signal and imaging problems, such as biomedical imaging. When incorporated into generalized linear models (GLMs), naive vectorization can destroy their multi-way structure and lead to high-dimensional, ill-posed estimation. To address this challenge, Low Separation Rank (LSR) decompositions reduce model complexity by imposing low-rank multilinear structure on the coefficient tensor. A representative approach for estimating LSR-based tensor GLMs (LSR-TGLMs) is the Low Separation Rank Tensor Regression (LSRTR) algorithm, which adopts block coordinate descent and enforces orthogonality of the factor matrices through repeated QR-based projections. However, the repeated projection steps can be computationally demanding and slow convergence. Motivated by the need for scalable estimation and classification from such data, we propose LSRTR-M, which incorporates Muon (MomentUm Orthogonalized by Newton-Schulz) updates into the LSRTR framework. Specifically, LSRTR-M preserves the original block coordinate scheme while replacing the projection-based factor updates with Muon steps. Across synthetic linear, logistic, and Poisson LSR-TGLMs, LSRTR-M converges faster in both iteration count and wall-clock time, while achieving lower normalized estimation and prediction errors. On the Vessel MNIST 3D task, it further improves computational efficiency while maintaining competitive classification performance.
Xiao Liang, Shuang Li
Department of Electrical and Computer Engineering, Iowa State University, Ames, 50014, Iowa, USA
We study rank selection for low-rank tensor regression under random covariates design. Under a Gaussian random-design model and some mild conditions, we derive population expressions for the expected training-testing discrepancy (optimism) for both CP and Tucker decomposition. We further demonstrate that the optimism is minimized at the true tensor rank for both CP and Tucker regression. This yields a prediction-oriented rank-selection rule that aligns with cross-validation and extends naturally to tensor-model averaging. We also discuss conditions under which under- or over-ranked models may appear preferable, thereby clarifying the scope of the method. Finally, we showcase its practical utility on a real-world image regression task and extend its application to tensor-based compression of neural network, highlighting its potential for model selection in deep learning.
Haoming Shi, Eric C. Chi, Hengrui Luo
Department of Statistics, Rice University · School of Statistics, University of Minnesota