Joint Precision Neural Networks: Task-Aware Dependency and Predictive Learning

Andrea Cavallo; Samuel Rey; Antonio G. Marques; Elvin Isufi

cs.LGOct 5, 2026

Joint Precision Neural Networks: Task-Aware Dependency and Predictive Learning

Authors: Andrea Cavallo, Samuel Rey, Antonio G. Marques, Elvin Isufi

Organizations: INSY Department, Delft University of Technology · Department of Signal Theory and Communications, King Juan Carlos University

Abstract

Exploiting meaningful latent structures from data to solve downstream tasks is a fundamental challenge in signal processing and machine learning. While Principal Component Analysis (PCA) and coVariance Neural Networks (VNNs) successfully leverage the covariance matrix to process data, they inherently capture both direct and indirect correlations. The precision matrix (inverse covariance) overcomes this by explicitly encoding conditional independencies, making it largely studied in graphical lasso and graph topology identification. However, finite-sample precision estimates are notoriously unstable, and regularized estimators remain task-agnostic. In this work, our principal contribution is tackling the challenging problem of task-aware graph inference. We propose Precision Neural Networks-Joint (PNN-Joint), a framework that jointly estimates a sparse, statistically grounded precision matrix alongside graph neural network weights via an alternating optimization scheme. As a foundational framework to support this, we introduce Precision Neural Networks (PNNs), a broader class of graph convolutional networks operating on precision estimators, and establish their spectral connections to PCA and VNNs alongside their stability to finite-sample errors. Extensive empirical evaluations on synthetic data, as well as real-world neuroimaging and motion sensor datasets, demonstrate that PNN-Joint yields highly interpretable task-aware graphs, exhibits remarkable robustness in low-data regimes, and consistently achieves the best or second-best performance among competitors on real-world tasks.

Figures & tables

Fig. 1 : Spectral responses of PCA, covariance and precision filters. PCA applies a high-pass filter in the covariance spectrum, selecting only the eigenvectors corresponding to the largest eigenvalues, which translates to a low-pass filter in the precision spectrum. Covariance filters (VFs) and precision filters (PFs) learn smooth polynomial functions in the same spaces. The figure illustrates two example polynomials of order 10 approximating the PCA filtering.

Fig. 2 : Comparison of covariance and precision-based methods for downstream tasks. Traditional pipelines preprocess the data via (task-agnostic) PCA and apply a separate ML model for the task. VNNs and PNNs perform task-aware covariance/precision processing, while PNN-Joint learns both the precision and the neural weights jointly in a task-aware manner.

Panel of Figure 3 — Fig. 3 : Regression task on synthetic datasets under three settings: (i-top row) increasing structural mismatch between the statistical precision matrix $\mathbf{\Theta}_{0}$ and the task precision matrix $\mathbf{\Theta}_{\textnormal{task}}$ ; (ii-bottom row, left and center panel) increasing number of samples; (iii-bottom right) increasing sparsity of the true precision matrix. To assess downstream performance, we report the normalized mean squared error (NMSE). To evaluate precision recovery, we report the scale-invariant normalized Frobenius error and the F1-score of the recovered support for both $\mathbf{\Theta}$ and $\tilde{\mathbf{\Theta}}$ .

Method	ADNI1		ADNI2		PPMI		ABIDE		MHEALTH		REALDISP
Method	MAE	NZ (%)	MAE	NZ (%)	MAE	NZ (%)	MAE	NZ (%)	Acc (%)	NZ (%)	Acc (%)	NZ (%)
MLP	9.39 $\pm$ 1.21	-	7.99 $\pm$ 0.35	-	7.26 $\pm$ 0.41	-	3.88 $\pm$ 0.38	-	82.1 $\pm$ 1.1	-	75.0 $\pm$ 4.7	-
PCA+MLP	5.59 $\pm$ 0.47	-	5.40 $\pm$ 0.54	-	6.66 $\pm$ 0.49	-	3.73 $\pm$ 0.18	-	81.7 $\pm$ 0.7	-	76.4 $\pm$ 0.7	-
SPCA+MLP	5.23 $\pm$ 0.30	-	5.50 $\pm$ 0.42	-	6.57 $\pm$ 0.56	-	3.62 $\pm$ 0.28	-	81.4 $\pm$ 3.4	-	72.0 $\pm$ 1.4	-
VNN [ 43 ]	5.31 $\pm$ 0.42	-	5.62 $\pm$ 0.29	-	6.90 $\pm$ 0.48	-	5.89 $\pm$ 0.79	-	87.3 $\pm$ 2.8	-	79.3 $\pm$ 3.1	-
S-VNN [ 8 ]	5.35 $\pm$ 0.39	-	5.78 $\pm$ 0.12	-	7.64 $\pm$ 0.59	-	6.46 $\pm$ 1.52	-	88.1 $\pm$ 2.4	-	80.7 $\pm$ 0.9	-

TABLE I : Regression error (MAE), classification accuracy and percentage of non-zero elements in the estimated precision matrix (NZ, after thresholding entries with magnitude lower than 0.1) for real-data experiments. First and second best-performing methods are highlighted. For PNN-Joint, NZ is computed on

\tilde{\mathbf{\Theta}}

Fig. 4 : Results on real datasets under finite-sample perturbations. The first row shows the task performance, the second the normalized Frobenius norm of the difference of the generated embeddings.

Fig. 5 : Visualization of the graphs learned on different subsets of REALDISP. Task 1 involves activities mainly concerned with the upper body, while Task 2 considers actions that more clearly depend on leg movements. The learned graphs reflect this distinction: on Task 1, PNN-GL only learns edges between upper-body sensors and PNN-Joint adds one edge from the chest to the upper legs, while on Task 2 more edges across sensors on the legs appear.

Explore similar work

Learning with Covariance Matrices: Principal Component Analysis Meets Learning with Graphs
Sep 9, 2026Saurabh Sihag, Andrea Cavallo, Elvin Isufi +2Covariance Graph Neural Networks
Department of Electrical and Computer Engineering at the University at Albany, SUNY, Albany, NY · Delft University of Technology, Delft, The Netherlands · Department of Electrical and Computer Engineering at the University of Rochester, Rochester, NY +1
The Regularization Parameter: Sparse Precision Matrix Estimation
Jul 7, 2026Aryan Eftekhari, Daniel Sergio Vega, Ernst-Jan Camiel Wit +1Maximum Likelihood Regularization
Institute of Computing, Faculty of Informatics, Universit`a della Svizzera italiana (USI), Lugano, Switzerland
Invariant-Based Weight Sharing for Message Passing
May 25, 2026Florian SeiffarthMessage Passing Neural Networks
University of Bonn, Bonn, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence, Bonn, Germany

Sep 9, 2026, cs.LGJ/K move · Enter open · S save

Learning with Covariance Matrices: Principal Component Analysis Meets Learning with Graphs

Saurabh Sihag, Andrea Cavallo, Elvin Isufi, Gonzalo Mateos+1

Department of Electrical and Computer Engineering at the University at Albany, SUNY, Albany, NY · Delft University of Technology, Delft, The Netherlands · Department of Electrical and Computer Engineering at the University of Rochester, Rochester, NY+1

Covariance Graph Neural Networks Principal Component Analysis

Open paper AskPDF arXiv

This feature article provides an overview of the theoretical foundations for coVariance neural networks (VNNs), i.e., graph neural networks (GNNs) operating on covariance matrices as graphs. Covariance matrices are ubiquitous across domains, and hence, the deployment of GNNs often leverages graphs of pairwise statistical dependencies. Existing theoretical contributions on GNNs consider abstract graph representations and cannot accommodate the data-driven nuances associated with covariance matrices. This tutorial brings into focus various novel theoretical insights via mathematical analyses of VNNs that have broad signal processing implications, including: (i) a conceptual equivalence between VNNs and principal component analysis (PCA)-based information processing; (ii) refined stability bounds on predictive outcomes in the presence of finite sample-induced covariance matrix perturbations; and (iii) refined characterization of transferability of VNNs across multiscale datasets. The theoretical insights discussed herein provide the underlying principles and justification towards adopting VNNs over workhorse PCA-based learning pipelines, in applications where covariance matrices are useful descriptors of data structure. We also convey how impact of these foundational advances permeates to \textit{principled} designs and applications of learning methods across broad domains where covariance matrices emerge. Notably, we elucidate the conceptual insights facilitated by VNNs to the specific task of characterizing brain age gap for neurodegenerative conditions using neuroimaging datasets, a timely problem in computational neuroscience. Broader impacts to other application domains are discussed as well.

Abstract

Explore similar work

Learning with Covariance Matrices: Principal Component Analysis Meets Learning with Graphs

The Regularization Parameter: Sparse Precision Matrix Estimation

Invariant-Based Weight Sharing for Message Passing