stat.MLSep 18, 2026

A Bayesian Vertical Federated Learning Framework for Multivariate Reduced-Rank High-Dimensional Regression

Authors: Brigham Halverson, Sharmistha Guha, Jessica Bernard, Rajarshi Guhaniyogi

Organizations: Department of Statistics Texas A&M University · Department of Psychological & Brain Sciences Texas A&M University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Federated learning (FL) has emerged as a leading privacy-preserving framework for collaborative machine learning across decentralized environments. While considerable progress has been made in horizontal federated learning (HFL), where data with common features is distributed across sites, vertical federated learning (VFL), where sites share observations across distinct feature sets, remains less explored. Advancing Bayesian high-dimensional multivariate reduced-rank regression methods for VFL poses unique challenges: (a) stringent privacy regulations preventing local site data sharing, and (b) fitting local regressions overlooks essential modeling aspects like inter-variable correlations. In contrast HFL allows each site to fit a comparable model independently. We present a novel Bayesian VFL framework for multivariate high-dimensional reduced-rank regression, termed BayesVFLReg, which enables precise coefficient estimation while safeguarding both feature and response privacy. Participating sites use a shared random sketching matrix to compress local variables into privacy-preserving sketches. A central server collects these sketches where Bayesian multivariate reduced-rank regression uses Gaussian scale mixture priors. For feature selection, we introduce a single-step post-processing strategy based on mixture-model clustering of the absolute posterior coefficient means to distinguish signal from noise per response variable. BayesVFLReg is computationally scalable for large, high-dimensional datasets and facilitates efficient variable selection. Theoretically, we establish sharp non-asymptotic bounds on the posterior probability that the fitted density falls within a Hellinger ball centered at the true data-generating density. Comparative simulation studies and real-world data analyses show that BayesVFLReg reliably identifies sparse feature effects, even under feature correlation.

Figures & tables

Figure 1: VFL for Multivariate Regression using Federated Sketch Construction
Figure 2: Figure shows a summary of the MSE (blue) and MSE nz (red) across the 50 replicates. The dots denote mean MSE with intervals around the mean showing standard errors over 50 replications. As privacy decreases (larger mm ), accuracy improves. At higher-degree of sparsity (smaller ss ), more privacy can be guaranteed while achieving competitive performance relative to the uncompressed model.
  ss = 10s=30s=30ss = 50
Avg. Accuracy nzm=50m=500.9950.9970.998
m=100m=1000.9940.9970.998
m=250m=2500.9940.9960.997
Avg. Accuracy zm=50m=500.9970.9970.998
m=100m=1000.9980.9980.999
m=250m=2500.9990.9990.999
Table 1: The top six rows depict the measure of accuracy of estimating the full posterior of C{\boldsymbol{C}} from compressed data, as expressed in 8 . Results are disaggregated by zero and non-zero features, varying sparsity levels, and different values of m=50,100,250m=50,100,250 representing differing privacy levels. This metric is averaged over 50 replicates; values closer to the upper bound of 1 indicate better posterior estimates with compressed data. These results demonstrate that the compressed approach closely approximates the centralized posterior. The bottom 4 rows present computational efficiency (see 9 ), also averaged over 50 replicates, illustrating that data compression substantially improves computational performance relative to uncompressed methods.
Figure 3: Figure depicts the MSE (blue) and MSE nz (red) of the four respective models; BayesVFLReg , dRidge , locus and srrr on the log scale. The points depict the average log(MSE) or log(MSE nz ) across replicates while the bars depict a 95% interval of the replicates. It can be observed that, as the correlation increases, the data sketching approach exhibits progressively better comparative performance.
Figure 4: Figure displays average coverage (blue) and average coverage nz (red) as bars, along with points denoting the average interval width under each simulation case. This demonstrates that BayesVFLReg achieves superior overall coverage; however, its interval widths are larger than those of the other two methods, which, in turn, exhibit notably poorer performance for coverage nz .
 Scenario 1Scenario 2Scenario 3
 s=10s=10s=30s=30s=50s=50s=10s=10s=30s=30s=50s=50s=10s=10s=30s=30s=50s=50
BayesVFLReg0.0040.0370.0630.0740.1170.1590.1320.0790.202
dRidge0.0000.0000.0000.0000.0000.0000.0000.0000.000
locus0.7890.5610.4350.7970.5750.4640.8040.5860.471
srrr0.3490.2410.1340.3910.3430.2520.3160.2880.271
Table 2: Table reports the average false discovery rate (FDR) . Given α~=0.05\tilde{\alpha}=0.05 , this value should be close to 0.05. While dRidge successfully separates non-zero from zero coefficients, as indicated by the coverage, it struggles to accurately estimate their magnitude.
Figure 5: Shows relative predictive performance, measured as the ratio of the Mean Squared Error (MSE Y ) of BayesVFLReg to that of each respective competitor. The horizontal reference line at 1.0 indicates performance parity with each competitor; values below this line demonstrate that BayesVFLReg outperforms the competitor.
Figure 6: The significant ROI-ROI correlations on the participants’ cognitive and motor functions are depicted here. Figure ( 6(a) ) depicts the brain highlighting the ROIs that were found among the 60 significant links. Figure ( 6(b) ) illustrates the significant ROI-ROI correlations with the edge width depicting the absolute strength of the relationship between the correlation and the cognitive and motor functions of the participants found by taking the row means of the posterior mean of C{\boldsymbol{C}} . A full list of significant ROI with their shorthand mappings can be found in Table 3 from the Appendix.
Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

IndexCodeRegion of Interest (ROI)
1L_IPLLeft inferior parietal lobule
2L_VSsLeft superior ventral striatum
3L_AMYGLeft amygdala
4L_DCaudLeft dorsal caudate
5R_VSsRight superior ventral striatum
6R_LPCRight lateral parietal cortex
Appendix
Table 3: 53 ROIs found in the 60 significant ROI-ROI connections with corresponding shorthand codes (short hand codes following Di Martino et al. [2008] , Desikan et al. [2006] , Diedrichsen et al. [2009] ).
More in this topic

High-Dimensional Linear Regression

See all 16 papers

Explore similar work

CardsList
Feb 17, 2026

Exploring New Frontiers in Vertical Federated Learning: the Role of Saddle Point Reformulation

The objective of Vertical Federated Learning (VFL) is to collectively train a model using features available on different devices while sharing the same users. This paper focuses on the saddle point reformulation of the VFL problem via the classical Lagrangian function. We first demonstrate how this formulation can be solved using deterministic methods. More importantly, we explore various stochastic modifications to adapt to practical scenarios, such as employing compression techniques for efficient information transmission, enabling partial participation for asynchronous communication, and utilizing coordinate selection for faster local computation. We show that the saddle point reformulation plays a key role and opens up possibilities to use mentioned extension that seem to be impossible in the standard minimization formulation. Convergence estimates are provided for each algorithm, demonstrating their effectiveness in addressing the VFL problem. Additionally, alternative reformulations are investigated, and numerical experiments are conducted to validate performance and effectiveness of the proposed approach.

Aleksandr Beznosikov, Georgiy Kormakov, Alexander Grigorievskiy +7

Basic Research of Artificial Intelligence Laboratory (BRAIn Lab) · Federated Learning Problems Laboratory · Innopolis University +4

Federated Learning
AskPDF
Apr 15, 2026

Secure and Privacy-Preserving Vertical Federated Learning

We propose a novel end-to-end privacy-preserving framework, instantiated by three efficient protocols for different deployment scenarios, covering both input and output privacy, for the vertically split scenario in federated learning (FL), where features are split across clients and labels are not shared by all parties. We do so by distributing the role of the aggregator in FL into multiple servers and having them run secure multiparty computation (MPC) protocols to perform model and feature aggregation and apply differential privacy (DP) to the final released model. While a naive solution would have the clients delegating the entirety of training to run in MPC between the servers, our optimized solution, which supports purely global and also global-local models updates with privacy-preserving, drastically reduces the amount of computation and communication performed using multiparty computation. The experimental results also show the effectiveness of our protocols.

Shan Jin, Sai Rahul Rachuri, Yizhen Wang +2

Visa Research

Privacy-Preserving Federated LearningFederated LearningFederated Learning AggregationDifferentially Private Federated Learning
AskPDF
Apr 2, 2026

BVFLMSP : Bayesian Vertical Federated Learning for Multimodal Survival with Privacy

Multimodal time-to-event prediction often requires integrating sensitive data distributed across multiple parties, making centralized model training impractical due to privacy constraints. At the same time, most existing multimodal survival models produce single deterministic predictions without indicating how confident the model is in its estimates, which can limit their reliability in real-world decision making. To address these challenges, we propose BVFLMSP, a Bayesian Vertical Federated Learning (VFL) framework for multimodal time-to-event analysis based on a Split Neural Network architecture. In BVFLMSP, each client independently models a specific data modality using a Bayesian neural network, while a central server aggregates intermediate representations to perform survival risk prediction. To enhance privacy, we integrate differential privacy mechanisms by perturbing client side representations before transmission, providing formal privacy guarantees against information leakage during federated training. We first evaluate our Bayesian multimodal survival model against widely used single modality survival baselines and the centralized multimodal baseline MultiSurv. Across multimodal settings, the proposed method shows consistent improvements in discrimination performance, with up to 0.02 higher C-index compared to MultiSurv. We then compare federated and centralized learning under varying privacy budgets across different modality combinations, highlighting the tradeoff between predictive performance and privacy. Experimental results show that BVFLMSP effectively includes multimodal data, improves survival prediction over existing baselines, and remains robust under strict privacy constraints while providing uncertainty estimates.

Abhilash Kar, Basisth Saha, Tanmay Sen +1

SQC and OR Unit, Indian Statistical Institute, Kolkata, India · Department of Statistics, Presidency University, Kolkata, India

Multimodal Survival AnalysisFederated LearningUncertainty-Aware Multimodal FusionSplit Federated LearningDifferentially Private Federated Learning
AskPDF