Secure Multi-party Computation (MPC) enables untrusted parties to jointly compute a function without revealing their inputs. Its application to machine learning (ML) has gained significant attention, particularly for secure inference services deployed across multiple cloud virtual machines (VMs), where each VM acts as an MPC party. Model providers secret-share model weights, and users secret-share inputs, ensuring that each server operates only on random shares. While MPC provides strong cryptographic guarantees, it incurs substantial computational and communication overhead. Deep neural networks rely heavily on convolutional and fully connected layers, which require costly matrix multiplications in MPC. To reduce this cost, we propose leveraging low-rank decomposition (LRD) for linear layers, replacing one large matrix multiplication with two smaller ones. Each matrix multiplication in MPC incurs a round of communication, meaning decomposing one matrix multiplication into two leads to an additional communication round. Second, the added matrix multiplication requires an additional truncation step to maintain numerical precision. Since truncation itself requires communication and computation, these overheads can offset the gains from decomposition. To address this, we introduce two complementary optimizations: truncation skipping and efficient linear layer concatenation. Truncation skipping removes the extra truncation induced by LRD, while linear layer concatenation pipelines operations to hide the additional communication round. Together, these techniques mitigate the main overheads of LRD in MPC and improve overall efficiency. Our approach is broadly applicable across MPC protocols. Experiments show up to 25% speedup in n-PC and 33% in 3-PC protocols over full-rank baselines, along with up to 52% GPU energy savings and 88% reduction in offline-phase latency.
Figures & tables
Figure 1. n -PC system topology.
Figure 3. Trio ( Harth-Kitzerow et al., 2025 ) system topology.
Protocol
Runtime
Power
VGG
42.8%
72.4%
BERT LARGE
51.6%
67.8%
GCN
36.1%
85.3%
Table 1. Linear layer runtime (%) and power (%) in MPC inference online phase (5Gbps WAN) for n -PC .
Figure 6. Computation and communication workflow for n -PC and Trio matrix multiplication; boxes labeled with a router icon denote communication runtime, and boxes labeled with a GPU icon denote computation runtime.
Figure 7. Computation and communication workflow of low-rank matrix multiplication for n -PC and Trio protocol.
Figure 8. Computation and communication workflow of low-rank matrix multiplication for n -PC and Trio protocol with truncation skipping.
Figure 9. Trio online phase matrix multiplication computation and communication workflow.
Figure 10. n -PC online phase matrix multiplication computation and communication workflow with efficient layer concatenation.
Figure 11. LRD-MPC online phase performance improvement for Trio.
Figure 12. LRD-MPC online phase performance improvement for n -PC .
Protocol
Model
LAN
MAN
WAN
Trio
GCN
44.8%
44.45%
30.23%
WideResNet18
37.49%
28.06%
14.95%
VGG19
11.36%
10.92%
5.92%
BERT LARGE
39.64%
37.91%
32.99%
n -PC
GCN
37.41%
36.01%
13.96%
WideResNet18
174.76%
125.64%
20.83%
Table 2. Speedup of LRD-MPC on different network setups.
Model
Saving (%)
GCN
30.26%
WideResNet18
52.24%
VGG19
48.41%
BERT LARGE
34.09%
Table 3. LRD-MPC online phase GPU power consumption saving (%) for n -PC
Institutions may benefit from collaborative inference on time-series data. In settings where privacy is necessary, multi-party computation (MPC) is a straightforward approach to providing strong guarantees, yet it remains prohibitively expensive and scales poorly with modern transformer architectures. Vertical Federated Learning (VFL) offers efficiency but suffers from privacy leakage at the embedding level, and securing the entire VFL model head via MPC remains prohibitively slow and communication-heavy for larger models. To enable practical, secure inference at scale, we propose "Public/Private Hybrid Head-VFL" (PPHH-VFL). This hybrid architecture splits the model head into an efficient plaintext public head and a secure, lightweight MPC private head. By applying adversarial training to the public embeddings, we mitigate privacy leakage; concurrently, the small private head securely preserves the flow of sensitive information needed for high downstream utility. Empirical evaluations on models ranging up to 86 million parameters demonstrate that PPHH-VFL accelerates inference by up to six orders of magnitude compared to end-to-end MPC. Compared to a standard VFL+MPC baseline, our approach scales significantly better, achieving a speedup of up to 44.4x in WAN and a 91.2x reduction in communication costs (dropping from 1.7 GB to 19 MB per batch), while simultaneously improving downstream classification accuracy by 2.50% and regression RMSE by 40.7%.
We present Rationally Linearized Multilayer Perceptrons(RaLiMPs), a new approach to efficiently linearizing MLPs using low-degree rational approximations of standard activation functions. Inspired by polynomial-based MLP linearization, we show that rational approximations can provide accurate representations at substantially lower degree. RaLiMPs exploit these approximations to construct kernel representations with narrow bottlenecks, enabling efficient composition across MLP layers. We provide theoretical guarantees for the resulting approximation and evaluate RaLiMPs across applications ranging from implicit 3D models to large language models. Our experiments demonstrate favorable accuracy--efficiency tradeoffs, with RaLiMPs achieving competitive or improved accuracy while substantially reducing inference cost.
Arijit Sehanobish, Byeongchan Kim, Avinava Dubey +2
Kensho Technologies · Seoul National University · Google Research +2
Transformer-based language models achieve strong performance but incur substantial inference cost due to repeated high-dimensional matrix multiplications. We propose Reduced Matrix Multiplication (RMM), a training-free, input-adaptive inference method that reduces Transformer matrix products by selecting informative slices along their contraction dimensions, without modifying model weights. Under a simple retention-ratio control, RMM provides a smooth and predictable accuracy-efficiency trade-off. Across language models ranging from 1B to 70B parameters, we find that reduction tolerance depends on the model family, task, component, and retention ratio, although it often improves with model scale. Under moderate reduction, RMM remains robust across the evaluated discriminative, autoregressive generation, and long-context settings. We further show that the same principle extends to multimodal vision-language inference. Mechanistic ablations reveal a structural asymmetry within Transformers: attention-side computations are substantially more reducible than MLP components. Finally, wall-clock benchmarks with custom kernels on an NVIDIA A100 show that these computational savings can translate into practical runtime gains, especially at longer sequence lengths. Together, these results position RMM as a scalable direction for input-adaptive inference-time optimization.
Zixuan Lan, Yanhong Li, Jiawei Zhou
University of Chicago · Independent Researcher · Stony Brook University