Secure Multi-party Computation (MPC) enables untrusted parties to jointly compute a function without revealing their inputs. Its application to machine learning (ML) has gained significant attention, particularly for secure inference services deployed across multiple cloud virtual machines (VMs), where each VM acts as an MPC party. Model providers secret-share model weights, and users secret-share inputs, ensuring that each server operates only on random shares. While MPC provides strong cryptographic guarantees, it incurs substantial computational and communication overhead. Deep neural networks rely heavily on convolutional and fully connected layers, which require costly matrix multiplications in MPC. To reduce this cost, we propose leveraging low-rank decomposition (LRD) for linear layers, replacing one large matrix multiplication with two smaller ones. Each matrix multiplication in MPC incurs a round of communication, meaning decomposing one matrix multiplication into two leads to an additional communication round. Second, the added matrix multiplication requires an additional truncation step to maintain numerical precision. Since truncation itself requires communication and computation, these overheads can offset the gains from decomposition. To address this, we introduce two complementary optimizations: truncation skipping and efficient linear layer concatenation. Truncation skipping removes the extra truncation induced by LRD, while linear layer concatenation pipelines operations to hide the additional communication round. Together, these techniques mitigate the main overheads of LRD in MPC and improve overall efficiency. Our approach is broadly applicable across MPC protocols. Experiments show up to 25% speedup in n-PC and 33% in 3-PC protocols over full-rank baselines, along with up to 52% GPU energy savings and 88% reduction in offline-phase latency.
Figures & tables
Figure 1. n -PC system topology.
Figure 3. Trio ( Harth-Kitzerow et al., 2025 ) system topology.
Protocol
Runtime
Power
VGG
42.8%
72.4%
BERT LARGE
51.6%
67.8%
GCN
36.1%
85.3%
Table 1. Linear layer runtime (%) and power (%) in MPC inference online phase (5Gbps WAN) for n -PC .
Figure 6. Computation and communication workflow for n -PC and Trio matrix multiplication; boxes labeled with a router icon denote communication runtime, and boxes labeled with a GPU icon denote computation runtime.
Figure 7. Computation and communication workflow of low-rank matrix multiplication for n -PC and Trio protocol.
Figure 8. Computation and communication workflow of low-rank matrix multiplication for n -PC and Trio protocol with truncation skipping.
Figure 9. Trio online phase matrix multiplication computation and communication workflow.
Figure 10. n -PC online phase matrix multiplication computation and communication workflow with efficient layer concatenation.
Figure 11. LRD-MPC online phase performance improvement for Trio.
Figure 12. LRD-MPC online phase performance improvement for n -PC .
Protocol
Model
LAN
MAN
WAN
Trio
GCN
44.8%
44.45%
30.23%
WideResNet18
37.49%
28.06%
14.95%
VGG19
11.36%
10.92%
5.92%
BERT LARGE
39.64%
37.91%
32.99%
n -PC
GCN
37.41%
36.01%
13.96%
WideResNet18
174.76%
125.64%
20.83%
Table 2. Speedup of LRD-MPC on different network setups.
Model
Saving (%)
GCN
30.26%
WideResNet18
52.24%
VGG19
48.41%
BERT LARGE
34.09%
Table 3. LRD-MPC online phase GPU power consumption saving (%) for n -PC