CipherFlow: Compiling AI Programs for Efficient Cryptographically Secure Inference
Organizations: LoopLeap Lab
Abstract
Cryptographically secure inference faces two practical challenges: making secure programming accessible to AI developers and data scientists, and achieving efficient execution across deployment environments. Developers must compose cryptographic primitives correctly, while performance depends on hardware, networks, and evolving ciphertext states. We present CipherFlow, a compiler framework that takes a high-level AI computation graph, available cryptographic primitives, and a target hardware and network configuration to generate an executable, latency-optimized secure inference program. CipherFlow automates cryptographic execution decisions through three techniques: MicroBench profiles deployment-specific primitive costs; state-expanded planning jointly optimizes homomorphic encryption (HE) and secure multi-party computation (MPC) placement, conversions, and ciphertext maintenance under cryptographic-state constraints; and BlockFold summarizes branched regions to scale planning to Transformer graphs. Evaluation using measured HE/MPC backend costs across CPU/GPU and network settings shows up to 16.5 speedup on BERT-base and over 22 on ViT compared with static hybrid placement baselines. CipherFlow separates model programming from cryptographic execution decisions, enabling developers to express AI computations while the compiler constructs efficient secure programs.
Figures & tables
| Ablation | CPU latency ratio | GPU latency ratio |
|---|---|---|
| Full | 1.00 | 1.00 |
| MicroBench | 1.19 | 19.14 |
| StateExp | 1.00 | 1.00 |
| BlockFold | 1.00 | 1.00 |
| Bootstrap | 1.00 | 3.88 |
| Greedy per-operator | 1.37 | 5.75 |
| Ablation | Compilation time | Ratio vs. Full | Method |
|---|---|---|---|
| Full | 3.50 s | 1.00 | empirical |
| MicroBench | 3.39 s | 0.96 | empirical |
| StateExp | 2.36 s | 0.67 | empirical, infeasible |
| BlockFold | 278 h | search-space extrapolation |
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Refresh mechanism | 10 Mbps | 100 Mbps | 1 Gbps | 3 Gbps |
|---|---|---|---|---|
| HE MPC HE round trip | 63 s | 6.33 s | 0.672 s | 0.252 s |
| Direct bootstrap, CPU (51.6 s) | bootstrap | round trip | round trip | round trip |
| Direct bootstrap, GPU (1.69 s) | bootstrap | bootstrap | round trip | round trip |
| Platform | BW | RTT | Full | Bootstrap | Slowdown |
| GPU | 10 Mbps | 1 ms | 472 s | 1.83 ks | |
| GPU | 10 Mbps | 20 ms | 472 s | 1.83 ks | |
| GPU | 10 Mbps | 40 ms | 472 s | 1.83 ks | |
| GPU | 10 Mbps | 80 ms | 472 s | 1.83 ks | |
| GPU | 100 Mbps | 1 ms | 472 s | 582 s | |
| GPU | 100 Mbps | 20 ms | 472 s | 583 s |
| Kernel | Device | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| MatMul | CPU | 1 / 1 | 1 | 1.00 | 1.55 | 2.84 | 6.06 | 10.01 | 14.87 | 20.42 | |
| Softmax | CPU | 8 / 16 | 18 | – | – | – | – | – | 0.74 | 1.00 | 1.29 |
| LayerNorm | CPU | 3 / 18 | 18 | – | – | – | – | – | – | 1.00 | 1.23 |
| GeLU | CPU | 4 / 18 | 18 | – | – | – | – | – | – | 1.00 | 1.27 |
| MatMul | GPU | 1 / 1 | 1 | 1.00 | 1.17 | 1.67 | 2.70 | 3.70 | 4.49 | 5.36 | |
| Softmax | GPU | 8 / 16 | 16 | – | – | – | – | – | 1.00 | 1.20 | 1.39 |
| Region | Operator | Action | Before | After | Cost (s) |
|---|---|---|---|---|---|
| QK | MPC exec | (MPC) | (MPC) | 3.28 | |
| input of Softmax | MPC HE | (MPC) | (HE, 20) | 12.61 | |
| Softmax | HE exec | (HE, 20) | (HE, 12) | 0.26 | |
| AV | HE exec | (HE, 12) | (HE, 11) | 3.75 | |
| Out-Proj | HE exec | (HE, 11) | (HE, 10) | 3.75 | |
| skip operand of Add 1 | MPC HE | (MPC) | (HE, 20) | 6.31 |
| Transition type | State transition | Constraint | Cost |
|---|---|---|---|
| HE execution | |||
| MPC execution | - | ||
| HE MPC conversion | - | ||
| MPC HE conversion | - | ||
| Bootstrapping | - |
| BW | RTT | BB sys | BB map | BOLT | BLB | Ours | Speedup | Speedup fair | Mix |
|---|---|---|---|---|---|---|---|---|---|
| 10 Mbps | 1 ms | 10.8 ks | 16 ks | 14.4 ks | 12.8 ks | 7.93 ks | 12/120 | ||
| 10 Mbps | 20 ms | 22 ks | 16 ks | 14.4 ks | 12.9 ks | 7.95 ks | 12/120 | ||
| 10 Mbps | 40 ms | 33.7 ks | 16.1 ks | 14.5 ks | 12.9 ks | 7.97 ks | 12/120 | ||
| 10 Mbps | 80 ms | 57.2 ks | 16.1 ks | 14.5 ks | 12.9 ks | 8 ks | 12/120 | ||
| 100 Mbps | 1 ms | 2.67 ks | 9.18 ks | 9.02 ks | 8.86 ks | 1.27 ks | 0/132 | ||
| 100 Mbps | 20 ms | 13.8 ks | 9.19 ks | 9.02 ks | 8.86 ks | 1.3 ks | 0/132 |
| BW | RTT | BB sys | BB map | BOLT | BLB | Ours | Speedup | Speedup fair | Mix |
|---|---|---|---|---|---|---|---|---|---|
| 10 Mbps | 1 ms | 10.8 ks | 10.9 ks | 9.34 ks | 7.75 ks | 472 s | 132/0 | ||
| 10 Mbps | 20 ms | 22 ks | 11 ks | 9.36 ks | 7.76 ks | 472 s | 132/0 | ||
| 10 Mbps | 40 ms | 33.7 ks | 11 ks | 9.37 ks | 7.78 ks | 472 s | 132/0 | ||
| 10 Mbps | 80 ms | 57.2 ks | 11 ks | 9.41 ks | 7.81 ks | 472 s | 132/0 | ||
| 100 Mbps | 1 ms | 2.67 ks | 1.46 ks | 1.3 ks | 1.14 ks | 472 s | 132/0 | ||
| 100 Mbps | 20 ms | 13.8 ks | 1.48 ks | 1.32 ks | 1.16 ks | 472 s | 132/0 |
| BW | RTT | BumbleBee | BOLT | BLB | Ours | Speedup | Mix HE/MPC |
|---|---|---|---|---|---|---|---|
| 10 Mbps | 1 ms | 28.2 ks | 25.4 ks | 22.6 ks | 13 ks | 12/120 | |
| 10 Mbps | 20 ms | 28.2 ks | 25.4 ks | 22.6 ks | 13 ks | 12/120 | |
| 10 Mbps | 40 ms | 28.2 ks | 25.4 ks | 22.6 ks | 13 ks | 12/120 | |
| 10 Mbps | 80 ms | 28.2 ks | 25.4 ks | 22.7 ks | 13 ks | 12/120 | |
| 100 Mbps | 1 ms | 15.9 ks | 15.6 ks | 15.3 ks | 2.05 ks | 0/132 | |
| 100 Mbps | 20 ms | 15.9 ks | 15.6 ks | 15.3 ks | 2.08 ks | 0/132 |
| BW | RTT | BumbleBee | BOLT | BLB | Ours | Speedup | Mix HE/MPC |
|---|---|---|---|---|---|---|---|
| 10 Mbps | 1 ms | 19.1 ks | 16.3 ks | 13.5 ks | 789 s | 132/0 | |
| 10 Mbps | 20 ms | 19.1 ks | 16.3 ks | 13.5 ks | 789 s | 132/0 | |
| 10 Mbps | 40 ms | 19.1 ks | 16.3 ks | 13.5 ks | 789 s | 132/0 | |
| 10 Mbps | 80 ms | 19.1 ks | 16.3 ks | 13.6 ks | 789 s | 132/0 | |
| 100 Mbps | 1 ms | 2.54 ks | 2.26 ks | 1.99 ks | 789 s | 132/0 | |
| 100 Mbps | 20 ms | 2.55 ks | 2.28 ks | 2 ks | 789 s | 132/0 |
| Primitive family | Warmups | Repeats | Timing method |
|---|---|---|---|
| HE operators | 1 | 3 | wall-clock |
| HE MPC conversions | 2 | 5 | wall-clock |
| MPC operators on CPU | 1 | 3 | wall-clock |
| MPC operators on GPU | 1 | 3 | CUDA event |
| Model | Platform | Greedy | Random best | Random median | EncFormer-style |
|---|---|---|---|---|---|
| BERT-base | CPU | 1.02–1.37 | 1.63–10.92 | 1.86–16.31 | 1.61–21.22 |
| BERT-base | GPU | 1.01–5.75 | 1.74–14.02 | 2.08–18.58 | 2.42–16.53 |
| ViT-B/16 | CPU | 1.02–1.43 | 1.64–11.34 | 1.92–17.02 | 1.74–22.65 |
| ViT-B/16 | GPU | 1.02–5.39 | 1.88–13.94 | 2.27–18.57 | 2.52–17.21 |
| Operator / action | Domain / backend | |||
|---|---|---|---|---|
| Attention_QK | HE / CPU | 1.55e5 ms | 0 | 0 |
| Attention_QK | MPC / CPU | 211 ms | 6.2 MiB | 1 |
| Softmax | HE / CPU | 5.98e4 ms | 0 | 0 |
| Softmax | MPC / CPU | 2.72e3 ms | 79.8 MiB | 11 |
| GeLU | MPC / CPU | 1.05e3 ms | 30.9 MiB | 4 |
| HE MPC | Conv. / CPU | 10.7 ms | 37.5 MiB | 1 |
| Platform / network | Ours latency | Ours | BumbleBee | BOLT | BLB | All-MPC |
|---|---|---|---|---|---|---|
| GPU, 10 Mbps / 1 ms | 472 s | 0.0 | 12.3 | 10.4 | 8.5 | 10.5 |
| GPU, 3 Gbps / 80 ms | 127 s | 9.4 | 12.3 | 10.4 | 8.5 | 10.5 |
| CPU, 10 Mbps / 1 ms | 7.93 ks | 7.9 | 8.9 | 7.0 | 5.2 | 10.5 |
| CPU, 1 Gbps / 20 ms | 486 s | 10.5 | 8.9 | 7.0 | 5.2 | 10.5 |
Privacy-Preserving ML
Privacy-preserving machine learning develops methods to learn, share, and run models while limiting exposure of sensitive data.
See all 445 papers