GPU-Accelerated Non-Interactive Encrypted Neural Network Training with Batched Diagonal Packing
Organizations: Independent Researcher.
Abstract
Training a neural network under homomorphic encryption requires transposes, batch reductions, optimizer updates, and repeated refreshes in addition to forward matrix products. We combine rectangular diagonal packing, feature-major SIMD mini-batch lanes, baby-step giant-step evaluation, and compact optimizer state in an MIT-licensed Rust CKKS implementation. Its CUDA path keeps training and bootstrap ciphertext arithmetic on the GPU, while retaining host key generation and client operations. In a matched 100-update experiment on the main non-interactive MNIST model, device-resident bootstrapping reduces training time from 14,268.295 to 293.075 seconds, a ratio against our CPU implementation, with exactly equal decoded parameters and predictions. Including setup and final evaluation gives a ratio. A separate GPU run completes ten passes over 4,800 images in 2,967.095 seconds and achieves 90.35% test accuracy versus 90.41% for its plaintext twin. These are single-initialization results; exact CPU/GPU parameter agreement is verified at 100 updates, not for the full 1,000-update run. Earlier completed CPU runs support a larger-batch key-switch advantage and baseline-batch accuracy within the registered tolerance, but the baseline-batch normalized key-switch target is missed. The released ReBoot baseline does not meet the common security-estimator threshold, and its full-setting CPU time is unmeasured. We therefore distinguish measured acceleration of our implementation from conditional layout comparisons and do not claim unconditional superiority over ReBoot.
Figures & tables
| Operation | Rotations | Relins |
|---|---|---|
| BSGS encrypted matvec | ||
| Dedicated transpose | ||
| Replicated weight gradient |
| Run | Input | Placement | B | Config | Samples | hekit KS/sample | Boots | ReBoot non-boot | (a) | (b) | (c) | (d) | Ways 0.70 |
| 196 inputs, ReBoot full setting, 4800 samples x 10 epochs, seed 0 (headline) | |||||||||||||
| FULL-1 | 196 | WN | 64 | faithful | 48000 | 18.272 | 1500 | 37 (model) | 0.232 | 0.387 | 0.395 | 0.400 | abcd |
| FULL-2 | 196 | WN | 48 in 64 | faithful | 48000 | 28.990 | 4000 | 37 (model) | 0.368 | 0.781 | 0.627 | 0.534 | acd |
| 784 inputs, ReBoot encrypted-script subset, 1 epoch (supplementary) | |||||||||||||
| F1 | 784 | WN | 32 | faithful | 1008 | 33.488 | 68 | 35.0625 (Lazy, measured) | 0.435 | 0.779 | 0.756 | 0.742 | a |
| F2 | 784 | WN | 64 | faithful | 1008 | 17.885 | 36 | 35.0625 (Lazy, measured) | 0.233 | 0.414 | 0.404 | 0.397 | abcd |
| Run | Input | B | Samples | Train s | s/sample | vs ReBoot | Boot s/sample | Non-boot s/sample | Threads |
| 196 inputs, full setting (headline) | |||||||||
| FULL-1 | 196 | 64 | 48000 | 72050.8 | 1.501 | not measured | 0.435 | 1.066 | rayon 12 |
| FULL-2 | 196 | 48 in 64 | 48000 | 146229.9 | 3.046 | not measured | 1.170 | 1.877 | rayon 12 |
| 784 inputs, ReBoot encrypted-script subset | |||||||||
| ReBoot official (OpenFHE) | 784 | 48 | 1008 | 3599.9 | 3.571 | 1 | 2.585 | 0.986 | OpenMP 12 |
| F1 | 784 | 32 | 1008 | 3016.7 | 2.993 | 0.838 | 0.989 | 2.003 | rayon 12 |
| Row | Setting | Seeds | Test acc (%) | Twin (%) | Reference (%) | (pt) | Gate |
| Full setting: 196 inputs, 4800 samples x 10 epochs, B = 48 (the 0.5 pt gate) | |||||||
| ReBoot released code, plaintext (recomputed twin) | full | 10 | 90.65 0.27 (n = 10) | — | — | — | reference |
| hekit plaintext twin (ReBoot weights, order, lr) | full | 10 | 90.65 0.27 (n = 10) | — | 90.65 0.27 (n = 10) | +0.00 | twin check |
| hekit encrypted, B = 48 in 64 lanes, WN | full | 1 (seed 0) | 90.35 | 90.41 | 90.41 (seed 0) | -0.06 | PASS |
| hekit encrypted, B = 64, WN | full | 1 (seed 0) | 90.36 | 90.37 | 90.41 (seed 0) | -0.05 | not the gate (B 48) |
| eMLP-2 / eMLP-3 block MLPs, full setting, seed 0 | |||||||
| Dataset | Config | Epochs | Seeds | Fixed final epoch (%) | Best epoch, test-selected (%) | ReBoot Table 4 (%) | fixed (pt) | Within 1 pt |
|---|---|---|---|---|---|---|---|---|
| MNIST 14x14 | paper setting | 50 | 10 | 95.00 0.18 (n = 10) | 95.33 0.15 (n = 10) | 94.61 0.27 | +0.39 | yes |
| Fashion-MNIST 14x14 | paper setting | 50 | 10 | 85.22 0.17 (n = 10) | 85.53 0.12 (n = 10) | 85.12 0.20 | +0.10 | yes |
| Breast Cancer | fixed split, min-max (primary) | 100 | 10 | 96.40 1.27 (n = 10) | 99.30 0.37 (n = 10) | 99.03 1.78 | -2.63 | no |
| Breast Cancer | per-run split (sensitivity) | 100 | 10 | 95.26 1.61 (n = 10) | 97.89 0.94 (n = 10) | 99.03 1.78 | -3.77 | no |
| Breast Cancer | z-score (diverges) | 100 | 10 | diverged (10/10 seeds) | — | 99.03 1.78 | — | no |
| Parameters | N | log PQ | Secret | BDGL16 | MATZOV | >= 128 (both) |
|---|---|---|---|---|---|---|
| hekit (a): n16_boot main key | 2^16 | 1732 (1731.998) | dense ternary (P( 1) = 1/4 each, h N/2) | 131.5 | 129.3 | yes |
| hekit (b): SSE key s -> s’ (h’ = 32) | 2^16 | 294 (q_0 P) | sparse ternary, h’ = 32 | 133.8 | 134.2 | yes |
| ReBoot main key (R-REB-4: OpenFHE’s own chain) | 2^16 | 1730 (1729.585) | sparse ternary, h = 192 (97 / 95) | 121.8 | 121.5 | no |
| Operation | Level | hekit GPU | CPU 1 thr | CPU 12 thr | GPU speed-up (1 / 12 thr) | HEonGPU (RTX 4090) | GPU / HEonGPU |
|---|---|---|---|---|---|---|---|
| add | 31 | 0.111 | 15.674 | 14.611 | 141x / 132x | none published | — |
| relinearize | 31 | 4.671 | 639.639 | 240.070 | 137x / 51x | 2.582 (l = 36, I = 4 primes per digit) | 1.81x (nominal) |
| mul + relin + rescale | 31 | 5.823 | 725.209 | 337.731 | 125x / 58x | none published | — |
| rotate(1) | 31 | 3.990 | 622.046 | 223.880 | 156x / 56x | none published | — |
| hoisted rotate x16 | 31 | skipped: needs 5580 MiB (2377 MiB free) | 3765.068 | 689.471 | — | none published | — |
| hoisted rotate x16 (capped keys) | 9 | 4.984 | 1005.331 | 172.870 | 202x / 35x | none published | — |
| Source | Backend / level | hoist bsgs lazy | Rotations | Relins | KS/iter | Closed form | Step median (s) | Step / default | Max | W| vs default |
|---|---|---|---|---|---|---|---|---|---|
| CPU, n16_train, level 31 | cpu, level max | 1 1 1 | 571 | 62 | 633 | match | 134.040 | 1.000 | 0 |
| CPU, n16_train, level 31 | cpu, level max | 1 1 0 | 571 | 114 | 685 | match | 137.170 | 1.023 | 4.81e-07 |
| CPU, n16_train, level 31 | cpu, level max | 1 0 1 | 610 | 101 | 711 | match | 142.124 | 1.060 | 4.14e-07 |
| CPU, n16_train, level 31 | cpu, level max | 1 0 0 | 610 | 114 | 724 | match | 143.025 | 1.067 | 3.60e-07 |
| CPU, n16_train, level 31 | cpu, level max | 0 1 1 | 571 | 62 | 633 | match | 134.069 | 1.000 | 0 |
| CPU, n16_train, level 31 | cpu, level max | 0 1 0 | 571 | 114 | 685 | match | 132.606 | 0.989 | 4.81e-07 |
| Metric | CPU | CUDA |
|---|---|---|
| Mean update incl. refresh (s) | 142.438 | 2.670 |
| Mean refresh work/update (s) | 55.430 | 1.859 |
| One-epoch training (s) | 14268.295 | 293.075 |
| Total incl. setup/evaluation (s) | 14325.222 | 352.247 |
| Test accuracy (%) | 81.92 | 81.92 |
Homomorphic Encryption
Homomorphic encryption lets data be computed on while encrypted, supporting privacy-preserving machine learning, analytics, and inference.
See all 44 papers