cs.CRNov 24, 2025

CrypTorch: Unified PyTorch-based Auto-tuning Compiler for Private Machine Learning Inference

Authors: Jinyu Liu, Gang Tan, Kiwan Maeng

Organizations: The Pennsylvania State University, University Park, PA, USA

Abstract

MPC-based ML uses secure multi-party computation (MPC) to execute machine learning (ML) workloads across multiple parties without requiring any party to reveal its private data or model parameters. Our characterization study finds that the performance of state-of-the-art MPC frameworks is often dominated by operator-approximation strategies, where ML operators not natively supported by MPC are replaced with iterative methods or polynomials. These approximations are typically chosen heuristically by each framework and applied globally. As a result, they can be unnecessarily expensive in some cases, yet insufficiently accurate in others. We find that the best approximation choice depends on the context, target framework, and application, making a single globally fixed approximation fundamentally suboptimal. To address this problem, we design CrypTorch, a multi-stage compiler for MPC-based ML that supports multiple MPC frameworks as backends. At the core of CrypTorch is its auto-tuning stage, which extracts cost estimates from the target MPC backend and quality estimates from a proxy PyTorch execution, and uses them to automatically select a set of approximations that maximizes performance while satisfying the given quality constraint. We show that CrypTorch's auto-tuner improves performance across multiple MPC backends by 1.14--1.60x without degrading accuracy, and by 1.17--1.92x with less than 3% accuracy degradation. CrypTorch's unified design also improves programmability and usability by allowing multiple backends to share other common compiler functionality and optimizations. We further show that the auto-tuner extends to a related fully homomorphic encryption (FHE) backend, reducing the number of bootstrapping operations by 1.31--1.65x.

Figures & tables

Explore similar work

Oct 16, 2026

TorchTPU - A Hybrid Eager/Compiled Architecture for Native PyTorch Execution on TPU

TorchTPU is an open-source collaboration between Google and Meta that introduces a native and principled PyTorch backend for Google TPUs. It enables seamless out-of-the-box model execution and TPU migration while delivering industry-competitive inference and training performance through eager execution optimizations, torch.compile integration, and native hardware acceleration.
Nov 1, 2026

CipherFlow: Compiling AI Programs for Efficient Cryptographically Secure Inference

Cryptographically secure inference faces two practical challenges: making secure programming accessible to AI developers and data scientists, and achieving efficient execution across deployment environments. Developers must compose cryptographic primitives correctly, while performance depends on hardware, networks, and evolving ciphertext states. We present CipherFlow, a compiler framework that takes a high-level AI computation graph, available cryptographic primitives, and a target hardware and network configuration to generate an executable, latency-optimized secure inference program. CipherFlow automates cryptographic execution decisions through three techniques: MicroBench profiles deployment-specific primitive costs; state-expanded planning jointly optimizes homomorphic encryption (HE) and secure multi-party computation (MPC) placement, conversions, and ciphertext maintenance under cryptographic-state constraints; and BlockFold summarizes branched regions to scale planning to Transformer graphs. Evaluation using measured HE/MPC backend costs across CPU/GPU and network settings shows up to 16.5×\times speedup on BERT-base and over 22×\times on ViT compared with static hybrid placement baselines. CipherFlow separates model programming from cryptographic execution decisions, enabling developers to express AI computations while the compiler constructs efficient secure programs.
Feb 16, 2026

LRD-MPC: Efficient MPC Inference through Low-rank Decomposition

Secure Multi-party Computation (MPC) enables untrusted parties to jointly compute a function without revealing their inputs. Its application to machine learning (ML) has gained significant attention, particularly for secure inference services deployed across multiple cloud virtual machines (VMs), where each VM acts as an MPC party. Model providers secret-share model weights, and users secret-share inputs, ensuring that each server operates only on random shares. While MPC provides strong cryptographic guarantees, it incurs substantial computational and communication overhead. Deep neural networks rely heavily on convolutional and fully connected layers, which require costly matrix multiplications in MPC. To reduce this cost, we propose leveraging low-rank decomposition (LRD) for linear layers, replacing one large matrix multiplication with two smaller ones. Each matrix multiplication in MPC incurs a round of communication, meaning decomposing one matrix multiplication into two leads to an additional communication round. Second, the added matrix multiplication requires an additional truncation step to maintain numerical precision. Since truncation itself requires communication and computation, these overheads can offset the gains from decomposition. To address this, we introduce two complementary optimizations: truncation skipping and efficient linear layer concatenation. Truncation skipping removes the extra truncation induced by LRD, while linear layer concatenation pipelines operations to hide the additional communication round. Together, these techniques mitigate the main overheads of LRD in MPC and improve overall efficiency. Our approach is broadly applicable across MPC protocols. Experiments show up to 25% speedup in n-PC and 33% in 3-PC protocols over full-rank baselines, along with up to 52% GPU energy savings and 88% reduction in offline-phase latency.