cs.CRNov 3, 2026

Perseus: A Full-Stack System for Homomorphic Model Inference

Authors: Alessandro Zirilli, Davide Marincione, Evgenios M. Kornaropoulos, Giuseppe Ateniese, Emanuele Rodolà

Organizations: Sapienza University of Rome · Paradigma · George Mason University

Abstract

Large Language Models (LLMs) have made it exceptionally convenient for users to send their private data to model providers, which process them in plaintext. Fully Homomorphic Encryption (FHE) promises to secure remote inference by computing directly on encrypted data, yet it has remained prohibitively slow. The effort to make it practical remains fragmented, with most existing frameworks being closed-source, which keeps progress from compounding. We present Perseus, a free and open Python software library that supports the full stack of encrypted model inference, from the cryptographic backend and data packing, to encrypted nonlinearities and end-to-end model execution. Perseus ships with built-in novel optimizations, one at each level of the stack: in-kernel regeneration of key-switching keys at the (i) backend level, a complex variant of Cachemir packing at the (ii) data level, and a state-of-the-art bootstrap placer that is both packing- and magnitude-aware at the (iii) model level. Perseus achieves state-of-the-art latency across multiple models and modalities, within 3%3\% of the plaintext models' accuracy and perplexity. Remarkably, it is 3.96×3.96\times faster at single-token decoding than any other implementation on Llama-3-8B. We initiate, for the first time, a public speedrun on encrypted GPT-2, so that progress can compound.

Explore similar work

CardsList
  1. HEAT: Faster Fully Homomorphic Inference via Approximations-Weights Co-Adaptation

    Sep 1, 2026Alessandro Zirilli, Davide Marincione, Evgenios M. Kornaropoulos +2Privacy-Preserving Language ModelsHomomorphic Encryption

  2. Cerium: A Multi-GPU Framework for Terabyte-Scale Encrypted Inference

    Dec 12, 2025Siddharth Jayashankar, Joshua Kim, Michael B. Sullivan +2Homomorphic EncryptionGPU Kernel Optimization

  3. Pacifo: An Efficient Fully Homomorphic Encryption-based Lightweight ResNet for Trustworthy Machine Learning as a Service

    Oct 24, 2026Yang Li, Gang He, Ruichen Zhang +2Privacy-Preserving MLEfficient Neural Network Inference