CipherGenome: Homomorphic Inference for Genomic Mixture-of-Experts

Guang Yang; Fengchen Liu

cs.CRSep 27, 2026

CipherGenome: Homomorphic Inference for Genomic Mixture-of-Experts

Authors: Guang Yang, Fengchen Liu

Organizations: University of California, Los Angeles · University of California, Berkeley

Abstract

Genome foundation models are growing into sparse mixture-of-experts (MoE) networks whose expert weights no longer fit on the machines that hold the sequences, yet sending a private genome to rented accelerators exposes it: we show that a single server hosting one expert recovers the input nucleotides with 99.8% top-1 accuracy. We present CipherGenome, a protocol that keeps the embedding, attention and router of a 15.1B-parameter MoE genome model on a trusted thin client and outsources every expert projection, 95.8% of the parameters, to untrusted and possibly colluding GPU servers under module-LWE encryption. The design exploits three structural facts: expert layers are linear between two SwiGLU gates, expert weights are public, and GPU integer tensor cores can evaluate a ciphertext-weight product exactly modulo $2^{48}$ in a single GEMM. The client evaluates the nonlinearity exactly and re-encrypts with fresh secrets, so no polynomial approximation or bootstrapping is ever needed. On 72 windows from 12 bacterial genomes, encryption adds $2.54 \times 10^{-4}$ nats per token of KL divergence (95% CI upper bound $3.95 \times 10^{-4}$ ), below a pre-registered non-inferiority margin and indistinguishable from bf16 inference, while the same inversion attack falls to chance level. A reusable public hint cuts end-to-end latency by 3.54 times, wire compression reduces traffic 6.8 times, per-layer padding reduces routing leakage from 54.9% to 8.9% accuracy, and HE-compatible int4 experts remain non-inferior to their plaintext counterparts. Per expert and token, the server-side cost is more than six orders of magnitude below a CKKS baseline.

Figures & tables

Figure 1: CipherGenome at a glance. A trusted thin client keeps the sequence, the keys and the 4% of weights outside the experts. Routed tokens leave only as module-LWE ciphertexts. Untrusted, possibly colluding servers holding the public expert weights compute one exact integer GEMM per job. The client evaluates SwiGLU exactly and re-encrypts. Headline numbers are from Section 4 .

Figure 2: Fidelity and confidentiality. (a) KL divergence of next-token distributions to the fp32 model, 72 windows; bf16 is standard local inference, exact evaluates experts in fp64. (b) KL added by encryption relative to exact experts, with 95% cluster-bootstrap CIs and the pre-registered

5\times 10^{-4}

margin. (c) Top-1 token recovery by a server that hosts one expert and trains a decoder on its inputs, mean over three seeds; plaintext jobs versus CipherGenome jobs.

Figure 3: Latency. (a) Median seconds per 1024-token window at batch size 1 for the multi-process system on the same two H100 GPUs, 12 windows per arm; green labels give the speedup over compact mode. Plain EP is expert parallelism without encryption. (b) Seconds per token for one expert on a CKKS baseline (Microsoft SEAL via TenSEAL, CPU, lower bound by tile counting) and for the CipherGenome server job (H100).

System	Primitive	Nonlinearity	Client interaction	Colluding servers	MoE
THE-X, NEXUS, THOR	CKKS	polynomial approx.	none	n/a (one server)	no
BumbleBee, PUMA, BOLT	2PC	approx. + protocols	many rounds	no	no
CryptoMoE, SecMoE	2PC	approx. + protocols	many rounds	no	yes
Two-server sharing	additive shares	exact (client)	2 per layer	no	any
CipherGenome (ours)	module-LWE	exact (client)	2 per layer	yes	yes

Table 1: Design comparison with representative private transformer inference systems. CipherGenome is the only design that keeps nonlinearities exact, needs no trusted second party, and tolerates collusion of all servers; the price is that weights must be public, which is the norm for released genome models.

Figure 4: Traffic and routing leakage. (a) Online megabytes per token before and after wire compression for int16 and int4 experts, in compact mode with batch 8 and in hint mode; every compressed configuration is non-inferior to its plaintext model on 72 windows. The dashed line is unprotected expert parallelism with the same client-side SwiGLU. (b) Accuracy of a 16-way taxon classifier that sees only per-expert token counts, against the number of slots sent relative to compact mode,

T=1024

Figure 5: Quantized deployment. (a) KL divergence between the CipherGenome model and the plaintext model with the same public quantized experts, fp32 backbone, 72 windows, with 95% CIs and the non-inferiority margin. (b) Server job speedup over int16 at

Q=48

for the up/gate and down projections at 4096 tokens; all kernels are bit-exact.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Estimated security of

n=2048

module-LWE with ternary secrets and

\sigma=3.24

as a function of the modulus. The deployed moduli are

Q=48

(int16 experts),

Q=40

(int8) and

Q=32

(int4).

$\log_{2}q$	BKZ block size $b$	Primal, BKZ-sieve	Primal, core-SVP (classical)	Dual, core-SVP
24	984	318.6	287.3	366.0
32	689	232.5	201.2	246.7
40	513	181.1	149.8	179.3
48	399	147.9	116.5	136.8

Appendix

Table 2: Concrete security estimates for

n=2048

(bits). The BKZ-sieve column follows the cost model of the HE standard tables.

Accession	Organism	Accession	Organism
NC_000913.3	Escherichia coli K-12 MG1655	NC_002516.2	Pseudomonas aeruginosa PAO1
NC_000915.1	Helicobacter pylori 26695	NC_003197.2	Salmonella enterica Typhimurium LT2
NC_000962.3	Mycobacterium tuberculosis H37Rv	NC_003210.1	Listeria monocytogenes EGD-e
NC_000964.3	Bacillus subtilis 168	NC_003888.3	Streptomyces coelicolor A3(2)
NC_002505.1	Vibrio cholerae N16961 chr. I	NC_007795.1	Staphylococcus aureus NCTC 8325
NC_009089.1	Clostridioides difficile 630	NC_016845.1	Klebsiella pneumoniae HS11286

Appendix

Table 3: Evaluation genomes.

Shape (out $\times$ in)	$S$	int16, $Q{=}48$ (11)	int8, $Q{=}40$ (5)	int4, $Q{=}36$ (5)	int4, $Q{=}32$ (4)
up/gate, $16384\times 3072$	256	6.49	3.63	3.64	3.03
	1024	8.52	4.86	4.74	3.89
	4096	16.90	9.27	9.26	7.50
down, $3072\times 8192$	256	6.56	5.01	5.00	4.12
	1024	8.54	6.42	6.41	5.29
	4096	16.48	12.30	12.31	10.01

Appendix

Table 4: Server job time (ms, H100) including mask expansion, GEMM and modulus switch, for

S

tokens per job. Units are int8 GEMM units after limb-pair pruning. All results are bit-exact.

Arm	KL to fp32 model	Added KL vs exact [95% CI]	Top-1 agreement	Non-inferior
bf16 local inference	0.0033	$3.63\times 10^{-4}$ [ $1.19\times 10^{-4}$ , $6.40\times 10^{-4}$ ]	94.4	no
Exact fp64 experts	0.0030	$0.00\times 10^{0}$ [ $0.00\times 10^{0}$ , $0.00\times 10^{0}$ ]	94.5	yes
Plaintext int16 experts	0.0032	$1.95\times 10^{-4}$ [ $3.48\times 10^{-5}$ , $3.80\times 10^{-4}$ ]	94.4	yes
CipherGenome, $Q^{\prime}{=}24$ (default)	0.0032	$2.54\times 10^{-4}$ [ $1.21\times 10^{-4}$ , $3.95\times 10^{-4}$ ]	94.3	yes
CipherGenome, $Q^{\prime}{=}28$	0.0030	$6.48\times 10^{-5}$ [ $-3.22\times 10^{-5}$ , $1.56\times 10^{-4}$ ]	94.4	yes
Exact experts, gold routing	0.0004	$-2.58\times 10^{-3}$ [ $-3.17\times 10^{-3}$ , $-2.02\times 10^{-3}$ ]	95.9	yes

Appendix

Table 5: Fidelity on 72 windows. KL to the fp32 model and the KL added by encryption relative to exact fp64 experts, with 95% cluster-bootstrap CIs.

Layer	Expert	Plaintext job	CipherGenome job	Wilson upper bound	Majority baseline	Pass
0	0	99.8	1.20	1.56	0.94	yes
0	3	96.6	0.42	0.66	0.64	yes
6	0	61.9	0.27	0.46	0.63	yes
6	3	69.9	0.22	0.40	0.30	yes
12	0	49.4	0.26	0.42	0.28	yes
12	3	41.8	0.19	0.35	0.18	yes

Appendix

Table 6: Token recovery top-1 (%) by a server hosting one expert, mean over seeds, best of two decoders. Positive controls at layer 0: reused secret 98.8%, zero secret 98.8%.

Figure 7: Accuracy of a 12-way genome classifier from routing information visible at different granularities, as a function of the request length; best of logistic regression and MLP, pooled over three seeds.

Tokens	Policy	Accuracy (%)	95% CI	Slots vs compact
1024	Compact (256-slot blocks)	54.9	[44.2, 65.5]	1.00
	Layermax, 256-slot blocks	8.9	[3.6, 15.2]	1.38
	Layermax, 16-token columns	25.0	[19.1, 31.5]	0.95
	Oblivious	6.25 (chance)	by construction	2.71
4096	Compact (256-slot blocks)	78.8	[71.5, 85.9]	1.00
	Layermax, 256-slot blocks	23.3	[16.2, 32.8]	1.22

Appendix

Table 7: 16-way genome identification from routing counts under padding policies (chance 6.25%), with slots sent relative to compact mode.

Configuration	Latency, batch 1	Latency, batch 4	MB/token, batch 1	MB/token, batch 4
Plaintext expert parallelism (no privacy)	1.33	1.53	2.95	2.95
CipherGenome, compact	24.28	9.57	31.32	12.41
CipherGenome, hint	6.85	5.37	8.94	6.81
CipherGenome, hint + layermax	8.23	6.43	12.09	7.87
CipherGenome, hint, int8 AWQ+GPTQ, $Q{=}32$	5.03	4.56	7.30	5.60

Appendix

Table 8: End-to-end latency (median seconds per 1024-token window) and online traffic, 12 windows per arm.

Experts	$Q$	$n$	$g$	$Q^{\prime}_{A}/Q^{\prime}$	$t$	$b$	KL [95% CI]	Expert error	MB/tok compact	MB/tok hint
int16	48	2048	256	24/24	0	1	$1.5\times 10^{-4}$ [ $1.1\times 10^{-4}$ , $2.0\times 10^{-4}$ ]	$2.9\times 10^{-4}$	31.34	8.92
int16	48	2048	16	24/24	0	1	$1.5\times 10^{-4}$ [ $9.8\times 10^{-5}$ , $2.1\times 10^{-4}$ ]	$2.9\times 10^{-4}$	28.64	6.22
int16	48	2048	16	24/22	0	1	$1.7\times 10^{-4}$ [ $1.1\times 10^{-4}$ , $2.5\times 10^{-4}$ ]	$2.9\times 10^{-4}$	28.40	5.98
int16	48	2048	16	24/20	0	1	$1.5\times 10^{-4}$ [ $1.1\times 10^{-4}$ , $2.1\times 10^{-4}$ ]	$3.1\times 10^{-4}$	28.16	5.74
int16	48	2048	16	24/22	8	1	$1.4\times 10^{-4}$ [ $1.1\times 10^{-4}$ , $1.9\times 10^{-4}$ ]	$3.0\times 10^{-4}$	27.84	5.43
int16	48	2048	256	24/24	0	8	$1.9\times 10^{-4}$ [ $1.1\times 10^{-4}$ , $2.7\times 10^{-4}$ ]	$2.9\times 10^{-4}$	9.22	6.41

Appendix

Table 9: Wire compression, 72 windows.

g

: token granularity;

Q^{\prime}_{A}

Q^{\prime}

: downlink widths of the

A

and

B

parts;

t

: uplink bits rounded away;

b

: windows per batch. KL is to the plaintext model with the same public experts.

Experts	Method	$Q/Q^{\prime}$	KL [95% CI]	Top-1 (%)	Non-inferior
int8	RTN	40/24	$1.5\times 10^{-4}$ [ $9.1\times 10^{-5}$ , $2.1\times 10^{-4}$ ]	99.72	yes
int8	AWQ	40/24	$1.0\times 10^{-4}$ [ $6.2\times 10^{-5}$ , $1.5\times 10^{-4}$ ]	99.86	yes
int8	GPTQ	40/24	$1.8\times 10^{-4}$ [ $1.4\times 10^{-4}$ , $2.2\times 10^{-4}$ ]	99.64	yes
int8	AWQ+GPTQ	40/24	$8.0\times 10^{-5}$ [ $5.2\times 10^{-5}$ , $1.1\times 10^{-4}$ ]	99.85	yes
int8	AWQ	32/24	$3.0\times 10^{-4}$ [ $1.8\times 10^{-4}$ , $4.4\times 10^{-4}$ ]	99.55	yes
int8	AWQ+GPTQ	32/24	$1.9\times 10^{-4}$ [ $1.4\times 10^{-4}$ , $2.5\times 10^{-4}$ ]	99.58	yes

Appendix

Table 10: CipherGenome with quantized experts against the plaintext model with the same experts, fp32 backbone, 72 windows.

Experiment	Primary endpoint	Control
Correctness gate	Eleven unit checks: ring products, encryption round trip, GEMM bit-exactness, modulus switch, byte accounting	All must pass
Fidelity ( Section 4.1 )	KL added by encryption vs exact experts; non-inferior if CI upper bound $<5\times 10^{-4}$	Known-bad $Q^{\prime}{=}16$ must fail
Inversion ( Section 4.1 )	Ciphertext-view top-1 per (layer, expert) cell	Reused and zero secrets must succeed
Routing leakage ( Section 4.4 )	Genome accuracy from block counts, $T=1024$	Chance level
System ( Section 4.2 )	Median latency ratio compact over hint, paired by window	Same GPUs for all arms
Quantized experts ( Section 4.5 )	KL to same quantized plaintext model	Known-bad $Q^{\prime}{=}14$ must fail

Appendix

Table 11: Experiments, primary endpoints and controls.

Mixture-of-Experts Inference

Mixture-of-experts inference studies how to run sparsely activated expert models efficiently while preserving their quality.

See all 164 papers

GenomeOcean Anywhere: Private WebGPU Inference for Genome MoEs
Sep 27, 2026Guang Yang, Fengchen LiuGenomic Foundation Models Distributed Inference
University of California, Los Angeles · University of California, Berkeley
Good-Enough LLM Obfuscation (GELO)
Mar 5, 2026Anatoly Belikov, Ilya FedotovPrivacy-Preserving Language Models LLM Security
SingularityNET Foundation, Baarerstrasse 141, Zug, 6300, Switzerland · Singularity Compute
HEAT: Faster Fully Homomorphic Inference via Approximations-Weights Co-Adaptation
Sep 1, 2026Alessandro Zirilli, Davide Marincione, Evgenios M. Kornaropoulos +2Privacy-Preserving Language Models Homomorphic Encryption
Sapienza University of Rome · George Mason University · Paradigma

Sep 27, 2026J/K move · Enter open · S save

GenomeOcean Anywhere: Private WebGPU Inference for Genome MoEs

Guang Yang, Fengchen Liu

University of California, Los Angeles · University of California, Berkeley

Genomic Foundation Models Distributed Inference Privacy-Preserving Language Models Mixture-of-Experts Inference Genomics

Open paper AskPDF arXiv

Genome foundation models are most useful where sequences are generated, yet the largest models need datacenter accelerators and a place to send private DNA. We ask whether a 15-billion-parameter genome mixture-of-experts (MoE) model can instead run on volunteers' web browsers, with the experts spread across many untrusted devices, without changing its predictions and without revealing the sequence to any single device. We build a system in which a trusted coordinator runs attention and routing while browser workers run every expert feed-forward network through hand-written WebGPU kernels, and we protect the expert inputs with real-valued Lagrange coded computing: each worker receives only a Gaussian-padded share, computes the expert's linear maps, and the coordinator decodes from any two of three workers. On GenomeOcean-MoE (8 experts, top-2 routing, 24 layers), the browser path matches native llama.cpp at every quantization level, the distributed path stays at the BF16 numerical noise floor (KL 0.0036 nats per token), and an unfitted latency model predicts decode time within 0.74% (median) under emulated wide-area links. We first show that plaintext expert inputs are not private: a probe recovers the token from a single vector at every depth, and one worker can identify the source genome from 300 unordered tokens with 92% accuracy. With coded experts, an adaptive attacker trained on shares falls to the most-frequent-token baseline, one worker's information about each token is bounded below one bit per forward pass, and the fidelity cost stays below the BF16 noise floor; in Chrome, coded decoding runs at 220 to 376 ms per token, depending on how much of the routing is hidden, and continues without replicas when a worker fails.

Abstract

Mixture-of-Experts Inference

Explore similar work

GenomeOcean Anywhere: Private WebGPU Inference for Genome MoEs

Good-Enough LLM Obfuscation (GELO)

HEAT: Faster Fully Homomorphic Inference via Approximations-Weights Co-Adaptation