cs.CROct 14, 2026
SaveA Graph-Based Analysis of Comparability, Scale, and Attribution in Private Transformer Inference
Organizations: Texas A&M University, College Station, TX, USA
Abstract
Rising privacy concerns around large language models have accelerated research in private Transformer inference (PTI). Synthesizing the empirical claims in this line of research, however, is hindered by multi-dimensional cryptographic progress and inconsistent evaluation contracts. In this work, we systematize the field by constructing an empirical graph of 62 PTI systems, mapping system attributes to node features and evaluation compatibility to edges. A three-tiered analysis of this graph illuminates the state of private inference. Node-level attribution shows that cryptographic progress drives performance gains across most systems, yet always co-occurs with other factors (model fidelity, hardware, cost accounting). Edge-level auditing shows that only 1 of 1891 candidate system pairs is directly comparable under shared assumptions. Finally, graph-level analysis shows that the scaling frontier of private inference has plateaued at 13B parameters and projects the cost of deploying current PTI techniques on production-scale Transformers. Overall, this graph-based framework equips the community to evaluate PTI claims transparently and channel technical speedups into deployment-scale private inference.
Figures & tables
TABLE I: Comparison between this SoK and prior PTI surveys and systematizations. Whereas existing work taxonomizes mechanisms, this work audits whether published evidence is mutually comparable and demonstrates progress toward production-scale private inference.
Fig. 1: Cryptographic cost breakdown of one Transformer encoder layer, drawn as in the BERT-family encoders most PTI systems evaluate. Data flows left to right with the residual bypasses drawn below. The bars above and below each operation sketch its relative cost on the HE rail (multiplicative depth, up) and the MPC rail (rounds and communication, down). Weight multiplications (blue) are comparatively cheap in depth and rounds, less so when the weights are themselves secret shared, the activation products and cost more, and the nonlinear steps (red: softmax, GELU, LayerNorm) dominate. The two rails need not agree: GELU bites hardest in MPC communication but is a shallow polynomial under HE, while LayerNorm’s inverse square root cuts deepest under HE. This op-level skew is why many reported speedups reshape the nonlinearities rather than the protocol (RQ2).
| Backbone | Encoder | Decoder |
|---|---|---|
| MPC-based | MPCFormer [ 6 ] , MPCViT [ 29 ] , RNAViT [ 48 ] , SALViT [ 30 ] , SecBERT [ 15 ] , SecFormer [ 49 ] , PrivTF [ 50 ] , Curl [ 51 ] [MM], Ditto [ 52 ] [MM], PUMA [ 4 ] [MM], PrivMLLM [ 31 ] [MM], SHAFT [ 32 ] [MM], SIGMA [ 5 ] [MM], FuseFSS [ 53 ] [MM], Mosformer [ 54 ] [MM] | Comet [ 55 ] , MARILL [ 56 ] , SecureGPT [ 57 ] , SecureTLM [ 58 ] , Curl [ 51 ] [MM], Ditto [ 52 ] [MM], MERGE [ 27 ] [MM], PUMA [ 4 ] [MM], PrivMLLM [ 31 ] [MM], SHAFT [ 32 ] [MM], SIGMA [ 5 ] [MM], FLUTE [ 59 ] , MPCache [ 60 ] , FuseFSS [ 53 ] [MM], Mosformer [ 54 ] [MM] |
| HE-based | EncryptionFriendlyLLM [ 9 ] , THEX [ 8 ] , NEXUS [ 7 ] [MM], PowerSoftmax [ 61 ] [MM], AEGIS [ 39 ] , ATLAS [ 62 ] [MM], ELLMo [ 63 ] , Euston [ 64 ] [MM], Powerformer [ 36 ] , PriTran [ 65 ] , THOR [ 34 ] | ENSI [ 66 ] , EncryptedLLM [ 67 ] , HEDecoding [ 68 ] , PracticalLLM [ 69 ] , NEXUS [ 7 ] [MM], PowerSoftmax [ 61 ] [MM], ATLAS [ 62 ] [MM], Cachemir [ 13 ] , Euston [ 64 ] [MM], ScalingPPML [ 41 ] |
| Hybrid MPC–HE | BOLT [ 11 ] , CipherFormer [ 70 ] , East [ 71 ] , Iron [ 10 ] , LLMEncryptedPrompt [ 72 ] , Nimbus [ 14 ] , Primer [ 73 ] , BLB [ 74 ] [MM], BumbleBee [ 12 ] [MM], CipherPrune [ 75 ] [MM], EncFormer [ 40 ] [MM], SMASH [ 76 ] [MM], SecMoE [ 38 ] [MM] | AERO [ 77 ] , CipherGPT [ 78 ] , PermLLM [ 79 ] , BLB [ 74 ] [MM], BumbleBee [ 12 ] [MM], CipherPrune [ 75 ] [MM], CryptoGen [ 37 ] , CryptoMoE [ 80 ] , EncFormer [ 40 ] [MM], SMASH [ 76 ] [MM], SecMoE [ 38 ] [MM] |
| Other / adjacent | STIP [ 81 ] [MM], CASCADE [ 82 ] [MM] | STIP [ 81 ] [MM], CASCADE [ 82 ] [MM] |
TABLE II: Composition of the audited corpus by cryptographic backbone and evaluated inference regime. Systems marked [MM] evaluate multiple regimes and appear in both columns. We also audited three encoder-decoder systems: PPTIF [ 46 ] , Privformer [ 28 ] , MERGE [ 27 ] . Rhombus [ 47 ] is audited at the protocol level: it contributes a matrix-vector multiplication kernel rather than an evaluation in a specific inference regime, so it is not placed in either column.
Fig. 2: An overview of the graph-based framework. (a) Every paper in the corpus enters the audit as a Private Inference Contract (PIC) card. (b) Each pair of cards is evaluated through the comparability gates – , which aggregate into a single edge type in the comparability graph (RQ1); the same gates, read against a paper’s own baseline, identify its source of progress (RQ2). (c) Reading the per-year graphs together traces the private scaling frontier against the public one (RQ3).
| Contract category | Fields audited |
|---|---|
| Scale and workload | Largest evaluated model, parameter count, inference mode (encoder / decoder / encoder–decoder) |
| Privacy and threat model | Protected asset, leakage scope, adversary model, party model |
| Mechanism and fidelity | Mechanism (MPC / HE / hybrid), fidelity class (exact, approximate, architecture-modified, task-limited) |
| Cost and deployment | Latency, communication, rounds, cost scope (online / offline / end-to-end), amortization, network regime |
| Baseline and reporting | Named baseline, in-corpus or external baseline, missing or limiting fields, sufficiency for cross-paper comparison |
TABLE III: Evaluation contract card used to interpret PTI progress claims. A headline number is directly comparable only when the relevant contract fields are aligned and sufficiently reported.
Fig. 3: The PIC card instantiated for BumbleBee [ 12 ] , filled from the source paper through the card in Table III . A check marks a fully reported field and a half-circle a partially reported one (here, rounds only at the component level).
| Gate | PIC category | Evaluation predicate |
|---|---|---|
| G1: Privacy | Privacy and threat model | Protected assets and leakage scope match |
| G2: Threat | Privacy and threat model | Adversary model and party topology match |
| G3: Workload | Scale and workload | Evaluated model architecture and workload match (e.g., encoder vs decoders) |
| G4: Fidelity | Mechanism and fidelity | Model preservation or approximation degree matches |
| G5: Cost | Cost and deployment | Cost accounting boundaries and execution phases match (e.g., end-to-end vs online-only) |
| G6: Amortization | Cost and deployment | Batch and sequence amortization assumptions match |
TABLE IV: Refinement of the five PIC card categories into eight comparability gate types.
Fig. 4: Per-gate outcome across all system pairs. The threat and party model (G2) and model fidelity (G4) drive most of the hard incompatibility, while amortization (G6) and reporting sufficiency (G8) are dominated by unreported fields.
Fig. 5: Pairwise comparability matrix across the 62 audited papers. Cell color indicates the number of hard-conflicting gates (green: no conflict; darker red: higher conflict). Rows and columns are ordered via agglomerative clustering, with boxed blocks marking the two comparable cohorts and the top bar tracking each paper’s count of conflict-free partners. See Figure 10 in Appendix A-B for the fully labeled matrix.
Fig. 6: The comparability collapse under cumulative gates (RQ1). Competitive pairs fall from 395 under a compatible workload alone to a single pair under the full contract, with most of the drop at the early substantive gates rather than at reporting; invalid pairs rise to match, and descriptive pairs are the residual.
Fig. 7: Primary progress-attribution labels over the 62-paper corpus. These labels attribute the main source of each paper’s reported progress claim; a primary protocol label marks the dominant driver, and every such paper still carries at least one secondary non-protocol caveat (Section VI-C ). Each bar is split by our confidence in the assigned label: high when the paper names its baseline and states the changed component explicitly, medium when the label is clear but carries a stated secondary caveat, and low when the label requires inference from indirect evidence or sits near a category boundary. Dropping the low-confidence labels leaves the same findings (Appendix A-E ).
Fig. 8: Public LLM inference scale versus the audited conventional cryptographic PTI frontier, on a logarithmic parameter axis. The dense frontier enters at 0.34B (Iron, 2022), reaches 13B (SIGMA, 2024), and does not move through 2026, while public models pass a trillion parameters, a gap of nearly two orders of magnitude. CryptoMoE’s 16.4B mixture-of-experts (about 2.8B active per token) sits off the dense line, and FLUTE’s 70B partial-encryption result is kept off the frontier as a weaker contract. Each point carries its own contract (Table X ), so the frontier is a scale-context upper bound, not a like-for-like capability comparison.
| Family (env.) | Contract caveat | Metric | 7–13B (published) | 70B (proj.) | 405B (proj.) |
| FSS, 2PC+dealer (SIGMA) | online-only, GPU | time/pass | 23–38 s | 115 s | 384 s |
| comm/pass | 12–19 GB | 58 GB | 188 GB | ||
| FSS key/query | 255–419 GB | 1.4 TB | 4.9 TB | ||
| 3PC secret sharing (PUMA) | end-to-end, 20 Gbps LAN | time/token | 164 s | 27 min | 2.6 h |
| comm/token | 2.1 GB | 21 GB | 119 GB | ||
| Hybrid HE+MPC, 2PC (BumbleBee) | end-to-end, CPU | time/token | 832 s | 2.3 h | 13 h |
TABLE V: What one private forward pass (one generated token) would cost at deployment scale, extrapolated per family from published evidence under that family’s own contract. The 3PC per-token anchors are the marginal second token from the source’s own table, so prefill is not considered.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Status | Papers | Share |
|---|---|---|
| Sufficient | 1/62 | 1.6% |
| Partial | 43/62 | 69.4% |
| Insufficient | 18/62 | 29.0% |
TABLE VI: Reporting sufficiency for clean cross-paper latency comparison. The rule requires a headline claim, baseline, latency, communication, end-to-end rounds, network context, cost scope, and amortization context. This table measures sufficiency for comparison, not paper quality.
| Scenario | Competitive relations | Baseline-compatible claims | Protocol/mixed | Non-protocol primary |
|---|---|---|---|---|
| Locked labels | 1/1891 | 0/62 | 52/62 | 40/62 |
| Optimistic progress attribution | 1/1891 | 0/62 | 52/62 | 40/62 |
| Strict progress attribution | 1/1891 | 0/62 | 52/62 | 40/62 |
| Optimistic comparability | 93/1891 | 2/62 | 52/62 | 40/62 |
| Strict comparability | 1/1891 | 0/62 | 52/62 | 40/62 |
| Drop moderate/boundary confidence | 1/1891 | 0/62 | 45/49 | 31/49 |
TABLE VII: Sensitivity analysis for the two central conclusions: direct global leaderboard-style comparison is unsupported under published evidence, and progress claims are multi-causal. Protocol/mixed counts papers whose attributions include protocol progress (as primary or secondary factor) or whose primary label is mixed; non-protocol primary counts papers whose primary label is any non-protocol category. The reviewer-favorable row is an optimistic upper bound, not the locked result.
Fig. 9: Distribution of the 62 audited papers over the categorical PIC-card fields. The corpus is near-constant on the threat axes that rarely block comparison (55/62 semi-honest; 49/62 protect both input and model) but spreads across mechanism, model fidelity, party topology, and workload regime, the fields that carry the comparability gates. Long tails are consolidated: “other” under party topology aggregates five settings with three or more parties.
Fig. 10: Full comparability graph with all 62 papers labeled. As in Figure 5 , each cell shades a pair by the number of comparability gates that hard-conflict (green marks no hard conflict), with rows and columns ordered by agglomerative clustering and the top bar counting each paper’s hard-conflict-free partners (hub systems in blue).
| Contract field | Agree | Disagree | V. silent | Both silent | H. silent | Rate |
|---|---|---|---|---|---|---|
| Cryptographic backbone | 61 | 1 | 0 | 0 | 0 | 98.4% |
| Inference regime | 37 | 1 | 23 | 1 | 0 | 60.7% |
| Privacy scope | 58 | 4 | 0 | 0 | 0 | 93.5% |
| Adversary model | 54 | 2 | 6 | 0 | 0 | 87.1% |
| Party model | 39 | 19 | 4 | 0 | 0 | 62.9% |
| Model fidelity | 46 | 16 | 0 | 0 | 0 | 74.2% |
TABLE VIII: Per-field agreement of the independent blind re-extraction with the recorded human values, over the 62-paper corpus. Agree / Disagree are value-vs-value; V. silent is where the blind reader declined to extract a value the human took from context (e.g., a parameter count implied by a model name); Both silent is concordant agreement that the paper states nothing; H. silent is where no human value was recorded (both excluded from the rate). The rate is . The fields the comparability gates lean on most (cryptographic backbone, privacy scope, adversary model, end-to-end status) agree in 87–98% of papers, with 70.0% raw agreement pooled across all factual fields; lower rates are dominated by values the blind reader declines to infer and by concordant silence. Every flagged cell was adjudicated against the source papers; across the two adjudication passes, 40 of 992 fields were corrected corpus-wide. The final row is the judgment-label inter-rater agreement, not a factual field.
| Dominant deployment constraint | Tends to favor | Why |
|---|---|---|
| Interactive, WAN / high round-trip time | Low-round HE or few-round hybrid | Each interaction round pays the round-trip, so round-heavy MPC inflates wall-clock latency even when its LAN numbers look strong. |
| Batch / high throughput, amortizable | HE with SIMD packing; amortized-offline hybrid | Ciphertext batching and reusable preprocessing amortize across many queries, where per-query latency is secondary. |
| LAN / high bandwidth, low round-trip time | MPC or hybrid | Cheap rounds make interactive protocols competitive or fastest; HE ciphertext and compute overhead rarely pays off here. |
| Bandwidth-constrained link | HE or low-communication designs | MPC communication volume can dominate cost; HE trades bytes for local computation. |
| Model weights must stay private | Input-and-model 2PC (HE or hybrid); avoid weight-revealing split or statistical schemes | Protecting weights is a hard constraint largely orthogonal to speed; some fast schemes assume public weights. |
| A non-colluding helper or third party is acceptable | 3PC or helper-assisted MPC | A helper trusted for non-collusion unlocks cheaper nonlinear and truncation protocols. |
TABLE IX: Prescriptive backbone guidance under the contract view (Section VIII-E ). These are cost-structure tendencies conditioned on the deployment contract, not a universal ranking: the preferable backbone is the one whose cost structure matches the dominant constraint, which is also why a single speed leaderboard misleads.
| Year | System | Params | Mechanism | Party model | Model fidelity | Cost scope |
|---|---|---|---|---|---|---|
| 2022 | Iron | 0.34B | Hybrid MPC–HE | 2PC | exact | end-to-end |
| 2023 | PUMA | 7B | MPC | 3PC | approximate | end-to-end |
| 2024 | SIGMA | 13B | MPC | helper-assisted | approximate | online-only |
TABLE X: The conventional cryptographic PTI frontier of Figure 8 , annotated with each frontier point’s evaluation contract. The points span different party models (2PC, 3PC, helper-assisted) and cost scopes (end-to-end, online-only), so the cumulative envelope is a generous upper bound over heterogeneous contracts, not a single comparable trajectory. Tightening to one fixed contract could only lower the private-inference frontier.
| Pattern | Why it looks comparable | Contract issue | Gate | Sound reading |
|---|---|---|---|---|
| Online latency vs. end-to-end cost | Both report private-inference latency. | One number is online-only or amortized; the other counts more of the end-to-end cost or assumes different reuse. | G5, G6 | Not directly comparable unless cost scope and reuse assumptions are aligned. |
| LAN result vs. WAN claim | Both report latency for the same broad model family. | The measurement is LAN, simulated, or unspecified while the motivating deployment is WAN-like; rounds convert round-trip time into wall-clock cost. | G7, G8 | Supports a LAN or local-deployment claim, not an unrestricted WAN claim. |
| Encoder classification vs. decoder generation | Both are Transformer systems built on secure-computation primitives. | The workloads differ in regime and in sequence/generation semantics (per-token vs. per-sequence vs. per-batch). | G3 (often G4) | Different comparison neighborhoods unless workload and fidelity are normalized. |
| Model-preserving vs. model-modifying | Both target the same model family or benchmark. | The faster result depends on approximation, retraining, pruning, distillation, quantization, or architecture change. | G4 | Possibly workload/fidelity-shift progress; not pure protocol progress. |
TABLE XI: Four recurring comparison patterns and their contract reading. Each looks like a direct comparison at the headline level; the gates identify the mismatched or missing field that makes it descriptive or invalid.
More in this topic
Transformer Inference
Transformer inference studies how trained Transformer models produce outputs efficiently, accurately, and securely across hardware and context lengths.
See all 57 papers