cs.CRSep 3, 2026
SavePrivacy Failure in Split-LLM Training, The Returned Gradient Nullifies the Decoys
Abstract
We present a systems-security case study of a two-node split-LLM training system whose privacy evaluation passed while leaving an observable channel untested. The Trusted Local Node (TLN) sends protected activations to the Untrusted Cloud Node (UCN), the UCN returns its output, and TLN, holding the private loss, returns the output gradient. The frame the UCN receives mixes real rows with decoys, and the loss ignores the decoys. Their gradients are exactly zero, so the pattern of zeros reveals which rows were real. We measure it with a protocol fixed in advance: a leak injected at known strength to prove the instrument can see one, a shuffled-label control to prove it does not report absent leaks, and a threshold set before the runs. Across nine seeds, the zeros identified the real rows on every frame, 4,096 of 4,096 per run. An attack on the frame contents recovered about one extra token per hundred over a constant-guess baseline (+0.65 to +1.50 percentage points); the shuffled controls recovered nothing. A second set of runs repeated this on a configuration that keeps model quality within budget, so the finding is not confined to a setting nobody would deploy. On both datasets, every such run passed the forward-channel privacy check and the quality check, yet failed that same check once the returned gradient was included. Clipping and noising each row of the gradient closed the leak for about 0.01 nats of held-out cross-entropy. The system is not thereby safe: five classes of attack, including those accumulating observations across training steps, were never measured.
Figures & tables
| field | evaluated setup |
|---|---|
| model and runtime | Qwen3-0.6B; Transformers 5.13.0 runtime; exact weights SHA-256 f47f71177f32bcd101b7573ec9171e6a57f4f4d31148d38e382306f42996874b |
| topology | Main configuration and replication runs: split after layer 14, resume at 26 (11 delegated layers). Mitigation runs: split 21/26 (4 delegated layers). |
| corpus and implemented split | WikiText-2; corpus SHA-256 78b6bfb90cfd718f0c27d42b1fd2231b139d1dda75d7d796e6a603b2e5cd7efe . The runner tokenizes the flat corpus into disjoint, sequential fixed-width blocks, then slices train followed by evaluation; this is not a document-level split. |
| frame and latent | 32 real 48 decoy 80 rows per frame; . |
| boundary protection | Forward and returned activations: per-row and Gaussian . Main configuration and replication runs, outbound gradient: open. Mitigation outbound gradient: and Gaussian . |
| known-plaintext latent probe | coordinate_plus_invariants , invariant_only , and invariant_graph ; 20 epochs; batch 512; 3 restarts. |
Table 1: Experimental setup for the main configuration and for the replication and mitigation runs. Full fingerprints identify the exact model weights and corpus used by the evaluated runner.
Figure 1: The three hops of one training step. Only tln can compute the gradient, because only tln holds the loss, so hop 3 travels left to right just as hop 1 does: “backward” names the pass the tensor belongs to, not the direction it moves. Hops 1 and 2 are clipped and noised, and the original evaluation attacked hop 1 and passed its gate. Hop 3 crossed raw and carried the leak. The zero-support signal is an implementation and system-design defect; the false pass is an uninstrumented-channel evaluation failure.
Figure 2: Anatomy of one training cycle, per frame. SEALED never crosses the boundary; OBFUSCATED/PROTECTED are the defence’s active layers; LEAK marks the unprotected backward wire and the partition mechanism it discloses. The forward probe passes at step 6 while steps 10 and 11 leak: the paper’s finding in one diagram.
| tln (trusted) keeps | ucn (compromised) sees/holds |
|---|---|
| Private corpus; token IDs; plaintext I/O | Gauged frame rows only (80 rows/frame: 32 real 48 decoy) |
| Frozen LM prefix, embedding, LM tail | A 100-parameter gauge-equivariant surrogate it trains itself |
| Private encoder and decoder weights | Unclipped, unnoised -width output gradients (the omitted channel) |
| Per-request gauge masters (rotation, permutation, scale; SHA-256 counter-mode, 128-bit) | Protocol metadata: frame sizes, timing, session shape digests |
| Honest evaluation labels; session keys; optimiser state for trusted parts | TLS 1.3 endpoints and ciphertext (pinned CA) |
| Never crosses: -width activations, canonical coordinates, token order, token scale, plaintext tokens. | |
Table 2: Trust split at the evaluated operating point. The output gradient crosses the backward wire unclipped and unnoised.
| family | applicable metrics | positive control | status |
|---|---|---|---|
| forward_only | token_top1 , rare_token_top1 , token_cross_entropy | injected leak on the forward wire; codeword injection | measured |
| gradient_only | token_top1 , rare_token_top1 , token_cross_entropy | injected_leak on the backward wire | measured |
| joint_forward_gradient | token_top1 | scaled joint concatenation (the exploratory frequent-token gradient result) | measured |
| accumulated_history | token_top1 | none | unmeasured (no positive control) |
| stateful_remote_state | token_top1 | none | unmeasured (no positive control) |
| timing_metadata | — | none | unmeasured (no applicable metric) |
Table 3: The declared channel families and their status. “Unmeasured” means the family has no applicable metric or no positive control, so it cannot support a primary claim.
Figure 3: Dose–response per metric on the coordinate-mode sweep (amplitude 1.0). The four emitting metrics disagree: rare_token_top1 is most sensitive; token_cross_entropy barely moves until the leak dominates; membership_auc does not detect the low-dose token signal because the injection is not a membership signal. A single calibration run across 19 doses.
| metric | cov 0.02 | 0.04 | 0.06 | 0.10 | 0.30 | 1.00 |
|---|---|---|---|---|---|---|
| token_top1 (pp) | 0.35 | 0.35 | 0.98 | 4.19 | 22.51 | 94.88 |
| rare_token_top1 (pp) | 0.08 | 0.32 | 0.67 | 1.97 | 14.18 | 100.0 |
| token_cross_entropy | 1.114 | 1.114 | 1.114 | 1.115 | 1.130 | 7.025 |
| membership_auc (AUC ) | 0.093 | 0.093 | 0.095 | 0.095 | 0.096 | — |
Table 4: Coordinate-mode coverage sweep at amplitude 1.0. The calibration reference is the zero-dose source cell (coverage 0, same bundle family) and the constant baseline (the most common evaluation token); the calibration sweep contains no shuffled-label arm, because those negative controls are introduced only at scoring time. The top-1 rows show point effects, while the gate statistic uses their Bonferroni–Wilson upper-95 value; thus the gate first breaks at coverage 0.06 although the displayed point effect is pp. Cross-entropy is a proper scoring rule; the values shown are the effect over the constant baseline, and at scoring time CE is read arm minus its shuffled-label negative control, never raw (Table 5 ). membership_auc values are flat. “—” marks the amplitude-axis section, not part of this row.
| metric | threshold set in advance | basis |
|---|---|---|
| token_top1 | gate: Bonferroni upper-95 excess pp | first displayed gate break at cov. 0.06 |
| rare_token_top1 | paired effect pp; report raw recovery | earliest subthreshold movement at cov. 0.04 |
| token_cross_entropy | none set; read arm minus shuffled-label negative control (never raw) | a confidently-wrong negative control inverts raw CE; no numerical gate |
| membership_auc | none (diagnostic only) | falsified as a channel; constant baseline vacuous |
Table 5: Per-metric thresholds, set in advance. A metric without a measured detection curve may not appear in a primary claim.
| boundary | Wilson upper-95 acc. | constant baseline | excess | note |
|---|---|---|---|---|
| naked ( , no defence) | 28.54% | 4.35% | pp | breaks |
| defended ( , noise, decoys) | 5.56% | 5.27% | pp | at floor |
| defended ( , dimension-matched) | 5.56% | 5.29% | pp | at floor |
Table 6: One probe architecture, one protocol, three boundaries. The probe that inverts the naked boundary at over six times its baseline reads nothing through the defence, at the operating width and at matched width, so the defended floor readings are not attacker blindness. Best accuracy is selected on the evaluation set over 50 epochs (a selection reading); the accuracy column reports its Wilson upper-95 bound, so each row’s excess is the displayed accuracy minus the baseline. The defended probes’ accuracy declines after its early epochs, consistent with overfitting noise rather than extracted signal. The committed three-restart rerun of the defended probe reads at the same floor ( pp); the other three committed deep-probe runs are the rows above.
| result category | result | interpretation |
|---|---|---|
| forward wire | gate passed | the original instrument’s only channel |
| backward wire | leaks structurally | 4,096/4,096 frames; row agreement 1.000, on each of nine seeds |
| frequent-token content effect | pooled over the nine verified seeds: pp, 95% interval | negative control at floor on every seed; per-seed values in Table 8 |
| nine audit cells | effect detected at 12 and 11 delegated layers, at floor at 8 and 6; detected at and , at floor at ; 2.5 more exposure does not amplify | one run each (seed 42); partition exact at every depth |
| dose–response calibration | token_top1 onset 0.04–0.06; rare_token_top1 at 0.04; CE flat until dominant; membership_auc flat and dose-insensitive | per-metric thresholds set in advance |
| internal scorer reimplementation | all nine cells pp | internal check (Appendix A ) |
Table 7: Result categories and where each is established. The final row refers to the four-layer mitigation topology; every row above it refers to the primary configuration.
| seed | gradient arm (pp) | forward arm (pp) | negative control (pp) | frames exact | verdict |
| 42 | 4,096/4,096 | detected | |||
| 43 | 4,096/4,096 | detected | |||
| 44 | 4,096/4,096 | detected | |||
| 45 | 4,096/4,096 | detected | |||
| 46 | 4,096/4,096 | detected | |||
| 47 | 4,096/4,096 | detected |
Table 8: Exploratory seeds 42–47 and the replication runs (seeds 48–50). Statistic: the paired effect of each arm over its constant baseline (the most frequent evaluation token), clustered by frame. The negative-control column reports the label-shuffled twin of the gradient arm, scored through the same statistic. Inferential unit: one training seed; each row one independent run, its interval a cluster bootstrap over that run’s frames. All nine committed seeds are shown. The forward arm is built on the oracle real/decoy split and therefore does not by itself describe an achievable attack. It is reported here because in this system the oracle is redundant: the gradient reproduces the partition exactly (Section 5.1 ), so the forward arm is what an attacker reads after the backward wire has supplied the partition. No label-shuffled control was run for the forward arm, so that column is uncontrolled; the negative-control column bounds the gradient arm only. Seed 42 was served by a cloud container built before the code was packaged for release; seeds 43–47 ran the released package on both nodes, and seeds 48–50 are captures made under the fixed protocol.
| cell (seed) | utility | forward gate | joint arm (gate) | joint arm (paired effect) | support leak |
|---|---|---|---|---|---|
| defended, gradient open (s51) | 1,024/1,024 | ||||
| defended, gradient open (s52) | 1,024/1,024 | ||||
| defended, gradient open (s53) | 1,024/1,024 | ||||
| defended, gradient clipped and noised (s51) | 0/1,024 | ||||
| defended, gradient clipped and noised (s52) | 0/1,024 | ||||
| defended, gradient clipped and noised (s53) | 0/1,024 |
Table 9: The mitigation runs. The gate columns report the gate statistic: Bonferroni-upper-95 excess over the constant baseline. The paired column is the cluster-bootstrapped paired effect of the best arm against its constant baseline. “Joint arm” is the forward frame and the output gradient of the same training step, concatenated: the view the compromised node actually holds, and the one the original gate never scored. This set comprises fourteen runs; the twelve scored cells are shown. The two omitted are a configuration smoke test that produced no scored arms and the packaged seed-42 rerun of the main configuration, reported separately in this section. † The naked control on seed 53 fails the utility gate ( loss against the bound); it is retained because it is a deliberately undefended sensitivity control, not a candidate configuration. In every one of these runs the per-row scale-and-sign gauge was disabled , as it is for the rotation-invariant cloud architectures (Appendix B ); the joint-view results below are conditional on that setting.
Figure 4: Attack-specific shape across audit cells. (a) depth ladder: the implemented emitter detected the effect at 12–11 delegated layers and not at 8–6. (b) the width sweep: the effect is detected at and , and not at . (c) The implemented probe’s paired effect did not increase between the two tested exposure budgets. Paired effect over the constant baseline with 95% CI on the gradient arm; one run per cell (seed 42).
| axis | steps | gradient arm (pp) | negative control (pp) | verdict | frames |
| depth (12 del.) | 2k | detected | 512/512 | ||
| depth (11 del., ) | 40k | detected | 4,096/4,096 | ||
| budget | 100k | detected | 4,096/4,096 | ||
| width ( ) | 10k | detected | 4,096/4,096 | ||
| depth (8 del.) | 2k | at floor | 512/512 | ||
| depth (6 del.) | 2k | at floor | 512/512 |
Table 10: The nine-cell audit. Paired effect of the gradient arm over its constant baseline, with each cell’s shuffled-label negative control beside it; frame-exact counts from the zero-support partition check. All nine committed cells, one run each (seed 42). The partition mechanism (zero-support match) is exact on every cell, whether or not the token effect is detected.
| attack | step (Fig. 2 ) | outcome | does it work? |
|---|---|---|---|
| pre-set 9-arm battery on forward frames | 6 | defended floor pp; naked pp | no, against the defence |
| deep-inversion transformer probe | 6 | naked split-14 pp; defended floor at and | no, against the defence |
| zero-support read (no training; just zeros) | 10 | partition exact on every frame, every seed | yes |
| gradient arm alone | 10 | floor on the private corpus ( ); breaks on 2 of 3 public-corpus seeds ( ) | only on one dataset |
| joint arm (frame gradient) | 6 10 | breaks the pp gate set in advance on 6 of 6 defended cells ( ) | yes |
| cross-Gram invariant | 6 10 | floor on every capture (cosine and raw forms) | no; the feature carries nothing |
Table 11: The attacker ledger: every attack family tried against the system, where it applies in the cycle of Figure 2 , and its outcome. Every committed attack family is listed; the supporting artefacts are indexed in Appendix A . The two leak rows are the finding; the closed row is the fix.
| system | role | injected calibration | shuffled-label negative control | predeclared gate |
|---|---|---|---|---|
| BiSR [ 6 ] | attack | none | none | none |
| DualGuard [ 18 ] | defence | none | none | none |
| Prompts to Responses [ 12 ] | attack + defence | none | no (unmatched baseline) | none |
Table 12: Selected audit verdicts. The three works report attack or defence performance against reconstruction baselines. “Prompts to Responses” includes an unmatched random-token baseline in its appendix.
| System | Threat model | Covers training? | Privacy evidence | Integrity | Cost |
|---|---|---|---|---|---|
| This work | compromised cloud; 9 declared channels: 3 measured, 1 constructible, 5 declared unmeasured | yes: 2k-step cells, 6 fresh seeds, 2 datasets; 40k leak cells | forward probe at floor on defended cells (conditional: gate fires 6% coverage); joint view breaks the gate on 6/6 cells ( pp) until clipping and noise close it (3/3 at floor) | median 100%/0% (one perturbation class) | WAN |
| AloePri [ 17 ] | honest-but-curious | no (inference) | point TTRSRs; VMA 13.5–25% on two models | none | 0% online; 482 min offline at 671B |
| GELO [ 2 ] | VRAM-read TEE island | no | p95 cosine/Gram error, no control | scoped out | 20–30% microbenchmark |
| PermLLM [ 34 ] | permutation obfuscation | no | broken by the matching family ( 99% reconstruction) | none | 3 s/token (6B) |
| ObfNet [ 32 ] | backend holds obfuscators | no | perceptual (10 volunteers) | none | 0.22–11 ms/sample edge |
| TEE [ 35 ] | trust silicon vendor | partial | hardware-rooted attestation | hardware-rooted | 7–20% |
Table 13: Positioning against the closest systems. “Empirical” = attacker-measured privacy; our excess is over a matched no-attack control with a confidence bound and a pre-declared gate, and the thresholds set in advance are calibrated by injected leaks (Section 4 ). Literature rows report those systems’ own published numbers; their evidence columns use their own authors’ metrics, the uncalibrated, no-matched-null pattern of Section 7 , so read the comparison as evaluation method , not as relative leak size.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| result category | committed input | committed code | re-derivable |
|---|---|---|---|
| exploratory gradient result | per-seed paired JSONs and negative controls; bundles are manifests to cluster-side prediction tensors | packaged scorer | displayed values from committed summaries |
| nine audit cells | paper-data/collected/diagnostic/gradaudit/ cell JSONs and summary | papers/paper-1/figs/build_figures.py | yes |
| dose–response calibration | paper-data/collected/diagnostic/w24_metric_sweep/ JSONs | papers/paper-1/figs/build_figures.py | yes |
| mitigation runs | phasec_2026-08-27/ manifests and paired summaries | packaged driver and scorer | displayed values from committed summaries |
| internal scorer check | internal-check report | uncommitted independent scorer | recorded verification only |
| complete transcript | hashes and manifest | verifier | no; host-only by policy |
Table 14: Per-result availability. The audit-cell and calibration rows regenerate from committed summaries; the exploratory and mitigation rows are redisplayed from committed derived artefacts whose raw prediction tensors remain on the cluster. Transcript hashes and the manifest are committed, but the payload remains on its host.
| Purpose | Technique | Detail | Status |
|---|---|---|---|
| Compression | adversarially trained MLP encoder/decoder (information bottleneck) | MLP with GELU, trained by minimax against embedded token/property/reconstruction probes plus a MINE MI penalty (the MI reading is not a certificate) | core |
| Noise | per-token-row L2 clip to additive Gaussian (0.40 on hybrid MoE), both directions | Gaussian mechanism with zCDP accounting (the is quoted as vacuous) | core |
| Key derivation | SHA-256 counter-mode KDF | 128-bit CSPRNG master per draw, domain-separated sha256(DOM S t) , exact uniforms, float64 Box-Muller | core |
| Rotation | Haar-random orthogonal | Gaussian matrix QR with positive-diagonal sign fix, at width; fresh per request | core |
| Permutation | Fisher–Yates over the KDF stream | without replacement; fresh per request | core |
| Scale gauge | per-row , , clamped | independent of the other gauges (separate KDF epoch); OFF for radial/invariant-MLP clouds | core |
Table 15: Techniques at a glance: what each mechanism is , concretely, where it lives, and its status (core / available / optional / withdrawn / invalidated) in the current evidence.
| Arm (fraction grid) | 0.6B trend | 35B-A3B trend | Reading |
|---|---|---|---|
| Wire capture 10/25/50/100% | flat, to | flat, to | capturing the whole wire does not help |
| Decoy identities revealed 0/25/50/90% | to | to | knowing which rows are decoys makes the probe worse |
| Gauge secrets leaked, 10–20% | ungauged / gauged | / | rows seen before gauging still read at guessing level |
| Known plaintext 1/5/20% | flat, to | flat, | knowing some tokens does not reveal the others |
Table 16: Compromise-fraction battery on the defended cell. Each row gives the attacker a growing share of some secret and asks whether the probe improves. Values are the probe’s accuracy in percentage points above or below its constant-guess baseline, so a negative number means the probe did worse than guessing. None of these arms approaches the pp gate, and none improves as the attacker is given more.
| Model | Gate statistic (pp) | Vector-matching top-1, against baseline | Nearest-signature top-1, against baseline |
|---|---|---|---|
| 0.6B (seed 42) | vs | vs | |
| 35B-A3B (seed 42) | vs | vs |
Table 17: External matching-attack families vs. the defended cell (regenerated bundles). Two adapted attack families on these captures. The positive-control sensitivity they presuppose is Table 6 .
| Cell | Verified frames | Flagged | Max rel. dev. | Loss | Ratio |
|---|---|---|---|---|---|
| 3 honest CPU nodes | 2,256 | 0 (0% FP) | |||
| 2 honest 1 malicious (lying) | 2,256 | 2,256 (100%) |
Table 18: Byzantine median verification. One large perturbation class on identical CPU replicas; no general active-security claim.
More in this topic
LLM Security
Large language model (LLM) security studies attacks, privacy risks, and defenses across model training, inference, and deployment.
See all 324 papers