cs.LGSep 27, 2026
SaveTraining Witnesses: Trusting the Training without Trusting the Trainer
Organizations: Microsoft Research
Abstract
Progress in machine learning cannot outpace our ability to verify it. With an explosion in papers today, every scientific claim rests initially on trust in the trainer, leading to uneven evaluation, baselines, and forestalling of reliable progress. Traditionally, the burden of verification falls on the reader, who must reproduce expensive training runs. This strategy is impractical due to an explosion in slop contributions, diversity of methods, and the sheer compute required. We put the burden of proof where it belongs, on the trainer, and in the process also cut the overall cost of verification significantly. We introduce Witnesses, a method for certifying training, data usage and evaluation in a neural network training run. Our key insight is that fast behavioral fingerprints with occasional replay challenges are sufficient for auditing neural network training. Our method is applicable at scale with minimal overhead to the trainer, is cheap for the verifier, rejects bad training runs with amplifiable probability, and allows for exact queries of both data inclusion and exclusion. We test our method on language model training runs from 100M to 2B scales, across DDP and FSDP, and demonstrate this minimal overhead. We also introduce a self-regulating leaderboard of "auto-certified" training runs that enables shared baselines and progress. We invite the community to participate in the leaderboard to improve reproducibility in machine learning.
Figures & tables
Figure 1: Training Witnesses. We present a system that creates privacy-preserving witnesses that allows a public verifier to cheaply check and trust what information is included in the training of a model without disclosing model training details, enabling trust in the artifacts without trusting the trainer.
Figure 2: Disclosure settings and what is certified. Our method can protect against dataset inclusion and exclusion attacks in the fully open setting, and data inclusion attacks in the open-weight setting.
Figure 3: (Top) Privacy and trust boundaries between our three interacting parties. The prover obtains a black-box functional training procedure, and the verifier obtains only a proof of training data with minimal privacy leakage. (Bottom) Interaction protocol between the parties. The Witnesses system uses a communicating algorithm between a potentially-malicious trainer and prover to generate a privacy-preserving public training witness which any verifier can cheaply check the validity of.
Figure 4: Our protocol’s key security insight . By using a geometric sketch, we can achieve fast binds of our model parameters to a certificate tape while ensuring that cheating which breaks the continuity of the tape cannot be done sparsely. Our protocol is cheap, fast and has a low memory footprint.
Table 1: Execution timelines of our system; colors represent single logical actions. Python driver (D) coordinates the training process, Rust-side worker (W) dispatches and coordinates training, Rust broker (B) coordinates the transitions and records the output certificate, and Accelerator (A) chip.
Figure 5: Our model achieves near-certain detection rates of data inclusion, exclusion, and substitution attacks at any amounts that would constitute meaningful poisoning . End-to-end data uinsertion attacks, sweeping across number of tokens inserted. Empirical checks performed during of updates, in line with all performance evaluations. (Left) : attack by substituting declared and undeclared sequences. (Middle) : attack by adding sequences. (Right) : attack by subtracting sequences. Attacks distributed uniformly. Shaded region represents literature-established rates of effective poisoning attacks; error bars are 10-90 Beta posterior credible intervals.
Figure 8: Even active optimization against our sketch still cannot evade it without meaningfully shifting the model towards training distribution collapse. We sweep across a Pareto tradeoff parameter to minimize eq. 4 . Left : directly minimizing eq. 4 with sketch-nullspace projection. Right : attempting to modify output on malicious KL without sketch error minimization or null space projection. All plotted points detected by sketch at floating point precision , .
Figure 9: Our method achieves remarkably low overhead at sharded training settings, where it is communication-bound . Communication-bound overheads. Left : amortized per-step training time; Right : mean HBM resident size (bytes in use) per GPU. Error bars from Student’s t-test confidence intervals over 100M tokens of training at check probability. DDP and FSDP sharding conducted over 2 NVIDIA H100 NVL72.
Figure 10: Our verification system maintains constant memory (RSS) and linear tape size to model size, meaning it can be effectively streamed . Left : Certificate length and verification time per token across a variety of scales. Right : verification procedure Resident Set Size (RSS) as a function of cert. length. Verification conducted using wait4 syscall under a single pinned physical core of an AMD EPYC 9554 processor. Error bars from Student’s t-test confidence intervals.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 11: Verification transcript of a training run showing the metadata available to the users.
Figure 12: Detail screen of a run with training data attested and revealed; unverified runs can also be listed on our leaderboard relying on the traditional method of trusting the trainer.
Figure 14: Cheating only in limited subspaces is also reliably detected by our sketch . Structured attacks on a 100M parameter model during a single transition, sweeping across different relative change RMS magnitude. Structured perturbations are applied across different components of the model, and we measure the next-token KL divergence induced on 6 orthonormal input embeddings. “Detected” means that the sketch outputs differ by the floating point difference threshold. Note that this same sketch would secure provenance and transition claims, and undetected changes result in no top-k changes and vanishingly small logit changes.
Figure 15: In compute-bound settings, our system can still maintain high throughput without a drastic cost to MFU . Saturated single-GPU overheads for our method with AdamW. 1 : token throughput overhead; 2 : amortized per-step training time; 3 : mean HBM resident size (bytes in use); 4 : MFU impact as measured by published theoretical BF16 Tensor Core GEMMs. 1.03B dense GPT-style model on single-GPU with sizes scaled for compute (memory) roofline saturation; profiling training steady-state at check probability. Measurement conducted on a single NVIDIA H100 NVL72.
Figure 16: Our performance cost scales log-log-linearly with model size, as is similar for model training itself . DDP time and memory overheads for applying our method with SGD across a variety of scales. Left : amortized per-step training time; Right : mean HBM resident size (bytes in use) per GPU. Single seed, conducted over 2 NVIDIA H100 NVL72. Axes are log-log scale.
Figure 17: Left : Our system does not change the actual training . Toy example of the very beginning of training of a 100M model. Our system does not change the training procedure and indeed the training is bitwise reproducible on the same hardware with and without our system. Right : Our overheads are caused primarily by tape write backpressure and state offloading costs . Breakdown of the structure of one update at 100M. Note that the update overhead comprises state offloading (amortized) time and certificate worker write overhead (instantiated as the update being held by flow control).