cs.SENov 2, 2026

Testing Processor Contracts in Speculative Decoding Engines

Authors: Tianyu Ding, Yaocheng Tong, Muzhi Yang, Lei Yang, Fei Jiang

Organizations: Independent Researcher

Abstract

Speculative sampling can accelerate decoding without changing the target distribution. Its standard guarantee assumes that logits processors, which enforce constraints or modify token scores, receive the appropriate token prefix and state at each verification position. We formalize this processor-prefix contract and present BADC (branch-aware differential conformance testing). BADC combines enumerated references on tiny random models, checks of processor inputs and state, and simultaneous confidence bounds on output-event probabilities. Preregistered, source-informed experiments on selected Transformers and llama-cpp-python releases confirm two causes: processors read unverified drafts or advance state without rollback. For one pilot-selected prompt on a trained checkpoint, fresh samples give a 60.26-percentage-point lower bound on the difference in the probability of a predefined first-token event between ordinary and prompt-lookup decoding, with nominal simultaneous 95% coverage under within-arm i.i.d. sampling. Controlled comparisons establish no cause-recovery advantage of directed selection over random selection, while permutation testing detects each tested family at least as often as BADC. These findings connect processor-interface failures to observable decoding differences without establishing diagnostic superiority or workload prevalence.

Figures & tables

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Correctness Forensics for Batch Speculative Decoding: Diagnosing the Ragged Tensor Problem

    Oct 26, 2025Ranran Haoran Zhang, Soumik Dey, Ashirbad Mishra +3Speculative DecodingEfficient Language Model Inference

  2. Entropy-Aware Token Rejection for Improving Speculative Decoding

    Dec 29, 2025Tiancheng Su, Meicong Zhang, Guoxiu HeSpeculative DecodingLLM Inference Acceleration