cs.AINov 7, 2026

How Short Is Too Short? Length-Conditional Calibration and Abstention for Detecting AI-Generated Text

Authors: Ritesh Janga, Justin Zhan

Organizations: University of Cincinnati, USA

Abstract

Detectors of AI-generated text are usually tuned and tested on passages of several hundred tokens, yet many real cases are short. We study score-based detection as a function of passage length. We give a closed form for how the false-positive rate of a threshold set on long text changes as length falls, and a formula for the number of tokens a detector needs to reach a target true-positive rate. We calibrate thresholds within length bins with conformal p-values, and we add a bin-level rule that abstains when the detector cannot be informative at that length. On synthetic scores the formulas agree with simulation, and binned calibration holds the false-positive rate at its nominal value at every length. On three zero-shot detectors (log-likelihood, log-rank, Binoculars), three corpora and two open-weight generators, binned calibration gives a mean false-positive rate of 5.0% in every detector-length cell from 10 to 200 tokens, averaged over 300 random document-level resplits. A single threshold set at 200 tokens gives 29% for Binoculars at 10 tokens and about 1% for log-likelihood, as the closed form predicts. No detector reaches an 80% true-positive rate, and the best, Binoculars at 200 tokens, reaches 59%, so the rule abstains in every cell at that floor and releases only the 200-token bins at floors of 0.3 and 0.5.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. 'Your AI Text is not Mine': Redefining and Evaluating AI-generated Text Detection under Realistic Assumptions

    Jun 3, 2026Nils Dycke, Marina Sakharova, Nico Daheim +1AI-Generated Text DetectionGenerative AI Evaluation

  2. Log-Likelihood, Simpson's Paradox, and the Detection of Machine-Generated Text

    May 7, 2026Tom Kempton, Viktor Drobnyi, Maeve Madigan +1AI-Generated Text DetectionLanguage Model Calibration

  3. On the Salience of Low-Probability Tokens for AI-Generated Text Detection: A Multiscale Uncertainty Perspective

    Jun 1, 2026Yikai Guo, Bin Wang, Xilai Fan +2AI-Generated Text DetectionLLM Uncertainty Estimation