cs.CLNov 6, 2026

Beyond Truncation: When Can LLMs Safely Omit Generated Text?

Authors: Mahmoud Alyosify, Mirna Imbabi, Eman Elkhamisi, Rahatara Ferdousi

Organizations: School of Computing, Queen’s University, Kingston, Ontario, Canada

Abstract

Instruction-tuned large language models (LLMs) answer at greater length than a request needs. We ask whether a training-free controller inside the decoding loop can remove that surplus while preserving content, judged against the simplest possible baseline: truncating the model's own answer to the same length. An entropy-gated logits processor removes 25.5-34.9% of tokens on MT-Bench development questions and 19.9-34.2% on a sealed test split within pre-registered BERTScore non-inferiority margins, yet we prove and confirm by audit that its output is always a prefix of the unmodified answer: the saving is truncation. A survey of 23 training-free mechanism families finds none that selects content better than truncation without oracle labels. We therefore introduce Entailment-Licensed In-Decoding Elision (ELIDE), which, at each sentence boundary, withholds a span from the reader only when a natural-language inference model judges it entailed by the text already shown, while leaving the model's generation unchanged. ELIDE beats matched-length truncation on three model families, but reading 936 of its elisions on 670 public prompts shows that 59% harm the answer, a failure that no automatic measure detects. Scoping the licence to its discourse unit reduces the harmful share to about one elision in ten on two fresh tests.

Figures & tables

Explore similar work

CardsList
  1. Beyond Truncation: Rethinking LLM Decoding as Ensemble Pruning

    Sep 16, 2026Dunyao Xue, Chengshuo Du, Zhengbo Wang +2Language Model DecodingDiverse Text Generation

  2. Say Less, Mean Better: Training-Free Verbosity Control for Language Models

    Nov 3, 2026Mahmoud Alyosify, Mirna Imbabi, Eman Elkhamisi +1Efficient Language Model Inference

  3. Constrained Decoding: Harmful or Beneficial? Revisiting Its Impact on LLM Performance

    Oct 26, 2026Ngoc Trinh Hung Nguyen, Alonso Silva, Armen Aghasaryan +1Constrained DecodingConstraint-Guided Reasoning in Language Models