cs.CLNov 8, 2026

Language model capabilities fail in a shared, predictable order under compression and perturbation

Authors: Zhixiang Wang, Ulas Bagci

Organizations: Department of Radiology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA

Abstract

Compression can preserve a language model's average accuracy while selectively degrading individual capabilities. Whether these losses follow method-specific patterns or a transferable order is unclear. We applied seven perturbation and compression methods to eight instruction-tuned models, scoring 16 tasks in four answer formats against preregistered predictions. Short-answer capabilities failed in nearly the same order under four random perturbations with different mechanisms (median correlation 0.90) and, within each model, under pruning and quantization (0.875). Intact answer margin, answer frequency and answer format predicted this order. Fragility profiles measured under random perturbation predicted which tasks and facts compression removes first and, after four-task calibration, held-out task accuracy (median absolute error 0.11-0.12, against 0.20-0.21 for uniform decline); profiles borrowed from other model families did almost as well. The order did not transfer across models under GPTQ quantization or hold for long-form tasks. Within short answers, what compression removes is predictable before deployment.

Figures & tables

Explore similar work

CardsList
  1. ACE-Compression: An Ability-Centric Evaluation of Compressed Large Language Models

    Oct 20, 2026Jiahui Ning, Liang Chen, Yuchen Liang +2LLM CompressionLLM Evaluation

  2. Compression Technique Interactions in Small Language Models: A Factorial Ablation Study

    Oct 30, 2026Bhushit Nema, Siddharth Krishnamurthy, Anjali KadiyalaSmall Language ModelsLLM Quantization

  3. Does Compression Preserve Uncertainty? A Unified Benchmark for Quantized and Sparse LLMs via Conformal Prediction

    Jun 1, 2026Yujia Tong, Yuxi Wang, Yunyang Wan +3LLM QuantizationLLM Evaluation