TopoQuant: Topological Framework to Characterize Behavioral Shifts in Quantized Large Language Models

Baha Rababah; Shahzeb Qamar; Lorenz Sparrenberg; Rafet Sifa; Murat Kantarcioglu; Carson K. Leung; Cuneyt Gurcan Akcora

cs.AIOct 31, 2026

TopoQuant: Topological Framework to Characterize Behavioral Shifts in Quantized Large Language Models

Authors: Baha Rababah, Shahzeb Qamar, Lorenz Sparrenberg, Rafet Sifa, Murat Kantarcioglu, Carson K. Leung, Cuneyt Gurcan Akcora

Organizations: Department of Computer Science, University of Manitoba, Winnipeg, MB, Canada · Applied Computer Education Department, Red River College Polytechnic, Winnipeg, MB, Canada · Bonn-Aachen International Center for IT (b-it), University of Bonn, Bonn, Germany · Department of Computer Science, Virginia Tech University, Blacksburg, VA, USA ·

Abstract

Post-training quantization (PTQ) reduces the com putational cost of large language models (LLMs), but it can introduce internal structural and behavioral changes that are not captured by traditional metrics such as accuracy and per plexity. We present TopoQuant, a topological framework that uses topological data analysis (TDA) to quantify structural drift in self-attention weights and relate it to behavioral changes in quantized LLMs. Across four LLM families and multiple quantization levels, we show that topological drift remains stable at moderate compression levels but increases sharply below 4 bit precision, closely tracking declines in attention agreement. Our results reveal a strong relationship between topological drift and attention agreement, indicating that structural changes in model parameters closely reflect behavioral degradation under quantization. We further demonstrate that topological features can accurately predict attention consistency using lightweight machine learning models, achieving R2 scores of up to 0.85. Predicting attention agreement enables efficient assessment of be havioral consistency in quantized LLMs, allowing quantization induced degradation to be detected without running expensive downstream tasks. The results indicate that topological drift pro vides an efficient early-warning signal for quantization-induced degradation and can help identify safe compression limits without extensive downstream evaluation.

Figures & tables

Fig. 1: Overview of the TopoQuant framework. The framework consists of four stages: model quantization and weight extraction, topological drift measurement, attention agreement analysis, and lightweight machine learning prediction. Topological features extracted from persistence diagrams are combined with model characteristics to predict attention consistency between base and quantized models.

Fig. 2: Sliced Wasserstein Distance (SWD) drift across quantization levels for Query ( $\mathbf{Q}$ ), Key ( $\mathbf{K}$ ), and Value ( $\mathbf{V}$ ) projection matrices.

		Llama-3B			Vicuna-7B			Mistral-7B			Llama-8B
Metric	Quant	Arc	Hell.	Wino.	Arc	Hell.	Wino.	Arc	Hell.	Wino.	Arc	Hell.	Wino.
Top-1 Agreement	Q8_0	0.74	0.88	0.68	0.99	1.00	1.00	0.99	1.00	0.99	0.75	0.87	0.71
	Q6_K	0.74	0.88	0.67	0.97	1.00	0.99	0.97	0.98	0.96	0.75	0.87	0.71
	Q5_0	0.73	0.88	0.68	0.94	0.99	0.98	0.93	0.96	0.92	0.73	0.86	0.68
	Q5_K	0.74	0.88	0.67	0.96	0.99	0.99	0.96	0.97	0.95	0.75	0.86	0.71
	Q4_0	0.72	0.87	0.67	0.88	0.99	0.95	0.89	0.93	0.83	0.60	0.85	0.68

TABLE I: Attention Agreement Metrics across Quantization Configurations. Evaluation of Top-3 Intersection over Union (IoU) and Top-1 Agreement for ARC, HellaSwag, and Winogrande datasets across four LLM architectures.

Fig. 3: Quantization degradation trajectories across attention projection matrices. Each curve tracks an LLM model across eight quantization level (ranging from high-precision $\mathbf{q8\_0}$ to extreme low-bit $\mathbf{q2\_k}$ , indicated by marker shape). The $x$ -axis shows the average Sliced Wasserstein Distance (SWD) structural drift, while the $y$ -axis displays downstream behavioral preservation measured via Top-3 IoU Similarity on the ARC benchmark. The exponential divergence in Query ( $Q$ ) and Key ( $K$ ) projections highlights their structural vulnerability, validating SWD as an efficient proxy for tracking performance cliffs without full benchmark re-evaluation.

Feature Matrix	Algorithm	LLM	TDA	Comb.	LLM	TDA	Comb.	LLM	TDA	Comb.
		Arc			Hell.			Wino.
Query ( $\mathbf{Q}$ )	LinearRegression	0.68	0.26	0.76	0.53	0.27	0.83	0.75	0.22	0.81
	XGBoost	0.49	0.21	0.66	0.47	0.11	0.51	0.56	0.20	0.66
	RandomForest	0.67	0.33	0.83	0.72	0.35	0.85	0.74	0.31	0.83
Key ( $\mathbf{K}$ )	LinearRegression	0.68	0.21	0.73	0.53	0.22	0.82	0.75	0.19	0.82
	XGBoost	0.49	0.21	0.65	0.47	0.09	0.50	0.56	0.19	0.65

TABLE II: Ablation Study of Feature Representations on Attention Agreement (

R^{2}

for Top-3 IoU) across Benchmarks. Comparison across LLM features only, TDA features only, and Combined (

\text{LLM}+\text{TDA}

) feature configurations. Top values per column are highlighted in green and lowest values in red .

Quant	TopoQuant	Downstream
Q8_0	862	4619
Q6_K	937	4647
Q5_0	931	4603
Q5_K	937	4627
Q4_0	903	4709
Q4_K	914	4636

TABLE III: TopoQuant overhead vs. downstream runtime: To demonstrate the effectiveness of our framework, we compare the time required to compute TopoQuant (persistence diagrams, topological distances, and attention agreement on the ARC dataset) against the time required for downstream task execution alone on Llama-3.1-8B .

LLM Quantization

Large language model quantization reduces numerical precision to lower memory use and accelerate training or inference.

See all 416 papers

The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs
Jul 9, 2026Baha Rababah, Cuneyt Gurcan Akcora, Carson K. LeungLLM Evaluation Quantization Sensitivity Analysis
Department of Computer Science, University of Manitoba, Canada · AI Initiative, University of Central Florida, USA
Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels
May 2, 2026Plawan Kumar Rath, Rahul MaliakkalLLM Quantization Social Bias in Language Models
Meta
From Signal Degradation to Computation Collapse: Uncovering the Two Failure Modes of LLM Quantization
Apr 21, 2026Chenxi Zhou, Pengfei Cao, Jiang Li +4LLM Quantization Post-Training Quantization
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences · College of Computer Science, Inner Mongolia University +1

Jul 9, 2026J/K move · Enter open · S save

The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs

Baha Rababah, Cuneyt Gurcan Akcora, Carson K. Leung

Department of Computer Science, University of Manitoba, Canada · AI Initiative, University of Central Florida, USA

LLM Evaluation Quantization Sensitivity Analysis Post-Training Quantization Low-Bit Quantization Neural Network Quantization

Open paper AskPDF arXiv

Post-training quantization is widely used to deploy large language models in resource-constrained settings, yet its evaluation relies almost exclusively on accuracy and perplexity. We show that these metrics fail to capture behavioral changes induced by quantization. We introduce correctness agreement, a decision-level metric that measures overlap in correct predictions between a base model and its quantized variants, independent of absolute accuracy. Across multiple models and quantization schemes from 8-bit to 2-bit, we find that behavioral divergence emerges under moderate quantization even when task performance appears preserved. To explain this effect, we analyze quantization as a structural operator on attention weights and quantify layer-wise distortions using statistical and distributional measures. Our results reveal non-linear breakpoints at low bit-widths and show that query and key projections are consistently more sensitive than value and output projections. These findings expose an illusion of equivalence between base and quantized models and motivate behavioral evaluation beyond conventional performance metrics.

Abstract

LLM Quantization

Explore similar work

The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs

Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels

From Signal Degradation to Computation Collapse: Uncovering the Two Failure Modes of LLM Quantization