Reasoning-Aware Compression: Identifying and Protecting Vulnerable Reasoning Circuits for Energy-Efficient LLM Deployment

Leonard Twagirayezu; Prasenjit Mitra

cs.AIAug 31, 2026

Reasoning-Aware Compression: Identifying and Protecting Vulnerable Reasoning Circuits for Energy-Efficient LLM Deployment

Authors: Leonard Twagirayezu, Prasenjit Mitra

Organizations: Carnegie Mellon University Africa Kigali, Rwanda

Abstract

Large Reasoning Models (LRMs) impose substantial energy costs during deployment, yet current compression methods apply uniform quantization across all components, risking damage to critical reasoning circuits. We present a reasoning-aware compression framework that benchmarks quantization conditions across five reasoning benchmarks, GSM8K, FOLIO, MATH-500, ProofWriter, and MuSiQue, with hardware-level GPU energy measurement; profiles per-module INT4 vulnerability across all 196-224 (layer, projection) pairs via a perturbation sweep on a held-out calibration split, then selectively restores the most sensitive circuits to FP16. Three findings emerge. First, INT4 quantization can increase energy by extending reasoning chains; a 25% power reduction becomes a net energy increase on GSM8K. Second, vulnerability is task-dependent: attention projections are more critical for mathematical reasoning, and sensitivity patterns differ by architecture in logical inference. Third, selective compression achieves Pareto-optimal points inaccessible to uniform methods: R1-Qwen-7B Top-10% on ProofWriter gains +12 pp over FP16 at -9.7% energy, validated on held-out data across five reasoning benchmarks.

Figures & tables

Dataset	Task Type	Split	Samples/Experiment	Experiments	Maximum Tokens.
GSM8K	Arithmetic	test	30 (non-overlap)	8	2,048
FOLIO	Logic/NLI	validation	30 (offset)	8	1,024
MATH-500	Competition math	test	30 (offset)	8	2,048
ProofWriter	Formal deduction	train	30 (offset)	8	1,024
MuSiQue	Multi-hop QA	validation	30 (offset)	8	1,024

Table 1: Dataset Configuration Summary

Condition	Description
FP16 Baseline	Full float16 precision — accuracy upper bound and energy lower bound.
Full INT4	Uniform 4-bit NormalFloat (NF4) quantization via BitsAndBytes across all linear projections.
INT4+Attention	Attention projections ( q, k, v, o ) restored to FP16; MLP projections remain INT4.
INT4+MLP	MLP projections ( gate, up, down ) restored to FP16; attention projections remain INT4.
Selective Comp.	INT4 with top 10%, 20%, 30%, 40% most vulnerable layers restored to FP16 via perturbation sweep rankings.

Table 2: Compression Conditions

	DeepSeek-R1-Distill-Llama-8B				DeepSeek-R1-Distill-Qwen-7B
Condition	Acc (%)	Energy (J)	Tokens	Power (W)	Acc (%)	Energy (J)	Tokens	Power (W)
FOLIO
FP16	32.50	2,759	500	182.7	45.83	2,196	454	171.3
Full INT4	36.25	2,824 ∗	504	135.4	44.17	2,043	423	134.3
INT4+Attention	35.42	2,649	488	152.0	45.00	2,079	439	144.6
INT4+MLP	34.58	3,000 ∗	502	162.1	42.92	2,206 ∗	430	150.7

Table 3: Grouped protection results (held-out evaluation split, 50/50 partition). Bold = best INT4 condition per model-dataset pair. ∗ = energy increase relative to FP16. Mean across 8 experiments of 30 samples each.

Condition	Prot.	FOLIO Acc	GSM8K Acc	FOLIO E (J)	GSM8K E (J)	FOLIO Tok	GSM8K Tok	FOLIO Acc	GSM8K Acc	FOLIO E (J)	GSM8K E (J)	FOLIO Tok	GSM8K Tok
		DeepSeek-R1-Distill-Qwen-7B						DeepSeek-R1-Distill-Llama-8B
FP16 Reference	all	44.00	94.00	2,301	3,308	460	679	33.33	79.33	2,590	4,314	499	771
Full INT4	0	43.33	93.33	2,125	3,208	425	669	36.67	70.00	2,595	6,197	504	1,092
Top 10%	19/22	38.67	92.00	2,195	3,471	442	691	35.33	70.67	2,562	6,358	504	1,148
Top 20%	39/44	36.67	92.00	2,261	3,513	450	703	34.00	78.00	2,597	6,082	491	1,089
Top 30%	58/67	36.67	92.67	2,221	3,431	435	689	36.67	75.33	2,633	5,794	500	1,038

Table 4: Selective compression: accuracy (%), energy (J/query), output tokens, and power (W) — FOLIO and GSM8K (held-out evaluation split). Bold = best accuracy per model-dataset pair. Shaded row = FP16 reference.

Condition	Prot.	MATH Acc	ProofW Acc	MATH E (J)	ProofW E (J)	MATH Tok	ProofW Tok	MATH Acc	ProofW Acc	MATH E (J)	ProofW E (J)	MATH Tok	ProofW Tok
		DeepSeek-R1-Distill-Qwen-7B						DeepSeek-R1-Distill-Llama-8B
FP16 Reference	all	66.25	72.00	7,934	1,931	1,467	391	57.50	34.00	8,092	2,596	1,396	494
Full INT4	0	66.25	82.00	7,519	1,796	1,443	368	56.67	36.00	8,469	2,612	1,508	501
Top 10%	19/22	57.08	84.00	8,095	1,743	1432	354	46.25	32.00	8,519	2,658	1491	509
Top 20%	39/44	57.50	79.33	8,033	1,869	1439	371	49.17	32.67	8,392	2,667	1412	506
Top 30%	58/67	59.17	79.33	7,996	1,868	1298	364	46.67	32.00	7,863	2,744	1440	507

Table 5: Selective compression: accuracy (%), energy (J/query), output tokens, and power (W) — MATH-500 and ProofWriter (held-out evaluation split for ProofWriter; full-dataset results for MATH-500). Bold = best accuracy per model-dataset pair. Shaded row = FP16 reference.

Condition	Prot.	Acc (%)	Energy (J)	Tokens	Power (W)	Acc (%)	Energy (J)	Tokens	Power (W)
		DeepSeek-R1-Distill-Qwen-7B				DeepSeek-R1-Distill-Llama-8B
FP16 Reference	all	7.87	2,552	462	181.2	14.31	2,658	475	165.4
Full INT4	0	8.00	2,465	448	147.9	14.51	2,543	471	138.7
Top 10%	19/22	6.53	2,492	462	147.7	11.92	2,736	475	141.3
Top 20%	39/44	5.27	2,430	454	148.3	13.42	2,603	478	139.4
Top 30%	58/67	8.71	2,450	464	154.6	12.88	2,592	481	140.1

Table 6: Selective compression: accuracy (%), energy (J/query), output tokens, and power (W) — MuSiQue (held-out evaluation split). Bold = best accuracy per model. Shaded row = FP16 reference.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 1: Vulnerability heatmap for R1-Qwen-7B all benchmark. Rows = projection types; columns = layer indices (0–27).

Figure 2: Vulnerability heatmap for R1-Llama-8B for all benchmarks. Rows = projection types; columns = layer indices (0-31). The sparser pattern relative to R1-Qwen-7B is consistent with Full INT4 frequently matching FP16 accuracy on Llama-8B.

Model	Dataset	Baseline	Max Drop	Safe Pairs
R1-Llama-8B	FOLIO	40.0%	$+$ 0.0 pp	224/224
	GSM8K	53.3%	$+$ 13.3 pp	214/224
	MATH-500	26.7%	$+$ 3.3 pp	221/224
	ProofWriter	56.7%	$+$ 36.7 pp	104/224
	MuSiQue	0.0%	$+$ 0.0 pp	224/224
R1-Qwen-7B	FOLIO	43.3%	$+$ 10.0 pp	175/196

Appendix

Table 7: Per-module vulnerability summary from calibration-split perturbation sweep. Max drop = accuracy drop of most vulnerable (layer, projection) pair. Safe = pairs with zero or negative drop.

R1-Llama-8B Full INT4 = 37.92%
Dataset	$K$ %	Top- $K$	Random- $K$	Bottom- $K$
FOLIO	10	38.33	37.64	40.00
FOLIO	20	38.33	38.61	40.83
FOLIO	30	38.33	14.72	13.33
FOLIO	40	17.50	15.42	9.17
R1-Llama-8B Full INT4 = 73.33%

Appendix

Table 8: Top-

K

% vs. Random-

K

% vs. Bottom-

K

% accuracy (%). Random-

K

% is mean of 3 seeds. Full INT4 baseline shown for reference.

Model	Dataset	Condition	$\Delta$ Acc (pp)	Sig.	$\Delta$ E (J)	Sig.
R1-Qwen-7B	FOLIO	Full INT4	$-$ 1.67	n.s.	$-$ 153	†
R1-Qwen-7B	FOLIO	INT4+Attn	$-$ 0.83	n.s.	$-$ 117	n.s.
R1-Qwen-7B	FOLIO	INT4+MLP	$-$ 2.92	n.s.	$+$ 10	n.s.
R1-Qwen-7B	GSM8K	Full INT4	$-$ 0.42	n.s.	$-$ 19	n.s.
R1-Qwen-7B	GSM8K	INT4+Attn	$-$ 1.25	n.s.	$-$ 187	n.s.
R1-Qwen-7B	GSM8K	INT4+MLP	$-$ 0.42	n.s.	$+$ 255	n.s.

Appendix

Table 9: Two-sample

t

-test

p

-values for accuracy and energy differences between each INT4 condition and FP16 baseline (

n{=}5

experiments, held-out split).

{}^{***}p{<}0.001

{}^{**}p{<}0.01

{}^{*}p{<}0.05

{}^{\dagger}p{<}0.10

, n.s. not significant.

LLM Quantization

Large language model quantization reduces numerical precision to lower memory use and accelerate training or inference.

See all 391 papers

What Survives When You Compress a Recursive Reasoner for the Edge?
Jun 25, 2026Pearse Jim, Steven Kolawole, Opegbemi Matthias Busoye +2Model Compression Efficient Neural Network Inference
ML Collective · Carnegie Mellon University
Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Models
Jun 24, 2026Xinyu Lian, Walid Krichene, Beichen Huang +4LLM Inference Efficiency LLM Quantization
University of Illinois Urbana-Champaign · Microsoft · Anyscale +1
Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoning
Apr 10, 2026Yi Sui, Chaozhuo Li, Dawei SongCoT Compression Efficient Language Model Reasoning
Beijing Institute of Technology, China · Beijing University of Posts and Telecommunications, China · The Open University, UK

Jun 25, 2026J/K move · Enter open · S save

What Survives When You Compress a Recursive Reasoner for the Edge?

Pearse Jim, Steven Kolawole, Opegbemi Matthias Busoye, Glory Bagai+1

ML Collective · Carnegie Mellon University

Model Compression Efficient Neural Network Inference Recursive Reasoning Models Structured Reasoning Neural Network Quantization Neural Network Pruning

Open paper AskPDF arXiv

Recursive reasoning models solve hard structured tasks with a few million parameters by iterating a latent state, but deploying them on edge hardware means compressing them -- and quantization noise compounds across recursive cycles rather than accumulating over output tokens, so single-pass intuitions fail. Here, we ask what survives. Across a full precision sweep and three tasks, aggressive compression preserves local prediction but destroys global reasoning: pruning, distillation, and linear attention drive puzzle-exact accuracy to zero while cell accuracy holds. Naive INT4 is different: its damage is architectural, striking MLP-mixing recursion but not attention on the same task, and per-channel calibration reverses it without retraining, where quantization-aware training does not. We also introduce carry-trajectory fidelity, the cosine similarity to the full-precision reasoning path, as a label-free signal that predicts both the damage and the recovery before any task evaluation, with a threshold that transfers across tasks. The result is a deployment recipe: flash-streamed embeddings remove a 99.4 MB bottleneck, one recursive cycle matches full-depth accuracy at 6x fewer FLOPs, and calibrated INT4 brings the backbone to 3.3 MB, inside a 4 MB budget.

Abstract

LLM Quantization

Explore similar work

What Survives When You Compress a Recursive Reasoner for the Edge?

Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Models

Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoning