We present ROCKET, a training-free model compression method that achieves state-of-the-art performance in comparison with factorization, structured-sparsification and dynamic compression baselines. Operating under a global compression budget, ROCKET comprises two key innovations: First, it formulates layer-wise compression allocation as a multi-choice knapsack problem, selecting the optimal compression level for each layer to minimize total reconstruction error while adhering to a target model size. Second, it introduces a single-step sparse matrix factorization inspired by dictionary learning: using only a small calibration set, it sparsifies weight coefficients based on activation-weights sensitivity and then updates the dictionary in closed form via least squares bypassing iterative optimization, sparse coding, or backpropagation entirely. ROCKET consistently outperforms existing compression approaches across different model architectures at 20-50% compression rates. Notably, it retains over 90% of the original model's performance at 30% compression without any fine-tuning. Moreover, when applying a light fine-tuning phase, recovery is substantially enhanced: for instance, compressing Qwen3-14B to an 8B-parameter model and healing it with just 30 million tokens yields performance nearly on par with the original Qwen3-8B. The code for ROCKET is at github.com/mts-ai/ROCKET/tree/main.
Figures & tables
Figure 1: Overview of the proposed method. Left: Budget allocation formulated as a shortest-path problem on a directed graph, where nodes represent compression options and edges encode cost (reconstruction error), solved via dynamic programming to find the best sequence of operations over the discretized candidate set. Right: The selected path determines per-layer compression parameters (rank kℓ and sparsity sℓ ), which are then applied to each layer via eigendecomposition followed by structured hard thresholding sparsification ( T(⋅) ) of coefficients.
Accuracy ↑
Perplexity ↓
Method
CR
PIQA
HellaSwag
LAMBADA
ARC-e
ARC-c
SciQ
Race
MMLU
Avg.
WikiText
LAMBADA
Qwen3 8B
–
77.7
74.9
64.1
80.7
56.7
95.7
40.9
73.0
70.5
12
4.6
SVD-LLM
73.8
63.9
62.2
68.7
45.7
90.1
40.5
54.7
62.5
21
6.4
CoSpaDi
76.5
68.0
65.6
72.2
48.9
93.2
40.7
60.8
65.7
18
4.9
ROCKET
0.2
77.6
72.9
66.0
75.8
53.9
94.5
41.4
67.2
68.7
15
4.7
SVD-LLM
70.4
55.2
53.8
59.3
37.1
87.2
38.4
44.8
55.8
27
11
Table 1: Performance comparison of ROCKET vs SOTA SVD-LLM and CoSpaDi methods on Qwen3-8B at different compression ratios (CR). Best results are highlighted with bold .
Figure 2: Comparison of Accuracy and Inverse Log Perplexity for Llama3-8B and Llama3.2-1B.
Figure 3: Comparison of ROCKET with alternative budget allocation methods (Uniform, ARS [ 10 ] , Dobi-SVD [ 38 ] , and ARA [ 42 ] ) on three model configurations: Qwen3-8B at 20% and 40% pruning, and Llama2-7B at 40% pruning. Subplots show normalized performance on eight benchmarks (C4 perplexity is inverted, meaning higher value is better), scaled to each model’s dense baseline (value=1.0). ROCKET consistently retains the best performance.
Models
CR
Method
Quant.
PIQA
HellaSwag
WinoGrande
ARC_e
ARC_c
Avg. ( ↑ )
Drop ( ↓ )
Llama3.1-8B
–
Baseline
✗
0.80
0.59
0.74
0.81
0.51
0.69
0%
0.4
LLM-Pruner
✗
0.66
0.32
0.54
0.58
0.23
0.46
33.3%
SliceGPT
✗
0.62
0.40
0.53
0.49
0.25
0.46
33.3%
Bonsai
✗
0.59
0.29
0.49
0.47
0.18
0.41
40.6%
Wanda-sp
✗
0.57
0.28
0.50
0.44
0.17
0.39
43.5%
Dobi-SVD
✓
0.76
0.52
0.72
0.73
0.39
0.63
8.70%
Table 2: Performance comparison of ROCKET against depth- and sparsity-based pruning methods on Llama3.1-8B across compression levels and benchmarks. “Avg” denotes the average accuracy across all benchmarks, “Drop” indicates the relative accuracy drop percentage compared to the dense model, and “Quant.” indicates whether post-compression quantization is applied to achieve the target CR.
Figure 4: Exact state-space graph matching. Each state (i,p) represents having processed the first i layers with p scaled parameters retained. From (i,p) , the algorithm branches to all options for layer i , producing states (i+1,p+Δp) . The sink is reachable only from states with p≤pmax , enforcing the global compression ratio by construction.
Figure 5: Comparison between PyTorch’s built-in sparse-matrix vector multiplication and Macko across MLP layers in Qwen3-8B. Macko shows consistently better running time for large coefficients (gate and up projections), while being on par with PyTorch for smaller sparse matrices (down projection).
Method
20%
30%
40%
SVD-LLM
24.36
24.31
24.72
CoSpaDi
25.45
25.76
25.62
ROCKET
26.74
26.60
26.36
Appendix
Table 9: Throughput (tokens/s) for Qwen3-8B with batch_size = 1 and context_length = 256.
Method
Model
GPU
CPU
Energy Consumed (kWh)
Duration (s)
CO 2 Emissions (kg eq)
CoSpaDi
Llama3.2-1B
1 × NVIDIA A100-SXM4-40GB
AMD EPYC 7742 64-Core Processor
7.88
90080.97
0.782
ROCKET
Llama3.2-1B
1 × NVIDIA A100-SXM4-40GB
AMD EPYC 7742 64-Core Processor
0.0765
930
0.0337
Appendix
Table 10: Comparison of energy consumption, runtime, and CO 2 emissions for CoSpaDi and ROCKET using the Llama3.2-1B model.
Table 12: Full prompt-length sweep, relative tokens/s vs. dense (format: CoSpaDi / ROCKET).
Method
Stored Parameters
Dense
d1⋅d2
Rank- k SVD
k(d1+d2)
ROCKET
d1⋅k+s⋅d2
Appendix
Table 13: Per-layer memory footprint comparison for W∈Rd1×d2 .
Accuracy ↑
Perplexity ↓
Method
CR
PIQA
Hella Swag
LAMBADA
ARC-e
ARC-c
SciQ
Race
MMLU
Avg.
Wiki Text
LAMBADA
Llama3.2 1B
–
74.5
63.7
63.0
60.5
36.2
88.3
37.8
37.0
57.6
12
5.7
SVD-LLM
62.1
36.4
24.4
36.0
25.1
64.9
29.0
23.0
37.6
170
170
CoSpaDi
66.1
42.9
38.4
39.9
26.0
71.6
31.7
24.8
42.7
64
35
ROCKET
0.2
71.9
56.7
46
56.7
32.4
88.6
36.4
30.8
52.4
18
13
SVD-LLM
55.7
30.1
9.1
30.5
21.5
45.9
25.8
23.2
30.2
590
2,500
Appendix
Table 16: ROCKET comparison vs low-rank and SDL counterparts in data-aware scenarios on Llama3.2-1B at different compression ratios (CR). Best results are provided in bold All experiments are in training-free setup.
Accuracy ↑
Perplexity ↓
Method
CR
PIQA
HellaSwag
LAMBADA
ARC-e
ARC-c
SciQ
Race
MMLU
Avg.
WikiText
LAMBADA
Llama3 8B
–
80.7
79.1
75.6
77.7
53.5
93.9
40.3
62.2
70.4
7.3
3.1
SVD-LLM
71.1
58.4
59.3
55.5
34.0
86.4
35.5
32.6
54.1
41
11
CoSpaDi
75.2
66.5
73.8
66.5
41.6
89.5
38.2
42.8
61.8
20
4.3
ROCKET
0.2
76.9
74.8
73.6
73.7
47.1
92.7
40.7
54.9
66.8
11
3.8
SVD-LLM
65.8
46.4
38.1
41.9
27.7
70.0
31.8
27.2
43.6
150
61
Appendix
Table 17: Performance comparison of ROCKET vs SOTA SVD-LLM methods on Llama3-8B at different compression ratios (CR). Best results are highlighted with bold .
Figure 6: Average accuracy (relative to uncompressed model) as a function of model size for two compression ratios.
Accuracy ↑
Method
CR
IFEval
BBH
MATH
GPQA
MUSR
MMLU-Pro
Qwen3 8B
—
39.21
60.86
52.57
36.16
43.12
47.72
SVD-LLM
0.2
25.54
41.00
1.06
28.36
39.81
26.30
CoSpaDi
0.2
28.90
45.25
1.96
28.61
42.06
31.46
ROCKET
0.2
31.89
50.33
11.10
31.45
39.68
36.65
SVD-LLM
0.3
22.90
34.42
0.98
25.59
41.40
18.82
Appendix
Table 18: Performance comparison of ROCKET against CoSpaDi and SVD-LLM on a new set of benchmarks.
Model
CR
Avg. Accuracy ( ↑ )
Lambada PPL ( ↓ )
Gemma3-12B (dense)
–
72.88
4.16
SVD-LLM
0.3
51.70
74.61
ROCKET
0.3
68.50
4.23
ROCKET
0.4
62.75
8.36
Phi-4-14B (dense)
–
72.05
3.49
SVD-LLM
0.3
64.62
4.39
Appendix
Table 19: ROCKET performance across diverse LLM families and scales.
Method
Training-free
BoolQ
PIQA
HellaSwag
WinoGrande
ARC-e
ARC-c
OBQA
Avg.
Baseline
–
76.5
79.8
76.1
70.1
72.8
47.6
57.2
68.59
LLM-Pruner
✗
66.79
77.58
68.48
64.96
64.06
37.88
39.00
59.82
LoRAPrune
✗
65.82
79.31
70.00
62.76
65.87
37.69
39.14
60.05
WANDA
✓
65.75
74.70
64.52
59.35
60.65
36.26
39.40
57.23
ShortGPT
✗
68.26
72.28
61.70
63.77
60.22
39.00
41.60
58.12
LoRAShear †
✗
70.17
76.89
68.69
65.83
64.11
38.77
39.97
60.63
Appendix
Table 20: Comparison with depth/width pruning methods on Llama1-7B (20% CR).
Method
Effective CR
Avg. Acc. ↑
PPL ↓
ROCKET (20% CR)
0.20
66.8
11.04
ROCKET (20% CR) + INT8 GPTQ
≈ 0.60
66.6
11.04
Appendix
Table 21: ROCKET combined with INT8 GPTQ on Llama3-8B.
Method
PIQA
HellaSwag
Lambada
ARC-e
ARC-c
SciQ
Race
MMLU
WikiText
Perplexity
Avg. Acc.
Qwen3-14B (dense)
79.86
78.85
67.88
82.82
60.23
96.50
43.25
77.20
11
3.7
73.32
Qwen3-8B (dense)
77.70
74.90
64.10
80.70
56.70
95.70
40.90
73.00
12
4.6
70.46
ROCKET-Qwen3-8B (training-free)
72.68
62.63
70.26
67.76
44.19
91.20
39.80
59.99
25
3.8
63.56
ROCKET-Qwen3-8B (healed)
78.51
74.67
65.55
75.29
53.07
93.50
37.89
65.23
13
4.7
67.96
Appendix
Table 22: Post-compression healing results on Qwen3-8B models.
Accuracy ↑
Perplexity ↓
Calibration Data
CR
PIQA
HellaSwag
LAMBADA
ARC-e
ARC-c
SciQ
Race
MMLU
Avg. Acc.
WikiText word
LAMBADA PPL
None (Baseline)
0
74.5
63.7
63.0
60.5
36.2
88.3
37.8
37.0
57.6
12
5.7
FineWeb
0.2
71.9
56.7
46.0
56.7
32.4
88.6
36.4
30.8
52.4
18
13
PTB
0.2
70.3
54.9
47.3
55.2
31.9
87.8
34.5
26.7
51.1
21
13
WikiText
0.2
70.7
56.3
50.9
57.9
33.6
88.1
35.5
28.3
52.7
17
11
Alpaca
0.2
73.7
57.2
53.2
57.9
34.9
87.5
35.4
30.5
53.8
20
9.7
Appendix
Table 23: Ablation on calibration data for Llama3.2-1B at 20% compression (CR = 0.2). The first row shows the uncompressed baseline. Higher accuracy and lower perplexity are better.
Accuracy ↑
Perplexity ↓
Sparsification Strategy
CR
PIQA
HellaSwag
LAMBADA
ARC-e
ARC-c
SciQ
Race
MMLU
Avg. Acc.
WikiText word
LAMBADA PPL
None (Baseline)
0
74.5
63.7
63.0
60.5
36.2
88.3
37.8
37.0
57.6
12
5.7
ROCKET
0.2
71.9
56.7
46.0
56.7
32.4
88.6
36.4
30.8
52.4
18
13
Per-Row Sparsification
0.2
67.8
48.1
32.5
48.6
29.4
79.9
31.6
26.7
45.6
32
38
Global Importance Sparsification
0.2
69.5
54.7
42.7
53.5
30.7
85.7
35.5
28.1
50.1
19
17
Whitened-Space Importance Only
0.2
71.5
55.7
47.3
56.3
32.9
87.9
35.4
28.9
52.0
19
13
Appendix
Table 24: Ablation on sparsification strategies for Llama3.2-1B at 20% compression (CR = 0.2). All methods use the same calibration data and global parameter budget.
Error Metric
Avg. Acc. ↑
Perplexity ↓
Dense (Baseline)
57.6
12
L1 Distance
35.2
180
Mean Cos Columns
51.1
19
Spectral Distance
51.3
18
Frobenius (ours)
52.4
18
Appendix
Table 25: Ablation on reconstruction error metric for layer profiling in ROCKET for Llama3.2-1B at 20% compression.
Post-training compression of transformer models commonly relies on truncated singular value decomposition (SVD) strategies leading to low-rank approximations of the original model weights. However, the underlying single shared subspace modeling can degrade accuracy even at moderate compression. Sparse dictionary learning provides a more flexible union-of-subspaces representation, but existing approaches often suffer from costly iterative dictionary and coefficient updates. We propose COMPOT (Calibration-Optimized Matrix Procrustes Orthogonalization for Transformers), a training-free compression framework that uses a small calibration dataset to estimate a sparse weight factorization. COMPOT employs orthogonal dictionaries that enable closed-form Procrustes updates for the dictionary and analytical single-step sparse coding for the coefficients, reducing significantly the overall optimization cost. To handle heterogeneous layer sensitivity under a global compression budget, COMPOT further introduces a one-shot dynamic allocation strategy that adaptively redistributes layer-wise compression rates. Extensive experiments across diverse architectures and tasks show that COMPOT consistently delivers a superior quality-compression trade-off over strong low-rank and sparse baselines, while remaining fully compatible with post-training quantization for extreme compression.
Efficient deployment of large language models (LLMs) on mobile devices requires balancing performance with compute and memory constraints. Post-training compression is an effective approach to managing this trade-off, while requiring much less computation than in-training counterparts. We propose DictLLM, a dictionary-based compression scheme in which weight matrices within grouped layers are decomposed into a shared dictionary and layer-specific sparse coefficients, directly obtained from a pre-trained model without further training. Specifically, we cast DictLLM as a four-step post-training optimization. First, we obtain shared dictionaries via singular value decomposition (SVD) of input-scaled weight matrices, amortizing rank across grouped layers; second, we sparsify the layer-specific coefficients to the target compression ratio using a dictionary-aware Hessian; and third, we refine the dictionaries given the resulting sparse coefficients. Finally, we aggressively quantize both components using factor-specific Hessian information: input covariance for the dictionaries and the dictionary-aware Hessian for the coefficients. The resulting representation reduces memory footprint by storing a single shared dictionary per layer group, along with sparse layer-specific coefficients. On language modeling and zero-shot reasoning benchmarks, DictLLM consistently outperforms existing post-training low-rank compression methods and pushes the Pareto frontier between model size and performance. DictLLM compresses the original LLaMA-7B model by 5.1x with only a 2.9-point increase in WikiText-2 perplexity, achieving a 3.6x higher compression ratio than the state-of-the-art low-rank method.
Training-free compression methods for large language models (LLMs) often use calibration data to guide compression decisions. ROCKET, a recent method combining sparse-dictionary factorization with multi-choice knapsack problem (MCKP) allocation, derives its per-layer factorization from an output reconstruction objective but uses weight-space Frobenius error as the MCKP allocation cost. We investigate whether aligning the allocation cost with the output-space objective improves compressed model fidelity. On Qwen3-8B at 50% compression, our ROCKET-ActCost achieves +0.8 percentage points higher average accuracy across 8 zero-shot benchmarks (53.1% vs 52.3%), but increases WikiText perplexity by 16% (61.46 vs 52.98). This accuracy-perplexity tradeoff reveals that different allocation objectives favor different downstream metrics. The high correlation (>0.99) between weight-space and output-space errors limits allocation divergence, explaining the modest effect size. On Llama-3.2-1B at 20% compression, the two methods produce near-identical results (53.3% vs 53.5% accuracy, 14.45 vs 14.66 PPL), suggesting that the effect of the cost function is minor at lower compression ratios.