We present ROCKET, a training-free model compression method that achieves state-of-the-art performance in comparison with factorization, structured-sparsification and dynamic compression baselines. Operating under a global compression budget, ROCKET comprises two key innovations: First, it formulates layer-wise compression allocation as a multi-choice knapsack problem, selecting the optimal compression level for each layer to minimize total reconstruction error while adhering to a target model size. Second, it introduces a single-step sparse matrix factorization inspired by dictionary learning: using only a small calibration set, it sparsifies weight coefficients based on activation-weights sensitivity and then updates the dictionary in closed form via least squares bypassing iterative optimization, sparse coding, or backpropagation entirely. ROCKET consistently outperforms existing compression approaches across different model architectures at 20-50% compression rates. Notably, it retains over 90% of the original model's performance at 30% compression without any fine-tuning. Moreover, when applying a light fine-tuning phase, recovery is substantially enhanced: for instance, compressing Qwen3-14B to an 8B-parameter model and healing it with just 30 million tokens yields performance nearly on par with the original Qwen3-8B. The code for ROCKET is at github.com/mts-ai/ROCKET/tree/main.
Figures & tables
Figure 1: Overview of the proposed method. Left: Budget allocation formulated as a shortest-path problem on a directed graph, where nodes represent compression options and edges encode cost (reconstruction error), solved via dynamic programming to find the best sequence of operations over the discretized candidate set. Right: The selected path determines per-layer compression parameters (rank kℓ and sparsity sℓ ), which are then applied to each layer via eigendecomposition followed by structured hard thresholding sparsification ( T(⋅) ) of coefficients.
Accuracy ↑
Perplexity ↓
Method
CR
PIQA
HellaSwag
LAMBADA
ARC-e
ARC-c
SciQ
Race
MMLU
Avg.
WikiText
LAMBADA
Qwen3 8B
–
77.7
74.9
64.1
80.7
56.7
95.7
40.9
73.0
70.5
12
4.6
SVD-LLM
73.8
63.9
62.2
68.7
45.7
90.1
40.5
54.7
62.5
21
6.4
CoSpaDi
76.5
68.0
65.6
72.2
48.9
93.2
40.7
60.8
65.7
18
4.9
ROCKET
0.2
77.6
72.9
66.0
75.8
53.9
94.5
41.4
67.2
68.7
15
4.7
SVD-LLM
70.4
55.2
53.8
59.3
37.1
87.2
38.4
44.8
55.8
27
11
Table 1: Performance comparison of ROCKET vs SOTA SVD-LLM and CoSpaDi methods on Qwen3-8B at different compression ratios (CR). Best results are highlighted with bold .
Figure 2: Comparison of Accuracy and Inverse Log Perplexity for Llama3-8B and Llama3.2-1B.
Figure 3: Comparison of ROCKET with alternative budget allocation methods (Uniform, ARS [ 10 ] , Dobi-SVD [ 38 ] , and ARA [ 42 ] ) on three model configurations: Qwen3-8B at 20% and 40% pruning, and Llama2-7B at 40% pruning. Subplots show normalized performance on eight benchmarks (C4 perplexity is inverted, meaning higher value is better), scaled to each model’s dense baseline (value=1.0). ROCKET consistently retains the best performance.
Models
CR
Method
Quant.
PIQA
HellaSwag
WinoGrande
ARC_e
ARC_c
Avg. ( ↑ )
Drop ( ↓ )
Llama3.1-8B
–
Baseline
✗
0.80
0.59
0.74
0.81
0.51
0.69
0%
0.4
LLM-Pruner
✗
0.66
0.32
0.54
0.58
0.23
0.46
33.3%
SliceGPT
✗
0.62
0.40
0.53
0.49
0.25
0.46
33.3%
Bonsai
✗
0.59
0.29
0.49
0.47
0.18
0.41
40.6%
Wanda-sp
✗
0.57
0.28
0.50
0.44
0.17
0.39
43.5%
Dobi-SVD
✓
0.76
0.52
0.72
0.73
0.39
0.63
8.70%
Table 2: Performance comparison of ROCKET against depth- and sparsity-based pruning methods on Llama3.1-8B across compression levels and benchmarks. “Avg” denotes the average accuracy across all benchmarks, “Drop” indicates the relative accuracy drop percentage compared to the dense model, and “Quant.” indicates whether post-compression quantization is applied to achieve the target CR.
Figure 4: Exact state-space graph matching. Each state (i,p) represents having processed the first i layers with p scaled parameters retained. From (i,p) , the algorithm branches to all options for layer i , producing states (i+1,p+Δp) . The sink is reachable only from states with p≤pmax , enforcing the global compression ratio by construction.
Figure 5: Comparison between PyTorch’s built-in sparse-matrix vector multiplication and Macko across MLP layers in Qwen3-8B. Macko shows consistently better running time for large coefficients (gate and up projections), while being on par with PyTorch for smaller sparse matrices (down projection).
Method
20%
30%
40%
SVD-LLM
24.36
24.31
24.72
CoSpaDi
25.45
25.76
25.62
ROCKET
26.74
26.60
26.36
Appendix
Table 9: Throughput (tokens/s) for Qwen3-8B with batch_size = 1 and context_length = 256.
Method
Model
GPU
CPU
Energy Consumed (kWh)
Duration (s)
CO 2 Emissions (kg eq)
CoSpaDi
Llama3.2-1B
1 × NVIDIA A100-SXM4-40GB
AMD EPYC 7742 64-Core Processor
7.88
90080.97
0.782
ROCKET
Llama3.2-1B
1 × NVIDIA A100-SXM4-40GB
AMD EPYC 7742 64-Core Processor
0.0765
930
0.0337
Appendix
Table 10: Comparison of energy consumption, runtime, and CO 2 emissions for CoSpaDi and ROCKET using the Llama3.2-1B model.
Table 12: Full prompt-length sweep, relative tokens/s vs. dense (format: CoSpaDi / ROCKET).
Method
Stored Parameters
Dense
d1⋅d2
Rank- k SVD
k(d1+d2)
ROCKET
d1⋅k+s⋅d2
Appendix
Table 13: Per-layer memory footprint comparison for W∈Rd1×d2 .
Accuracy ↑
Perplexity ↓
Method
CR
PIQA
Hella Swag
LAMBADA
ARC-e
ARC-c
SciQ
Race
MMLU
Avg.
Wiki Text
LAMBADA
Llama3.2 1B
–
74.5
63.7
63.0
60.5
36.2
88.3
37.8
37.0
57.6
12
5.7
SVD-LLM
62.1
36.4
24.4
36.0
25.1
64.9
29.0
23.0
37.6
170
170
CoSpaDi
66.1
42.9
38.4
39.9
26.0
71.6
31.7
24.8
42.7
64
35
ROCKET
0.2
71.9
56.7
46
56.7
32.4
88.6
36.4
30.8
52.4
18
13
SVD-LLM
55.7
30.1
9.1
30.5
21.5
45.9
25.8
23.2
30.2
590
2,500
Appendix
Table 16: ROCKET comparison vs low-rank and SDL counterparts in data-aware scenarios on Llama3.2-1B at different compression ratios (CR). Best results are provided in bold All experiments are in training-free setup.
Accuracy ↑
Perplexity ↓
Method
CR
PIQA
HellaSwag
LAMBADA
ARC-e
ARC-c
SciQ
Race
MMLU
Avg.
WikiText
LAMBADA
Llama3 8B
–
80.7
79.1
75.6
77.7
53.5
93.9
40.3
62.2
70.4
7.3
3.1
SVD-LLM
71.1
58.4
59.3
55.5
34.0
86.4
35.5
32.6
54.1
41
11
CoSpaDi
75.2
66.5
73.8
66.5
41.6
89.5
38.2
42.8
61.8
20
4.3
ROCKET
0.2
76.9
74.8
73.6
73.7
47.1
92.7
40.7
54.9
66.8
11
3.8
SVD-LLM
65.8
46.4
38.1
41.9
27.7
70.0
31.8
27.2
43.6
150
61
Appendix
Table 17: Performance comparison of ROCKET vs SOTA SVD-LLM methods on Llama3-8B at different compression ratios (CR). Best results are highlighted with bold .
Figure 6: Average accuracy (relative to uncompressed model) as a function of model size for two compression ratios.
Accuracy ↑
Method
CR
IFEval
BBH
MATH
GPQA
MUSR
MMLU-Pro
Qwen3 8B
—
39.21
60.86
52.57
36.16
43.12
47.72
SVD-LLM
0.2
25.54
41.00
1.06
28.36
39.81
26.30
CoSpaDi
0.2
28.90
45.25
1.96
28.61
42.06
31.46
ROCKET
0.2
31.89
50.33
11.10
31.45
39.68
36.65
SVD-LLM
0.3
22.90
34.42
0.98
25.59
41.40
18.82
Appendix
Table 18: Performance comparison of ROCKET against CoSpaDi and SVD-LLM on a new set of benchmarks.
Model
CR
Avg. Accuracy ( ↑ )
Lambada PPL ( ↓ )
Gemma3-12B (dense)
–
72.88
4.16
SVD-LLM
0.3
51.70
74.61
ROCKET
0.3
68.50
4.23
ROCKET
0.4
62.75
8.36
Phi-4-14B (dense)
–
72.05
3.49
SVD-LLM
0.3
64.62
4.39
Appendix
Table 19: ROCKET performance across diverse LLM families and scales.
Method
Training-free
BoolQ
PIQA
HellaSwag
WinoGrande
ARC-e
ARC-c
OBQA
Avg.
Baseline
–
76.5
79.8
76.1
70.1
72.8
47.6
57.2
68.59
LLM-Pruner
✗
66.79
77.58
68.48
64.96
64.06
37.88
39.00
59.82
LoRAPrune
✗
65.82
79.31
70.00
62.76
65.87
37.69
39.14
60.05
WANDA
✓
65.75
74.70
64.52
59.35
60.65
36.26
39.40
57.23
ShortGPT
✗
68.26
72.28
61.70
63.77
60.22
39.00
41.60
58.12
LoRAShear †
✗
70.17
76.89
68.69
65.83
64.11
38.77
39.97
60.63
Appendix
Table 20: Comparison with depth/width pruning methods on Llama1-7B (20% CR).
Method
Effective CR
Avg. Acc. ↑
PPL ↓
ROCKET (20% CR)
0.20
66.8
11.04
ROCKET (20% CR) + INT8 GPTQ
≈ 0.60
66.6
11.04
Appendix
Table 21: ROCKET combined with INT8 GPTQ on Llama3-8B.
Method
PIQA
HellaSwag
Lambada
ARC-e
ARC-c
SciQ
Race
MMLU
WikiText
Perplexity
Avg. Acc.
Qwen3-14B (dense)
79.86
78.85
67.88
82.82
60.23
96.50
43.25
77.20
11
3.7
73.32
Qwen3-8B (dense)
77.70
74.90
64.10
80.70
56.70
95.70
40.90
73.00
12
4.6
70.46
ROCKET-Qwen3-8B (training-free)
72.68
62.63
70.26
67.76
44.19
91.20
39.80
59.99
25
3.8
63.56
ROCKET-Qwen3-8B (healed)
78.51
74.67
65.55
75.29
53.07
93.50
37.89
65.23
13
4.7
67.96
Appendix
Table 22: Post-compression healing results on Qwen3-8B models.
Accuracy ↑
Perplexity ↓
Calibration Data
CR
PIQA
HellaSwag
LAMBADA
ARC-e
ARC-c
SciQ
Race
MMLU
Avg. Acc.
WikiText word
LAMBADA PPL
None (Baseline)
0
74.5
63.7
63.0
60.5
36.2
88.3
37.8
37.0
57.6
12
5.7
FineWeb
0.2
71.9
56.7
46.0
56.7
32.4
88.6
36.4
30.8
52.4
18
13
PTB
0.2
70.3
54.9
47.3
55.2
31.9
87.8
34.5
26.7
51.1
21
13
WikiText
0.2
70.7
56.3
50.9
57.9
33.6
88.1
35.5
28.3
52.7
17
11
Alpaca
0.2
73.7
57.2
53.2
57.9
34.9
87.5
35.4
30.5
53.8
20
9.7
Appendix
Table 23: Ablation on calibration data for Llama3.2-1B at 20% compression (CR = 0.2). The first row shows the uncompressed baseline. Higher accuracy and lower perplexity are better.
Accuracy ↑
Perplexity ↓
Sparsification Strategy
CR
PIQA
HellaSwag
LAMBADA
ARC-e
ARC-c
SciQ
Race
MMLU
Avg. Acc.
WikiText word
LAMBADA PPL
None (Baseline)
0
74.5
63.7
63.0
60.5
36.2
88.3
37.8
37.0
57.6
12
5.7
ROCKET
0.2
71.9
56.7
46.0
56.7
32.4
88.6
36.4
30.8
52.4
18
13
Per-Row Sparsification
0.2
67.8
48.1
32.5
48.6
29.4
79.9
31.6
26.7
45.6
32
38
Global Importance Sparsification
0.2
69.5
54.7
42.7
53.5
30.7
85.7
35.5
28.1
50.1
19
17
Whitened-Space Importance Only
0.2
71.5
55.7
47.3
56.3
32.9
87.9
35.4
28.9
52.0
19
13
Appendix
Table 24: Ablation on sparsification strategies for Llama3.2-1B at 20% compression (CR = 0.2). All methods use the same calibration data and global parameter budget.
Error Metric
Avg. Acc. ↑
Perplexity ↓
Dense (Baseline)
57.6
12
L1 Distance
35.2
180
Mean Cos Columns
51.1
19
Spectral Distance
51.3
18
Frobenius (ours)
52.4
18
Appendix
Table 25: Ablation on reconstruction error metric for layer profiling in ROCKET for Llama3.2-1B at 20% compression.