Dynamics as Code: On Model Compression via Dynamic System

Fan Gao; Wei Su; Juntong Fan; Renfeng Peng; Hongyu Liu; Jinqiao Duan; Feng-Lei Fan

cs.LGOct 8, 2026

Dynamics as Code: On Model Compression via Dynamic System

Authors: Fan Gao, Wei Su, Juntong Fan, Renfeng Peng, Hongyu Liu, Jinqiao Duan, Feng-Lei Fan

Organizations: School of Information Science and Engineering, Lanzhou University, Lanzhou, Gansu Province, China · Department of Data Science, City University of Hong Kong, Hong Kong, China SAR · Department of Mathematics, City University of Hong Kong, Hong Kong, China SAR · School of Science, Great Bay University, Dongguan, China

Abstract

The escalating size of pretrained neural networks has rendered model compression a prerequisite for deployment under stringent memory and compute constraints. With the irrational winding as an example, earlier work introduced a dynamic system (DS) paradigm that reconceptualizes compression as compact weight representation: high-dimensional parameters are encoded by the index of a trajectory produced by a dynamic system, from which the vector is recovered during decompression. This mechanism is fundamentally distinct from pruning, quantization, knowledge distillation, and low-rank decomposition. Along this direction, we prove that under a Diophantine condition, a finite trajectory of $M = O(ε^{-(d+ν)})$ states in the irrational winding constitutes an $ε$ -net over the $d$ -dimensional weight space, thereby linking state resolution, decompression error, and compression ratio in a predictable manner. Furthermore, we propose a generalized DS-based model compression framework by unifying four DS families---space-filling curves (Hilbert, Peano, Morton/Z-order, Snake), chaotic systems (Lorenz), congruential and pseudo-random generators (LCG, PCG), and low-discrepancy sequences (Halton). Also, we introduce the KD-tree and coordinate-template acceleration to scale to large models as well as outlier identification to control the error. Experiments on ResNet-18 and Qwen2.5-1.5B/Qwen1.5-7B validate that DS-based compression achieves competitive compression ratios without post-hoc retraining, with controllable decompression error and flexible state-space design, establishing it as a principled and practical compression approach.

Figures & tables

Fig. 1: Representative geometries generated by eight evaluated mapping families. Hilbert , Peano , Morton/Z-order , and Snake produce structured lattice traversals with different locality characteristics and traversal continuity. Lorenz generates a non-uniform attractor-induced distribution concentrated on a low-dimensional manifold. LCG and PCG produce deterministic pseudo-random point sets with different correlation structures, while Halton generates low-discrepancy coverage with reduced large-scale gaps. All methods are visualized after normalization to the same local domain.

Fig. 2: An overview of the proposed DS-based compression framework. The parameter tensor is flattened and partitioned into grouped weight vectors, which are mapped into a tensor-wise local domain. A dynamic system generates a finite trajectory of states, and each grouped vector is assigned to its nearest state and the associated index. The model is compressed by storing the resulting index instead of the original floating-point vectors, while the states are recovered from the index for decompression.

Method	$D$	Scale setting
Space-filling curves and traversal mappings
Hilbert	$2$ – $3$	$S=2^{k\cdot D},\;k\in\{4,5,6,7,8,9,10\}$
	$4$ – $6$	$S=2^{k\cdot D},\;k\in\{3,4,5,6,7,8,9,10\}$
Snake	$2$ – $3$	$S=2^{k\cdot D},\;k\in\{4,5,6,7,8,9,10\}$
	$4$ – $6$	$S=2^{k\cdot D},\;k\in\{3,4,5,6,7,8,9,10\}$
Z-order	$2$ – $3$	$S=2^{k\cdot D},\;k\in\{4,5,6,7,8,9,10\}$

TABLE I: Evaluated state-generation settings. For grid-based mappings,

2^{k}

(or

3^{k}

for Peano) denotes the per-axis resolution, yielding

S=2^{kD}

(or

S=3^{kD}

) system states. For trajectory- and sequence-based mappings,

S

denotes the system states number.

Fig. 3: Compression-ratio landscapes of the eight evaluated methods on ResNet-18. Each panel corresponds to one method, where the horizontal axis denotes the system scale, the depth axis denotes the grouping dimension $D$ , and the vertical axis reports the measured compression ratio. For each fixed dimension, the curve and translucent curtain connect the evaluated scale configurations, while the numerical labels indicate the corresponding compression ratios. Higher values represent stronger compression. Lorenz is evaluated only in the two-dimensional setting because its trajectory states are projected onto a two-dimensional representation.

Fig. 4: Top-1 accuracy-drop landscapes of the eight evaluated methods on ResNet-18. The vertical axis reports the Top-1 accuracy drop relative to the original model in percentage points. Lower values indicate better accuracy preservation.

Method	D	S	CR	Top-1 Acc. ( $\Delta$ )%	Top-5 Acc. ( $\Delta$ )%
Hilbert	3	$2^{15}$	5.77 $\times$	69.17 (-0.59)	88.67 (-0.40)
Peano	5	$3^{20}$	$4.80\times$	69.62 (-0.14)	88.91 (-0.16)
Lorenz	2	$2^{18}$	$3.36\times$	69.64 (-0.12)	89.07 (0.00)
LCG	3	$2^{18}$	$4.96\times$	69.52 (-0.24)	88.99 (-0.08)
PCG	2	$2^{12}$	$4.81\times$	69.49 (-0.27)	88.86 (-0.21)
QMC	3	$2^{16}$	$5.50\times$	69.18 (-0.58)	88.72 (-0.35)

TABLE II: Representative configurations selected from the method-wise Pareto frontiers on ResNet-18. CR denotes the measured compression ratio, and

\Delta

denotes the accuracy change relative to the original model, which achieves

69.76\%

Top-1 and

89.07\%

Top-5 accuracy.

Model	Method	CR	PPL ( $\Delta$ )	Avg. Score ( $\Delta$ )%
Qwen2.5-1.5B	Original	–	9.26	67.31
	Hilbert	$\underline{\textbf{2.67}}\times$	9.61 (+0.35)	66.52 (-0.79)
	Peano	2.37 $\times$	9.55 (+0.29)	$\underline{\textbf{67.70 (+0.39) }}$
	Lorenz	1.68 $\times$	$\underline{\textbf{9.29 (+0.03)}}$	67.31 (+0.00)
	QMC	2.56 $\times$	9.75 (+0.49)	66.44 (-0.87)
	LCG	2.36 $\times$	9.62 (+0.36)	66.79 (-0.52)

TABLE III: Compression results on Qwen2.5-1.5B and Qwen1.5-7B. CR denotes the compression ratio. PPL is evaluated on WikiText-2, and Avg. Score is averaged over 16 downstream tasks. Values in parentheses denote changes relative to the corresponding uncompressed model.

Fig. 5: Compression–PPL trade-offs on Qwen2.5-1.5B and Qwen1.5-7B. Dashed lines denote the PPL of the corresponding uncompressed models. Higher compression ratios and lower perplexity indicate better trade-offs.

Fig. 6: Compression–perplexity trade-offs of weight-only compression methods on Qwen2.5-1.5B and Qwen1.5-7B. Higher CR and lower PPL indicate better trade-offs.

	Reference	Dyn-Sys	Previous	Conventional PTQ						Advanced Weight-only
Metric / Task	Original	Hilbert	IR	ZQ4	ZQ8	SQ4	SQ8	PTQ4	PTQ8	GPTQ	AWQ	SpQR	QuIP
CR	–	2.67 $\times$	2.67 $\times$	2.74 $\times$	1.73 $\times$	2.74 $\times$	1.73 $\times$	2.74 $\times$	1.73 $\times$	2.68 $\times$	2.69 $\times$	2.50 $\times$	2.75 $\times$
PPL	9.26	9.61	9.84	13.50	9.27	13.53	9.27	15.14	9.28	10.24	9.98	9.45	9.63
( $\Delta$ PPL)	–	+0.35	+0.58	+4.24	+0.01	+4.27	+0.01	+5.88	+0.02	+0.98	+0.72	+0.19	+0.37
SciQ	93.30	94.20	93.70	89.60	93.40	89.60	93.40	86.30	93.30	93.70	93.80	93.40	93.40
WinoGrande	63.38	62.19	64.40	58.17	63.93	59.75	63.69	59.27	63.38	62.51	63.85	63.54	64.25

TABLE IV: Compression and downstream-task comparison on Qwen2.5-1.5B. CR denotes the compression ratio, PPL is evaluated on WikiText-2. Task scores are reported in percentage (%). The best downstream task score is highlighted in bold. Avg. Score is averaged over the 16 downstream tasks. ZQ, SQ, and PTQ denote ZeroQuant, SQuant, and vanilla PTQ, respectively.

	Reference	Dyn-Sys	Previous	Conventional PTQ						Advanced Weight-only
Metric / Task	Original	Hilbert	IR	ZQ4	ZQ8	SQ4	SQ8	PTQ4	PTQ8	GPTQ	AWQ	SpQR	QuIP
CR	–	3.00 $\times$	2.91 $\times$	3.21 $\times$	1.85 $\times$	3.21 $\times$	1.85 $\times$	3.21 $\times$	1.85 $\times$	2.63 $\times$	2.64 $\times$	2.47 $\times$	2.69 $\times$
PPL	11.30	11.81	12.47	13.98	11.30	13.97	11.30	14.57	11.31	11.92	11.68	11.29	11.99
( $\Delta$ PPL)	–	+0.51	+1.17	+2.68	+0.00	+2.67	+0.00	+3.27	+0.01	+0.62	+0.38	-0.01	+0.69
SciQ	83.40	85.30	82.70	79.50	83.80	80.20	83.80	80.80	83.90	79.00	84.80	83.10	83.90
WinoGrande	65.19	64.17	62.75	61.72	64.96	62.12	65.19	61.33	65.27	65.19	65.19	65.75	66.06

TABLE V: Compression and downstream-task comparison on Qwen1.5-7B. Metrics, notation, and highlighting follow Table IV .

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Fig. S1: Compression–accuracy method-wise Pareto frontiers on ResNet-18/ImageNet. The uncompressed model achieves $69.76\%$ Top-1 accuracy. For clarity, only configurations with a Top-1 accuracy drop of at most $4.0$ percentage points are shown in panel (a). Panel (b) enlarges the high-fidelity region indicated by the dashed rectangle in panel (a).

Fig. S2: Compression–accuracy global Pareto analysis on ResNet-18/ImageNet. The uncompressed model achieves $69.76\%$ Top-1 accuracy. For clarity, configurations with a Top-1 accuracy drop greater than $4.0$ percentage points are omitted.

Explore similar work

Neural Network Compression by Approximate Differential Equivalence
May 31, 2026Ravi Dhiman, Andrea Passarella, Mirco Tribastone +1Neural Network Compression
IMT School for Advanced Studies Lucca 55100, Italy · IIT CNR, Pisa 56124, Italy
NeuralZip: Reusable Setup for Fast Lossless Compression
Oct 7, 2026Martín Bravo, Samuel Horváth, Gonzalo Navarro +1Neural Network Compression Entropy Coding
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Department of Computer Science, University of Chile, Santiago, Chile · National Center for Artificial Intelligence (CENIA), Santiago, Chile
Requential Coding: Pushing the Limits of Model Compression with Self-Generated Training Data
Jul 13, 2026Shikai Qiu, Marc Finzi, Yujia Zheng +2Model Compression Neural Network Generalization
New York University · Carnegie Mellon University

May 31, 2026, cs.LGJ/K move · Enter open · S save

Neural Network Compression by Approximate Differential Equivalence

Ravi Dhiman, Andrea Passarella, Mirco Tribastone, Lorenzo Valerio

IMT School for Advanced Studies Lucca 55100, Italy · IIT CNR, Pisa 56124, Italy

Neural Network Compression

Open paper AskPDF arXiv

Neural network compression is commonly achieved by pruning parameters based on local importance scores, e.g., magnitude-based pruning. We propose a complementary approach that compresses models by aggregating neurons with similar functional behavior rather than removing weights independently. Our method encodes a trained network as a polynomial ODE system and applies a lumping method called Approximate Forward Differential Equivalence to identify neurons with approximately matching induced dynamics. A single tolerance parameter,

\varepsilon

, controls the compression level and induces a smooth trade-off between model size and predictive accuracy. We evaluate the method on synthetic datasets derived from nonlinear dynamical systems with known ground-truth behavior and on public regression benchmarks. Across both settings, the proposed approach achieves substantial parameter reduction while preserving accuracy, and consistently compares favorably with magnitude-based pruning and Wanda at similar compression levels. These results suggest that differential equivalence-based aggregation is a principled and effective alternative to conventional weight-centric pruning.

Abstract

Explore similar work

Neural Network Compression by Approximate Differential Equivalence

NeuralZip: Reusable Setup for Fast Lossless Compression

Requential Coding: Pushing the Limits of Model Compression with Self-Generated Training Data