OmniZip: Learning a Unified and Lightweight Lossless Compressor for Multi-Modal Data

Yan Zhao; Zhengxue Cheng; Junxuan Zhang; Dajiang Zhou; Qunshan Gu; Qi Wang; Li Song

cs.LGFeb 25, 2026

OmniZip: Learning a Unified and Lightweight Lossless Compressor for Multi-Modal Data

Authors: Yan Zhao, Zhengxue Cheng, Junxuan Zhang, Dajiang Zhou, Qunshan Gu, Qi Wang, Li Song

Organizations: Shanghai Jiao Tong University · Ant Group

Abstract

Lossless compression is essential for efficient data storage and transmission. Although learning-based lossless compressors achieve strong results, most of them are designed for a single modality, leading to redundant compressor deployments in multi-modal settings. Designing a unified multi-modal compressor is critical yet challenging, as different data types vary largely in format, dimension, and statistics. Multi-modal large language models offer a promising resolution but remain too complex for practical use. Thus, we propose OmniZip, a unified and lightweight lossless compressor for multi-modal data (like image, text, speech, tactile, database, and gene sequence). Built on a lightweight backbone, OmniZip incorporates three key components to enable efficient multi-modal lossless compression: a modality-unified tokenizer that reversibly transforms diverse data into tokens, a modality-routing context learning mechanism that enables flexible multi-modal context modeling, and a modality-routing feedforward design that further enhances the model's nonlinear representation flexibility. A reparameterization training strategy is used to enhance model capacity. OmniZip outperforms or matches other state-of-the-art compressors on multiple modalities, achieving 42%, 57%, 62% and 42%, 53% higher compression efficiency than gzip on CLIC-M, TouchandGo, enwik9, LibriSpeech, and WikiSQL datasets, respectively. It also supports near real-time inference on resource-constrained edge devices, reaching about 1MB/s on MacBook CPUs and iPhone NPUs. Our code is released at https://github.com/adminasmi/OmniZip-CVPR2026.

Figures & tables

Figure 1: Left : The world is multi-modal, motivating the need for unified lossless compression. Multi-modal compressors’ performance is shown across modalities, where points closer to the edge indicate better efficiency (lower bits/Byte). Right : Lightweight design ensures broad usability across platforms. OmniZip achieves near real-time inference (hundreds of KB/s to MB/s) even on edge devices.

Figure 2: Overview of the proposed OmniZip framework. Diverse data is first converted into a unified, fully reversible token space. A predictive model then estimates each token’s contextual probability, followed by arithmetic coding to generate the bitstream.

Model	#Params $\downarrow$	MACs $\downarrow$	bits/Byte $\downarrow$	Speed (KB/s) $\uparrow$
Transformer [ 65 ]	0.2M	0.88M	2.197	714
Transformer [ 65 ]	3.2M	4.43M	1.984	280
RWKV-7 [ 54 ]	0.2M	0.85M	1.910	2292
RWKV-7 [ 54 ]	3.2M	3.68M	1.658	856

Table 1: Comparison for model backbone selection. Models are evaluated using text lossless compression. Inference speed is measured on the MacBook Pro CPU, with a batch size of 128.

Figure 3: One block of OmniZip’s predictive model. The model stacks $N$ such blocks in total. Built on a lightweight RWKV7 backbone, it incorporates two modality-routing MoE modules for contextual learning and feedforward processing.

Model	Blocks	Embed Dim	#Params	MACs
OmniZip-S	2	320	4.8M	3.88M
OmniZip-M	2	912	38M	18.2M
OmniZip-L	3	1488	152M	56.0M

Table 2: Details of the proposed OmniZip models.

	Compressor	#Params $\downarrow$	MACs $\downarrow$	Speed (KB/s) $\uparrow$	Multi Modal	bits/Byte $\downarrow$
			MACs $\downarrow$	Speed (KB/s) $\uparrow$	Multi Modal	Kodak	CLIC-P	CLIC-M	DIV2K	TouchandGo	ObjectFolder	Axial	Coronal	Sagittal
Classical	gzip † [ 52 ]	-	-	14500 ′	✓	4.349 ${}_{\textbf{(0\%)}}$	4.039 ${}_{\textbf{(0\%)}}$	3.947 ${}_{\textbf{(0\%)}}$	4.224 ${}_{\textbf{(0\%)}}$	2.298 ${}_{\textbf{(0\%)}}$	3.969 ${}_{\textbf{(0\%)}}$	5.346 ${}_{\textbf{(0\%)}}$	4.563 ${}_{\textbf{(0\%)}}$	5.571 ${}_{\textbf{(0\%)}}$
	bzip2 † [ 59 ]	-	-	3300 ′	✓	4.359	4.033	3.931	4.208	2.288	4.031	5.403	4.617	5.628
	zstd † [ 48 ]	-	-	11000 ′	✓	4.350	4.042	3.952	4.229	2.263	3.966	5.343	4.560	5.568
	PNG † [ 11 ]	-	-	200 ′	✗	4.349	4.041	3.952	4.229	2.500	3.964	5.361	4.581	5.580
	FLIF † [ 62 ]	-	-	652 ′	✗	2.903	2.792	2.497	2.914	0.808	3.765	4.860	3.999	5.160

Table 3: Lossless compression results on image-like (natrual image, medical image, and tactile) datasets. Multi-modal compressors (except ours) are shown in gray.

*

denotes pretrained LLMs,

{\dagger}

indicates that some results are reprodeced by us.

\prime

indicates compression speeds measured on a MacBook CPU, and

{\ddagger}

denotes speeds on an A100 GPU (batch size 128). Other unmarked values are taken from their papers.

	Compressor	#Params $\downarrow$	MACs $\downarrow$	Speed (KB/s) $\uparrow$	Multi Modal	bits/Byte $\downarrow$
			MACs $\downarrow$	Speed (KB/s) $\uparrow$	Multi Modal	enwik9	Gutenberg	Spider	WikiSQL	GenoSeq	DNACorpus	LibriSpeech
Classical	gzip † [ 52 ]	-	-	14500 ′	✓	2.590 ${}_{\textbf{(0\%)}}$	3.819 ${}_{\textbf{(0\%)}}$	2.289 ${}_{\textbf{(0\%)}}$	1.672 ${}_{\textbf{(0\%)}}$	2.390 ${}_{\textbf{(0\%)}}$	2.318 ${}_{\textbf{(0\%)}}$	6.511 ${}_{\textbf{(0\%)}}$
	bzip2 † [ 59 ]	-	-	3300 ′	✓	2.082	3.607	2.326	1.403	2.202	2.148	5.647
	zstd † [ 48 ]	-	-	11000 ′	✓	2.364	3.703	2.258	1.520	2.404	2.324	6.467
	$\text{FLAC}^{{\dagger}}$ [ 18 ]	-	-	2000 ′	✗	-	-	-	-	-	-	4.961
Learning-based	$\text{Llama3}^{*{\dagger}}$ [ 22 ]	8B	7.80G	20 ‡	✓	0.722	1.030	0.770	0.645	1.889	1.808	3.616

Table 4: Lossless compression results on text-like (natural text, gene sequence, database) and speech datasets. Multi-modal compressors (except ours) are shown in gray.

*

denotes pretrained LLMs,

{\dagger}

indicates that some results are reprodeced by us.

\prime

indicates compression speeds measured on a MacBook CPU, and

{\ddagger}

denotes speeds on an A100 GPU (batch size 128). Other values are taken from their papers.

Figure 4: Comparison of learning-based lossless compressors across multi-modal datasets. The x-axis shows the model size (in millions of parameters), and the y-axis indicates compression efficiency (bits/Byte, lower is better). Models closer to the lower-left corner achieve better compression with fewer parameters. The dashed orange line represents the performance baseline of gzip.

Figure 5: OmniZip’s inference speed across various platforms (CPU of MacBook Pro, NPU of iPhone17 Pro, and GPU of NVIDIA A100) and batch sizes (1, 16, 128, 512, 1024).

Figure 6: Expert usage of OmniZip-S across blocks and routing modules. Each bar shows the expert usage percent (%) for a modality in a routing module (context learning or feedforward).

Unifed Tokenizer	Contextual Routing	Feedforward Routing	Reparam	Multi Modal	#Params $\downarrow$	MACs $\downarrow$	Speed (KB/s) $\uparrow$	bits/Byte $\downarrow$
Unifed Tokenizer	Contextual Routing	Feedforward Routing	Reparam	Multi Modal	#Params $\downarrow$	MACs $\downarrow$	Speed (KB/s) $\uparrow$	Kodak	TouchandGo	Coronal	enwik9	Spider	GenoSeq	LibriSpeech
✗	✗	✗	✗	✗	3.2M	3.68M	856	-	-	-	1.658	-	-	-
✗	✗	✗	✓	✗	3.2M	3.68M	856	-	-	-	1.590	-	-	-
✓	✗	✗	✓	✓	3.2M	3.68M	856	3.383	1.453	4.545	1.660	1.662	1.984	4.365
✓	✗	✓	✓	✓	4.0M	3.73M	633	3.352	1.431	4.314	1.424	1.448	1.915	4.295
✓	✓	✗	✓	✓	3.0M	3.82M	780	3.339	1.419	4.188	1.410	1.482	1.889	4.302

Table 5: Ablations on all our proposals: modality-unified tokenization, modality-routing contextual learning, modality-routing feedforward, and reparameterization training. Compression performance (bits/Byte) is evaluated on representative dataset from each modality, and the inference speed is measured on a MacBook CPU with a batch size of 128. The chosen configuration is colored in orange.

Contextual Routing			#Params $\downarrow$		MACs $\downarrow$	Speed (KB/s) $\uparrow$	bits/Byte $\downarrow$
Receptance	Key	Value	Total	Activated	MACs $\downarrow$	Speed (KB/s) $\uparrow$	Kodak	TouchandGo	Coronal	enwik9	Spider	GenoSeq	LibriSpeech
✓	✗	✗	4.8M	2.7M	3.88M	500	3.315	1.372	4.224	1.311	1.324	1.795	4.281
✗	✓	✗	4.8M	2.7M	3.88M	500	3.614	1.533	4.398	1.388	1.470	1.862	4.391
✗	✗	✓	4.8M	2.7M	3.88M	500	3.307	1.338	4.179	1.370	1.320	1.868	4.155
✓	✓	✗	5.4M	3.0M	4.02M	436	3.317	1.352	4.080	1.292	1.361	1.860	4.280
✓	✗	✓	5.4M	3.0M	4.02M	436	3.313	1.360	4.140	1.297	1.309	1.813	4.178

Table 6: Ablations on modality-routing contextual learning. Routing is applied to all blocks with 4 experts and top-

k

=2. We report the total parameters and parameters activated per token. Compression performance (bits/Byte) is evaluated on representative dataset from each modality, and the inference speed is measured on a MacBook CPU with a batch size of 128. The chosen configuration is colored in orange.

Blocks	top- $k$	Experts	Hidden Factor	#Params $\downarrow$		MACs $\downarrow$	Speed (KB/s) $\uparrow$	bits/Byte $\downarrow$
Blocks	top- $k$	Experts	Hidden Factor	Total	Activated	MACs $\downarrow$	Speed (KB/s) $\uparrow$	Kodak	TouchandGo	Coronal	enwik9	Spider	GenoSeq	LibriSpeech
1	1	4	$2\times$	3.6M	2.2M	3.65M	594	3.436	1.150	4.267	1.391	1.511	1.878	4.336
1	2	4	$2\times$	3.6M	2.5M	3.85M	570	3.358	1.420	4.230	1.380	1.344	1.896	4.307
2	2	4	$2\times$	4.8M	2.7M	3.88M	500	3.307	1.338	4.179	1.370	1.320	1.868	4.155
2	2	3	$2\times$	4.0M	2.7M	3.88M	500	3.355	1.424	4.154	1.369	1.410	1.882	4.300
2	2	5	$2\times$	5.2M	2.7M	3.88M	500	3.232	1.416	4.190	1.358	1.352	1.855	4.206

Table 7: Ablations on the configurations of the two modality-routing modules (e.g., routed blocks, expert count, top-

k

). We report the total parameters and parameters activated per token. Compression performance (bits/Byte) is evaluated on representative dataset from each modality, and the inference speed is measured on a MacBook CPU with a batch size of 128. The chosen configuration is colored in orange.

Figure 7: Key structural differences between RWKV-7 and OmniZip model. Left: an RWKV-7 block. Right: an OmniZip block.

Figure 8: Illustration of the reparameterization training strategy.

Dataset	Type	Train	Test	Description
Kodak [ 21 ]	image	✗	✓	24 film-scanned photos ( $768\times 512$ ) with balanced natural and synthetic scenes. We use it for testing.
CLIC-P [ 17 ]	image	✗	✓	41 high-resolution DSLR images (mostly 2K) with rich color and texture diversity. We use it for testing.
CLIC-M [ 17 ]	image	✗	✓	61 smartphone photos (mostly 2K) containing real-world noise and artifacts. We use it for testing.
DIV2K [ 2 ]	image	✓	✓	900 2K-resolution images with a large diversity of contents. We follow the official split (800 train, 100 test).
Axial [ 10 ]	medical	✓	✓	2D knee MRI slices ( $256\times 256$ ) in the axial plane. We follow the official split ( $\sim$ 38K train, $\sim$ 4.1K test).
Coronal [ 10 ]	medical	✓	✓	2D knee MRI slices ( $256\times 256$ ) in the coronal plane. We follow the official split ( $\sim$ 33K train, $\sim$ 3.5K test).

Table 8: Details of the utilized multi-modal datasets (16 datasets from 7 modalities).

Compressor	#Params $\downarrow$	Multi Modal	Adjusted bits/Byte $\downarrow$
	#Params $\downarrow$	Multi Modal	Kodak	CLIC-P	CLIC-M	DIV2K	TouchandGo	ObjectFolder	Axial	Coronal	Sagittal
$\text{Llama3}^{*{\dagger}}$ [ 22 ]	8B	✓	8490321	971320	553618	291465	322642	214197	693352	750368	970961
$\text{RWKV}^{*{\dagger}}$ [ 22 ]	7B	✓	7429031	849906	484417	255032	282312	187423	606683	656573	823341
DLPR ∗† [ 5 ]	22.3M	✗	23670	2710	1545	815	900	601	-	-	-
L3C ∗† [ 46 ]	5M	✗	5310	610	349	185	203	138	439	473	594
P2LLM [ 13 ]	8B	✗	8490319	971318	553616	291463	-	-	-	-	-

Table 9: Adjusted compression performance (bits/Byte) on image-like (natual image, medical image, and tactile) datasets.

*

denotes pretrained LLMs,

{\dagger}

indicates that some results are reprodeced by us. Other values are taken from their papers.

Compressor	#Params $\downarrow$	Multi Modal	Adjusted bits/Byte $\downarrow$
	#Params $\downarrow$	Multi Modal	enwik9	Gutenberg	Spider	WikiSQL	GenoSeq	DNACorpus	LibriSpeech
$\text{Llama3}^{*{\dagger}}$ [ 22 ]	8B	✓	129	362459	2125	10835	500	1138	212
$\text{RWKV}^{*{\dagger}}$ [ 22 ]	7B	✓	113	317152	1859	9480	437	996	186
tszip [ 9 ]	169	✗	4	7658	46	230	13	26	-
L3TC [ 71 ]	12	✗	2	546	6	18	3	4	-
OmniZip-S	4.8M	✓	1	219	3	8	2	3	4

Table 10: Adjusted compression performance (bits/Byte) on text-like (natural language, gene sequence, database) and speech datasets.

*

denotes pretrained LLMs,

{\dagger}

indicates that some results are reprodeced by us. Other values are taken from their papers.

Lossless Data Compression

See all 26 papers

Omni-C: Compressing Heterogeneous Modalities into a Single Dense Encoder
Feb 27, 2026Kin Wai Lau, Yasar Abbas Ur Rehman, Lai-Man Po +1Multimodal Learning Unified Multimodal Models
City University of Hong Kong · TCL AI Lab · University of Surrey, United Kingdom
OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models
Aug 4, 2026Wanshun Su, Yang Shi, Feihu Liu +10Efficient Multimodal Inference Audio Token Compression
Northwestern Polytechnical University · Peking University · Alibaba Group +1
OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models
Jul 25, 2026Jinsen Su, Yongdong Luo, Yuexiao Ma +3Efficient Multimodal Inference Audio Token Compression
Media Analytics and Computing Lab, Xiamen University, Xiamen, China · Institute of Artificial Intelligence, Xiamen University, Xiamen, China · School of Informatics, Xiamen University, Xiamen, China +1

Feb 27, 2026J/K move · Enter open · S save

Omni-C: Compressing Heterogeneous Modalities into a Single Dense Encoder

Kin Wai Lau, Yasar Abbas Ur Rehman, Lai-Man Po, Pedro Porto Buarque de Gusmão

City University of Hong Kong · TCL AI Lab · University of Surrey, United Kingdom

Multimodal Learning Unified Multimodal Models Multimodal Transformer Cross-Modal Representation Learning Efficient Multimodal Inference Multimodal Contrastive Learning

Open paper AskPDF arXiv

Recent multimodal systems often rely on separate expert modality encoders which cause linearly scaling complexity and computational overhead with added modalities. While unified Omni-models address this via Mixture-of-Expert (MoE) architectures with specialized experts and routing, they still inflate parameter counts and introduce routing overhead. In this paper, we propose Omni-C (Omni-Compress), a single dense Transformer-based encoder that learns competitive shared representations across heterogeneous modalities--images, audio, and text--through unimodal contrastive pretraining on large-scale unaligned data. By maximizing parameter sharing in the backbone and using lightweight modality-specific projection heads, Omni-C effectively mitigates inter-modality conflicts without requiring MoE, paired supervision, or routing. This design supports efficient deployment on memory-constrained systems via sequential modality processing and low-memory inference, eliminating the need for parallel expert loading or specialized hardware. Experiments show Omni-C achieves performance comparable to expert models in unimodal and cross-model tasks, with modest zero-shot degradation on audio and text that is largely recovered through lightweight linear probing or parameter efficient fine-tuning. The unified architecture substantially reduces inference memory usage compared to multi-encoder baselines, advancing efficient and scalable multimodal learning.

Abstract

Lossless Data Compression

Explore similar work

Omni-C: Compressing Heterogeneous Modalities into a Single Dense Encoder

OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models

OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models