This paper introduces a practical framework for semantic communication over wireless channels, focusing on the compression of high-dimensional embeddings under quantization and channel errors. Building upon a relaxed Gaussian Information Bottleneck (GIB) model, the approach provides analytical solutions for task-relevant information preservation, even with non-Gaussian targets. The framework relaxes traditional Gaussian assumptions, enabling effective compression for discrete and complex data such as text embeddings. It derives closed-form gradients for optimizing the compression matrix and establishes bounds linking continuous representations to discrete, noisy system performance. Comprehensive experiments on text classification tasks demonstrate that the proposed method outperforms task-agnostic quantization-based compression methods, maintaining high semantic fidelity at low bit budgets per embedding. The approach offers a scalable, interpretable, and resource-efficient solution for deploying semantic communication in general-purpose networks.
Figures & tables
Fig. 1: Schematic of the studied semantic communication system.
Parameter
Symbol
Value
Batch size
B
64
Monte Carlo samples per instance
K
120
Learning rate
η
0.001
Beta coefficient
β
1.0
Noise variance
ϵZ
0.01
Training epochs
nepochs
500
TABLE I: Compression matrix training parameters.
m
L
Acc(t)
Acc(t^)
E[∥ti−t^i∥]
L^emp
32
2
14.4
2.0
5.6365
2.88
4
15.0
4.3
5.8089
3.31
32
43.6
30.1
5.1920
4.01
64
76.5
67.5
4.5663
3.34
128
83.3
83.1
4.1703
2.88
256
83.7
87.0
3.8219
2.91
TABLE II: Empirical verification of the Lipschitz continuity assumption on Banking77. m : compressed dimension; L : codebook size; L^emp : empirical Lipschitz constant. All accuracy values in %, averaged over 100 MTEB experiments with standard MTEB training.
Fig. 2: Accuracy performances against bit budget per embedding over 10k MTEB experiments for Banking77. The rGIB curves represent the relaxed GIB-based compression performances, while PQ curve indicates performances of the compression method implemented in [ 18 ] .
K=120
K=220
K=1024
m
L
Acc(t)
Acc(t^)
Acc(t)
Acc(t^)
Acc(t)
Acc(t^)
32
1024
84.9
87.0
85.2
87.5
86.3
87.6
2048
85.0
85.9
86.1
86.8
86.6
87.3
4096
83.7
84.4
84.2
84.9
83.2
84.5
128
1024
74.8
77.0
78.0
80.1
82.5
84.1
2048
73.8
75.9
76.7
79.0
82.3
83.6
TABLE III: Effect of increasing the number of Monte Carlo samples K during compression matrix A training, for L∈{1024,2048,4096} and m∈{32,128} , extending the evaluation beyond the practical operating range shown in Fig. 2 . All accuracy values in %, averaged over 100 MTEB experiments with standard MTEB training.
TABLE IV: Classifier evaluated on quantized test vs. quantized training embeddings with standard MTEB training (616 samples, at left) and full data training (all 9993 samples, at right) for fixed anchor matrix AL . Acc(t) : unquantized baseline accuracy on fixed anchor A . All accuracy values in %, averaged over 100 MTEB experiments.
Fig. 3: Cumulative explained variance across principal components for the three dimensional spaces evaluated.
Recent high-performing text embedding models often produce high-dimensional real-valued vectors, resulting in substantial storage and computational costs. To address this issue, embedding compression methods based on dimensionality reduction or quantization have been proposed; however, the effects of their combination have not been sufficiently investigated. In this paper, we systematically investigate how dimensionality reduction and quantization jointly affect embedding compression across four task families in the Massive Text Embedding Benchmark (MTEB), using five pretrained embedding models. Our results show that combining dimensionality reduction and quantization can preserve near-original performance with substantially smaller storage footprints than either method alone. In particular, for classification, several models retain at least 99% of their original classification score using less than 0.1% of the storage required by the original embeddings. The preferred balance between dimensionality and bit-width varies across tasks, with retrieval generally being more sensitive to compression.
Building on recent advances in representation learning for wireless channels, this work investigates the cost-benefit trade-offs of high-dimensional channel embeddings in practical systems. We benchmark multiple wireless representations: high-dimensional learned embeddings from a wireless foundation model, compact autoencoder-based representations with significantly lower dimensionality, and raw data baselines, evaluating their performance across diverse downstream tasks. We then systematically analyze data efficiency, noise robustness, and computational complexity, explicitly characterizing the resource overhead associated with high-dimensional embeddings. Beyond standard tasks such as line-of-sight/non-line-of-sight (LoS/NLoS) classification and beam selection, we introduce power allocation as a new downstream task. Our results reveal clear trade-offs: while high-dimensional embeddings can perform well in few-shot regimes for certain tasks, they incur substantial latency and parameter overhead. In contrast, compressed latent representations learned by autoencoders demonstrate improved noise robustness and more stable performance across tasks, while significantly reducing computational and transmission costs.
Murilo Batista, Shirin Salehi, Saeed Mashdour +3
Chair of Information Theory and Data Analytics (INDA), RWTH Aachen University, Aachen, Germany · Centre for Telecommunications Studies, Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro 22541-041, Brazil
Our work explores learning a compressed latent representation of text, at the intersection of data compression and representation learning. We propose an autoencoder architecture that performs residual downscaling and upscaling of hidden representations along the time axis, with a residual low-dimension discrete bottleneck. We analyze our approach for different quantization methods, training objectives, and datasets. For different levels of compression, we evaluate the similarity between the original and reconstructed text both at the surface-level (BLEU) and at the semantic-level (LLM-based judge). Additionally, we evaluate our models on downstream question-answering and semantic text similarity benchmarks. Our approach results in compressed representations which are on par with lossless text compression algorithms at 2.24 bits per byte on web text data, while having good reconstruction and downstream task performance.
Vinko Sabolčec, Angelos Katharopoulos, David Grangier