This paper introduces a practical framework for semantic communication over wireless channels, focusing on the compression of high-dimensional embeddings under quantization and channel errors. Building upon a relaxed Gaussian Information Bottleneck (GIB) model, the approach provides analytical solutions for task-relevant information preservation, even with non-Gaussian targets. The framework relaxes traditional Gaussian assumptions, enabling effective compression for discrete and complex data such as text embeddings. It derives closed-form gradients for optimizing the compression matrix and establishes bounds linking continuous representations to discrete, noisy system performance. Comprehensive experiments on text classification tasks demonstrate that the proposed method outperforms task-agnostic quantization-based compression methods, maintaining high semantic fidelity at low bit budgets per embedding. The approach offers a scalable, interpretable, and resource-efficient solution for deploying semantic communication in general-purpose networks.
Figures & tables
Fig. 1: Schematic of the studied semantic communication system.
Parameter
Symbol
Value
Batch size
B
64
Monte Carlo samples per instance
K
120
Learning rate
η
0.001
Beta coefficient
β
1.0
Noise variance
ϵZ
0.01
Training epochs
nepochs
500
TABLE I: Compression matrix training parameters.
m
L
Acc(t)
Acc(t^)
E[∥ti−t^i∥]
L^emp
32
2
14.4
2.0
5.6365
2.88
4
15.0
4.3
5.8089
3.31
32
43.6
30.1
5.1920
4.01
64
76.5
67.5
4.5663
3.34
128
83.3
83.1
4.1703
2.88
256
83.7
87.0
3.8219
2.91
TABLE II: Empirical verification of the Lipschitz continuity assumption on Banking77. m : compressed dimension; L : codebook size; L^emp : empirical Lipschitz constant. All accuracy values in %, averaged over 100 MTEB experiments with standard MTEB training.
Fig. 2: Accuracy performances against bit budget per embedding over 10k MTEB experiments for Banking77. The rGIB curves represent the relaxed GIB-based compression performances, while PQ curve indicates performances of the compression method implemented in [ 18 ] .
K=120
K=220
K=1024
m
L
Acc(t)
Acc(t^)
Acc(t)
Acc(t^)
Acc(t)
Acc(t^)
32
1024
84.9
87.0
85.2
87.5
86.3
87.6
2048
85.0
85.9
86.1
86.8
86.6
87.3
4096
83.7
84.4
84.2
84.9
83.2
84.5
128
1024
74.8
77.0
78.0
80.1
82.5
84.1
2048
73.8
75.9
76.7
79.0
82.3
83.6
TABLE III: Effect of increasing the number of Monte Carlo samples K during compression matrix A training, for L∈{1024,2048,4096} and m∈{32,128} , extending the evaluation beyond the practical operating range shown in Fig. 2 . All accuracy values in %, averaged over 100 MTEB experiments with standard MTEB training.
TABLE IV: Classifier evaluated on quantized test vs. quantized training embeddings with standard MTEB training (616 samples, at left) and full data training (all 9993 samples, at right) for fixed anchor matrix AL . Acc(t) : unquantized baseline accuracy on fixed anchor A . All accuracy values in %, averaged over 100 MTEB experiments.
Fig. 3: Cumulative explained variance across principal components for the three dimensional spaces evaluated.
Chair of Information Theory and Data Analytics (INDA), RWTH Aachen University, Aachen, Germany · Centre for Telecommunications Studies, Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro 22541-041, Brazil