Organizations: School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China · Peng Cheng Laboratory, Shenzhen, China · Pazhou Laboratory (Huangpu), Guangzhou, China · School of Mechanical Engineering and Electronic Information, China University of Geosciences, Wuhan 430074, China · Peng Cheng Laboratory, Shenzhen 518055, China · State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China
Semantic communication shifts the focus from bit-level accuracy to task-relevant semantic delivery, enabling efficient and intelligent communication for next-generation networks. However, existing multi-modal solutions often process all available data modalities indiscriminately, ignoring that their contributions to downstream tasks are often unequal. This not only leads to severe resource inefficiency but also degrades task inference performance due to irrelevant or redundant information. To tackle this issue, we propose a novel task-aware distributed information bottleneck (TADIB) framework, which quantifies the contribution of any set of modalities to given tasks. Based on this theoretical framework, we design a practical coding scheme that intelligently selects and compresses only the most task-relevant modalities at the transmitter. To find the optimal selection and the codecs in the network, we adopt the probabilistic relaxation of discrete selection, enabling distributed encoders to make coordinated decisions with score function estimation and common randomness. Extensive experiments on public datasets demonstrate that our solution matches or surpasses the inference quality of full-modal baselines while significantly reducing communication and computational costs.
Figures & tables
Fig. 1: Performance improvement and computational overhead with respect to the number of modalities on MM-Fi. The more modalities involved (from left to right), the higher the computational cost. However, inference performance does not increase monotonically when redundant, difficult-to-learn, or irrelevant modalities (e.g., RGB images) are added.
Image recovering/classification + machine translation + video sentiment analysis
No
DL-based
CCMT [ 13 ]
Multi-view images
Multi-task image classification
No
DL-based + Infomax
TABLE I: Comparative Analysis of Representative Related Works in Semantic Communication
Symbol
Description
X,Y,Z
Generic source observation, task target variable, and semantic representation.
H(⋅∣⋅),I(⋅;⋅)
Conditional entropy and mutual information.
DKL(P∥Q)
Kullback-Leibler divergence from Q to P .
K,K,k
Set, number, and index of distributed transmitters (encoders).
M,M,m
Modality set, number of local modalities, and modality index.
T,T,t
Task/receiver set, number, and index.
TABLE II: Main Notations
Fig. 2: System model of a multi-modal task-aware semantic communication.
Fig. 3: Task-aware modality selection in general distributed networks. Modalities (tasks) are distributed at different transmitters (receivers).
(a) AV-MNIST
Method
Under limits
R
Rate
N-CE
Acc 1
Acc 2
Acc 3
p TADIB (CR)
Yes
0.56 ± 0.05
166.78 ± 0.96
-0.39 ± 0.05
98.60 ± 1.89
99.39 ± 2.17
96.50 ± 1.59
p TADIB (w/o CR)
Yes
0.91 ± 0.05
185.42 ± 1.54
-0.72 ± 0.05
97.50 ± 1.61
93.50 ± 3.64
87.75 ± 4.52
RS-DIB
Yes
0.94 ± 0.03
141.76 ± 1.08
-0.70 ± 0.03
87.20 ± 2.49
86.20 ± 3.95
92.60 ± 1.70
Greedy-DIB
Yes
0.66 ± 0.03
186.05 ± 1.23
-0.47 ± 0.03
98.25 ± 1.39
97.81 ± 2.45
96.91 ± 1.50
MI-DIB-C
Yes
0.77 ± 0.03
219.11 ± 1.02
-0.55 ± 0.03
89.99 ± 2.68
95.19 ± 3.17
95.68 ± 2.50
TABLE III: Performance comparison on (a) AV-MNIST and (b) MM-Fi (95% confidence intervals with 10 independent runs). “Under limits” indicates satisfaction of both the transmitter-side and receiver-side access constraints. Lower values are better for R , sum-rate, MPJPE, and PA-MPJPE, whereas higher values are better for N-CE and accuracy. Within the access-feasible group, the best value in each metric is shown in bold, all ties are bolded.
method
(a) AV-MNIST
(b) MM-Fi
time
pairs
cost
time
pairs
cost
p TADIB
1.64
8.00
1.81 × 10 7
5.12
5.00
6.20 × 10 7
∼ w/o CR
2.20
7.10
1.66 × 10 7
5.10
4.86
4.74 × 10 7
MI-DIB
11.69
18.00
3.47 × 10 7
22.15
24.00
1.24 × 10 8
VDDIB
17.52
27.00
4.95 × 10 7
24.87
48.00
2.40 × 10 8
TABLE IV: Physical training time, the number of active pairs, and computational cost on (a) AV-MNIST and (b) MM-Fi datasets. Training time is measured in seconds per epoch (s/epoch). The number of active modality-task matching pairs is averaged over 10 independent runs. The average computational cost is measured by FLOPs. The lowest value in each dataset-metric column is shown in bold.
K
candidate pairs
parameters
time
peak memory
cost
3
27
16.30
1.53
318.94
1.87 ×107
6
54
27.03
1.28
436.55
2.26 ×107
9
81
37.76
1.48
600.99
2.46 ×107
15
135
59.21
1.32
927.93
2.87 ×107
TABLE V: Controlled network-size scaling of p TADIB on AV-MNIST: Physical training time is measured in seconds per epoch (s/epoch), number of parameters is measured in millions (M), peak GPU memory is measured in MiB, and computational cost is measured in FLOPs.
Fig. 4: Rate-relevance tradeoffs on (a) AV-MNIST and (b) MM-Fi. Dashed curves marked by † emphasize unconstrained baselines.
Fig. 5: Training dynamics on information plane (sum-rate vs N-CE).
metric/ratio
1/16
1/8
1/4
1/2
1/1
2/1
4/1
(a)
sum-rate
174.43
187.97
183.25
165.08
140.42
142.47
143.13
N-CE
-0.72
-0.67
-0.61
-0.45
-0.92
-0.96
-0.95
(b)
sum-rate
93.92
72.71
47.60
49.31
52.50
53.12
76.75
N-CE
-0.40
-0.26
-1.81
-1.88
-1.98
-1.87
-0.31
TABLE VI: Overall performance of p TADIB varying with learning rate ratio r from 1/16 to 4/1 on (a) AV-MNIST and (b) MM-Fi datasets.
Fig. 6: Convergence of log-likelihood ( 19 ) (log-likelihood vs iteration round).
Fig. 7: Comparison of converged A with and without CR
Fig. 8: Convergence of randomized selection A on different datasets
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Fig. 9: Converged A without access limits at β =1e-3. For AV-MNIST, rate: 172.27, N-CE: -0.15. For MM-Fi, rate: 78.17, N-CE: -0.36.
Fig. 10: Converged A without access limits at β→ 0. For AV-MNIST, rate: 1690.59, N-CE: -0.08. For MM-Fi, rate: 4592.59, N-CE: -0.14.
Fig. 11: Capacity dependence of empirical TADIB score fidelity on AV-MNIST. (a) Spearman ordering correlation of candidate scores. Higher correlation indicates closer score ordering to the Base proxy. (b) Simple regret evaluated under the Base proxy.
Fig. 12: Channel robustness on AV-MNIST. † emphasizes the unconstrained VDDIB reference.
Fig. 13: Impact of β on the empirical TADIB matching score Rβ on AV-MNIST. Each group compares the four access-feasible methods.
The growing demand for efficient semantic communication systems capable of managing diverse tasks and adapting to fluctuating channel conditions has driven the development of robust, resource-efficient frameworks. This article introduces a novel channel-adaptive and multi-task-aware semantic communication framework based on a masked auto-encoder architecture. Our framework optimizes the transmission of meaningful information by incorporating a multi-task-aware scoring mechanism that identifies and prioritizes semantically significant data across multiple concurrent tasks. A channel-aware extractor is employed to dynamically select relevant information in response to real-time channel conditions. By jointly optimizing semantic relevance and transmission efficiency, the framework ensures minimal performance degradation under resource constraints. Experimental results demonstrate the superior performance of our framework compared to conventional methods in tasks such as image reconstruction and object detection. These results underscore the framework's adaptability to heterogeneous channel environments and its scalability for multi-task applications, positioning it as a promising solution for next-generation semantic communication networks.
Xiang Chen, Shuying Gan, Chenyuan Feng +2
School of Electronics and Information Technology, Sun Yat-sen University, China · EURECOM, Sophia Antipolis, France · Singapore University of Technology and Design, Singapore
Semantic communication is a novel communication paradigm that focuses on conveying the user's intended meaning rather than the bit-wise transmission of source signals. One of the key challenges is to effectively represent and extract the semantic meaning of any given source signals. While deep learning (DL)-based solutions have shown promising results in extracting implicit semantic information from a wide range of sources, existing work often overlooks the high computational complexity inherent in both model training and inference for the DL-based encoder and decoder. To bridge this gap, this paper proposes a rate-distortion-complexity (RDC) framework which extends the classical rate-distortion theory by incorporating the constraints on semantic distance, including both the traditional bit-wise distortion metric and statistical difference-based divergence metric, and complexity measure, adopted from the theory of minimum description length and information bottleneck. We derive the closed-form theoretical results of the minimum achievable rate under given constraints on semantic distance and complexity for both Gaussian and binary semantic sources. Our theoretical results show a fundamental three-way tradeoff among achievable rate, semantic distance, and model complexity. Extensive experiments on real-world image and video datasets validate this tradeoff and further demonstrate that our information-theoretic complexity measure effectively correlates with practical computational costs, guiding efficient system design in resource-constrained scenarios.
Jingxuan Chai, Yong Xiao, Guangming Shi
the School of Artificial Intelligence at the Xidian University, Shaanxi 710071, China · the School of Electronic Information and Communications at the Huazhong University of Science and Technology, Wuhan 430074, China · the Pengcheng Laboratory, Shenzhen, Guangdong 518055, China +2
Shannon's information theory deliberately excludes message semantics. This paper develops a rigorous framework for semantic communication that integrates formal proof systems with Shannon-theoretic tools. We introduce an axiomatic information model comprising Lsem-definable state sets linked by computable enabling maps, and define the semantic channel as a composition of Markov kernels whose supports respect the enabling structure. A fixed proof system induces an irredundant semantic core and a derivation-depth stratification, enabling four distortion measures of increasing semantic depth: Hamming, closure, depth, and a parameterized composite. Six families of computable semantic channel invariants are defined and their inter-relationships established, including a data processing bound, a semantic Fano bound, and an ideal-channel collapse theorem. The central quantitative result is a deductive compression gain: under closure-based fidelity, the minimum block length is determined by the irredundant core size rather than the full knowledge-base size. We instantiate the framework for heterogeneous multi-agent communication, introducing an overlap decomposition that yields necessary and sufficient conditions for closure-reliable communication. A semantic bottleneck phenomenon is identified in broadcast settings: vocabulary mismatch imposes irreducible fidelity limitations even over noiseless carriers. All results are verified on an explicit Datalog instance.
Jianfeng Xu
Koguan School of Law, China Institute for Smart Justice, School of Computer Science, Shanghai Jiao Tong University, Shanghai 200030, China