Doubly Adaptive Channel and Spatial Attention for Semantic Image Communication by IoT Devices
Authors: Soroosh Miri, Sepehr Abolhasani, Shahrokh Farahmand, S. Mohammad Razavizadeh, Jiguang He
Organizations: School of Electrical Engineering, Iran University of Science and Technology (IUST), Tehran, Iran · Great Bay University, Dongguan 523000, China
Internet of Things (IoT) networks face significant challenges such as limited communication bandwidth, constrained computational and energy resources, and highly dynamic wireless channel conditions. Utilization of deep neural networks (DNNs) combined with semantic communication has emerged as a promising paradigm to address these limitations. Deep joint source-channel coding (DJSCC) has recently been proposed to enable semantic communication of images. Building upon the original DJSCC formulation, low-complexity attention-style architectures has been added to the DNNs for further performance enhancement. As a main hurdle, training these DNNs separately for various signal-to-noise ratios (SNRs) will amount to excessive storage or communication overhead, which can not be maintained by small IoT devices. SNR Adaptive DJSCC (ADJSCC), has been proposed to train the DNNs once but feed the current SNR as part of the data to the channel-wise attention mechanism. We improve upon ADJSCC by a simultaneous utilization of doubly adaptive channel-wise and spatial attention modules at both transmitter and receiver. These modules dynamically adjust to varying channel conditions and spatial feature importance, enabling robust and efficient feature extraction and semantic information recovery. Simulation results corroborate that our proposed doubly adaptive DJSCC (DA-DJSCC) significantly improves upon ADJSCC in several performance criteria, while incurring a mild increase in complexity. These facts render DA-DJSCC a desirable choice for semantic communication in performance demanding but low-complexity IoT networks.
Figures & tables
Fig. 1 : System model of the proposed DA-DJSCC approach.
Fig. 2 : Overall architecture of the proposed DA-DJSCC model.
Fig. 3 : Channel-wise attention module.
Fig. 4 : Spatial attention module.
Fig. 5 : PSNR performance for k/n=1/12 .
Fig. 6 : SSIM performance for k/n=1/12 .
Fig. 7 : Classification accuracy for k/n=1/12 .
Model
Complexity Metrics
Parameters
Memory (MB)
Latency (ms)
DJSCC
0.143M
0.547
0.5
ADJSCC
0.157M
0.601
2
Proposed DA-DJSCC
0.159M
0.607
3
a All values are measured under the same hardware and batch size conditions.
TABLE I : Comparison of model complexity for proposed model and baseline DJSCC models.
Deep learning based semantic communication has achieved significant progress in wireless image transmission, but most existing schemes rely on fixed models and thus lack robustness to diverse image contents and dynamic channel conditions. To improve adaptability, recent studies have developed adaptive semantic communication strategies that adjust transmission or model behavior according to either source content or channel state. More recently, MoE-based semantic communication has emerged as a sparse and efficient adaptive architecture, although existing designs still mainly rely on single-driven routing. To address this limitation, we propose a novel multi-stage end-to-end image semantic communication system for multi-input multi-output (MIMO) channels, built upon an adaptive MoE Swin Transformer block. Specifically, we introduce a dynamic expert gating mechanism that jointly evaluates both real-time CSI and the semantic content of input image patches to compute adaptive routing probabilities. By selectively activating only a specialized subset of experts based on this joint condition, our approach breaks the rigid coupling of traditional adaptive methods and overcomes the bottlenecks of single-driven routing. Simulation results indicate a significant improvement in reconstruction quality over existing methods while maintaining the transmission efficiency.
Haowen Wan, Qianqian Yang
Polytechnic Institute, Zhejiang University, Hangzhou, China · College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou, China
Unmanned Aerial Vehicles (UAVs) have emerged as a key enabler technology for data collection from Internet of Things (IoT) devices. However, effective data collection is challenged by resource constraints and the need for real-time decision-making. In this work, we propose a novel framework that integrates semantic communication with UAV command-and-control (C&C) to enable efficient image data collection from IoT devices. Each device uses Deep Joint Source-Channel Coding (DeepJSCC) to generate a compact semantic latent representation of its image to enable image reconstruction even under partial transmission. A base station (BS) controls the UAV's trajectory by transmitting acceleration commands. The objective is to maximize the average quality of reconstructed images by maintaining proximity to each device for a sufficient duration within a fixed time horizon. To address the challenging trade-off and account for delayed C&C signals, we model the problem as a Markov Decision Process and propose a Double Deep Q-Learning (DDQN)-based adaptive flight policy. Simulation results show that our approach outperforms baseline methods such as greedy and traveling salesman algorithms, in both device coverage and semantic reconstruction quality.
Assane Sankara, Daniel Bonilla Licea, Hajar El Hammouti
College of Computing, Mohammed VI Polytechnic University (UM6P), Ben Guerir, Morocco
Semantic image communication seeks to preserve task-relevant scene structure under limited channel resources, but carriers are often dense latent tensors or grid-aligned semantic layouts that do not explicitly encode region-level relations. This work introduces a segmentation-derived region adjacency graph (RAG) carrier, termed channel-adaptive RAG (CA-RAG), for joint source-channel coding-style image communication. Nodes store interpretable region attributes, edges preserve adjacency, channel-adaptive graph simplification (CGS) controls the node budget, and semantic belief propagation refines noisy graph embeddings before diffusion-based reconstruction. On Cityscapes, pre-channel RAG payloads are several times smaller than compressed class-index layouts in a 2,000-image study. Under additive white Gaussian noise at signal-to-noise ratios from 0 to 15 dB, CA-RAG reports higher semantic consistency than deep joint source-channel coding and a same-decoder layout baseline, with comparable perceptual quality. At 10 dB, the full-budget rate-sweep point reaches mean intersection over union (mIoU) = 0.329 at approximately 3.3 x 10^3 channel uses, while the default adaptive-CGS setting reports mIoU = 0.294 at approximately 2.6 x 10^3 channel uses.
Karim Abdallah, Maria Slim, Mariette Awad +1
Department of Electrical and Computer Engineering American University of Beirut, Beirut, Lebanon