Task-Oriented Semantic Communication

Ask about this topic

Momentum

3 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 12

All topics
CardsList
Oct 1, 2026eess.SP

Token Communication-Assisted Collaborative Embodied Artificial Intelligence: Concepts, Framework, and Opportunities

Collaborative embodied artificial intelligence (CEAI) enables multiple physical agents to perceive, reason, and act cooperatively in dynamic environments. Effective communication is essential for CEAI, yet CEAI agents must exchange not only large multimodal observations but also task-relevant insights, intents, and interactive information over long horizons. This article investigates token communication (TokCom) as a native intelligence interface for CEAI, in which tokens serve jointly as compact semantic carriers for communication and fundamental inference units for generative foundation models (GFMs). We first discuss how TokCom supports insight sharing, intent alignment, and interactive control among embodied agents. We then propose a TokCom-assisted CEAI framework driven by a task-adaptive communication protocol. Comprising a compact codebook, syntax rules, and contextual examples, this protocol guides GFM-based transceivers to distill messages into compact tokens and reconstruct them after wireless transmission. A case study on collaborative object transport demonstrates that the proposed TokCom framework substantially reduces the source payload bit consumption while preserving task efficiency and showing robustness under noisy channels. Finally, we outline future research directions.

Peng Yi, Ying-Chang Liang

National Key Laboratory of Wireless Communications, and also with the Center for Intelligent Networking and Communications (CINC), University of Electronic Science and Technology of China (UESTC), Chengdu 611731, China · Institute of Fundamental and Frontier Sciences and the Center for Intelligent Networking and Communications (CINC), University of Electronic Science and Technology of China (UESTC), Chengdu 611731, China

Agent Communication ProtocolsTask-Oriented Semantic CommunicationEmbodied AIMulti-Agent Communication
AskPDF
Sep 28, 2026cs.MA

Embodied Semantic Communication for Collective Autonomous Agents: A Tutorial on Representation, Wireless Delivery, and Closed-Loop Coordination

As autonomous systems and embodied intelligence enter the dynamic physical world, multi-agent collaboration calls for a paradigm shift in communication design. However, existing communication paradigms overlook that agents form action understanding from their own states, environmental observations, and collaboration relations through a process that evolves as a task unfolds. Consequently, reliable bit delivery, general semantic recovery, or single-task utility optimization alone cannot ensure that heterogeneous agents form coordinated actions compatible with their own conditions from shared information during task execution. To address this gap, this paper proposes embodied semantic communication (ESC) as a paradigm that transforms information transmission into action-oriented semantic interaction. Specifically, ESC characterizes how an explicit communication link can encapsulate multimodal perceptual states, intrinsic hardware capabilities, and collaborative intents into unified actionable semantic representations, thereby enabling heterogeneous receiving agents to parse, align, and ground them in local motor control. This paper clarifies the conceptual boundary, system characteristics, and environment-constrained technical pathways of ESC. It maps the underlying mathematical tools, including semantic information theory, world models, and multi-agent decision theory. Finally, this paper summarizes key open challenges, including measurable semantic reliability, ambiguity-triggered interaction under dynamic environments and tasks, and bandwidth-adaptive semantic transmission, outlining a roadmap for collective embodied networks.

Yizheng Huang, Wensheng Lin, Lixin Li +3

School of Electronics and Information, Northwestern Polytechnical University, Xi’an, Shaanxi 710129, China · School of Information and Communications Engineering, Xi’an Jiaotong University, Xi’an 710049, China · School of Telecommunications Engineering, Xidian University, Xi’an 710071, China +1

Multi-Agent CoordinationSemantic CommunicationTask-Oriented Semantic CommunicationMulti-Agent CollaborationMulti-Agent Communication
AskPDF
Sep 17, 2026cs.CV

Task-Oriented Semantic Feature Transmission for Multi-Task Satellite Remote Sensing over Low-SNR Channels

Conventional satellite remote sensing transmission follows a reconstruct-then-infer paradigm that optimizes pixel-level fidelity, creating an objective mismatch with downstream tasks such as classification and detection, especially at low SNR. This paper investigates a task-oriented framework that bypasses image reconstruction and directly transmits semantic features extracted by a multitask-pretrained backbone. A lightweight channel adaptation module (CAM) compresses feature dimensionality for bandwidth reduction, and a feature restorer recovers task-relevant structure after channel corruption. With the backbone frozen, the CAM and task-specific downstream heads are jointly optimized with task and feature-level supervision under random-SNR training. Under the adopted AWGN setting, experiments on scene classification and object detection show consistent gains over reconstruction-oriented JSCC baselines across different SNR conditions, with the largest improvements in the low-SNR regime.

Shuoyuan Sun, Hongyu Wang, Mugen Peng +1

State Key Laboratory of Networking and Switching Technology Beijing University of Posts and Telecommunications

Remote Sensing Image UnderstandingSemantic CommunicationTask-Oriented Semantic CommunicationRemote Sensing
AskPDF
Jul 30, 2026cs.RO

When Robots Exchange Meaning: A Demo of Goal-Oriented Semantic Communications for Collaborative Robotics

Collaborative robotics is a representative task-oriented 6G use-case, where communication quality should be reflected in mission execution, environment understanding, and closed-loop operation rather than packet delivery alone. This demo paper presents a robot-edge semantic communication (SemCom) testbed integrating robot-side visual compression, edge-side semantic mapping, and dashboard-based mission interaction. A mobile robot equipped with RGB-D sensing and LiDAR runs ROS 2, while a Jetson Orin edge node performs reconstruction, RTAB-Map mapping, semantic object handling, and browserbased visualization. As an initial proof of concept, RGB frames are encoded on the robot into VQ-VAE tokens using an ONNX Runtime encoder and reconstructed on the edge using a PyTorch decoder. A 320 X 240 image is represented by an 80 X 60 token grid with a packed payload of 5400 bytes, corresponding to a 42.67X reduction relative to model-input RGB bytes. The reconstructed visual stream is further associated with depth, pose, and 3D mapping information to generate a semantic map for downstream robotic applications. The demo exposes the full path from semantic visual transport to object-level map interaction, and provides a practical platform for future task-aware 6G networking studies at the intersection of SemCom, embodied AI, and physical AI-enabled robotics. A video of the demo is available at https://tinyurl.com/Tos09

Peizheng Li, Xinyi Lin, Sajida Gufran +1

Bristol Research and Innovation Laboratory, Toshiba Europe Ltd., U.K.

Semantic CommunicationTask-Oriented Semantic Communication
AskPDF
Jul 23, 2026cs.LG

Semantic-Aware Task Clustering for Constructive and Cooperative Multi-Tasking

Cooperative multi-task semantic communication (CMT-SemCom) improves task execution performance by leveraging shared representations. However, as we demonstrated in [1], cooperative multi-tasking can be either constructive or destructive, depending on the semantic relationships among tasks. To ensure constructive cooperation, we propose a semantic-aware task clustering method for CMT-SemCom. We have formulated a sequential multi-stage optimization problem in which semantically aligned tasks are clustered once after a short initial training phase, and then end-to-end (E2E) joint training is conducted exclusively within the discovered groups. Specifically, the problem decomposes into two stages: (i) a semantic clustering problem leveraging hierarchical density-based spatial clustering, and (ii) an intra-cluster E2E CMT-SemCom learning problem. Simulation results demonstrate that the proposed framework effectively mitigates destructive cooperation and negative transfer, yielding accuracy gains compared to unclustered multi-tasking and individual training baselines.

Ahmad Halimi Razlighi, Maximilian H. V. Tillmann, Edgar Beck +2

Department of Communications Engineering, University of Bremen, Germany · Communication Systems Group, Paderborn University, 33098 Paderborn, Germany

ClusteringSemantic CommunicationMulti-Task LearningTask-Oriented Semantic CommunicationNegative Transfer
AskPDF
Jul 16, 2026cs.RO

Goal-Oriented Semantic Communication for Distributed ISAC-Enabled Vehicle Coordination

Vehicle coordination at unsignalized intersections relies on accurate real-time vehicle state acquisition and reliable command-and-control (C&C) signal delivery. However, existing studies typically treat sensing, communication, and control separately, which may lead to redundant transmissions, outdated state information, and unreliable vehicle coordination. In this paper, we investigate a new scenario of distributed integrated sensing and communication (ISAC)-enabled vehicle coordination at intersections, where multiple roadside units (RSUs) collaboratively transmit sensing signals for vehicle state acquisition and C&C signals for vehicle movement control under the management of a central base station (BS). To improve signaling efficiency, we propose a unified goal-oriented semantic communication (GSC) framework, which transmits sensing and C&C signals only when they are semantically important for improving intersection traffic throughput. Specifically, an extended Kalman filter (EKF) is adopted to predict vehicle states and fuse distributed sensing measurements. A masked hybrid proximal policy optimization (MHPPO) framework is then developed to jointly determine sensing transmission decisions, C&C transmission decisions, and C&C signal contents based on a value-of-information (VoI) reward. Furthermore, we propose an uncertainty-aware transmission design (UTD), including robust beamforming and VoI-based time-division power allocation, to improve sensing and communication reliability under vehicle state uncertainty and inter-RSU interference. Simulation results show that our proposed framework achieves 100% collision-free vehicle coordination with significantly reduced signaling overhead compared with predictive ISAC baselines adapted from state-of-the-art related studies and several ablation baselines.

Wenjie Liu, Yansha Deng

Department of Engineering, King’s College London, Strand, London WC2R 2LS, U.K.

Multi-Agent CoordinationIntegrated Sensing and CommunicationIntelligent Transportation SystemsSemantic CommunicationTask-Oriented Semantic Communication
AskPDF
Jun 12, 2026eess.IV

Explainable Task-Oriented Token Communication for AI-Native 6G Networks

The integration of Foundation Models (FMs) and wireless communications is driving the evolution of image communication from bit-accurate transmission toward task-oriented transmission. However, existing task-oriented image communication methods still face three major challenges: insufficient task-oriented Token representation, inadequate collaboration between Visual Tokens and Task Tokens, and limited interpretability of task decisions. To address these challenges, we propose an Explainable Task-Oriented Token Communication (ET-TokenCom) framework. By treating Tokens as unified units for information representation and transmission, the proposed framework constructs an end-to-end communication link that spans visual perception, wireless transmission, and task reasoning. At the transmitter, the ET-TokenCom framework extracts Visual Tokens from images to preserve low-level visual information. Meanwhile, Task Tokens generated by the FM are introduced to represent the target information and decision intent required by the current task. A Cross-Modal Attention (CMA) fusion mechanism is further designed, enabling Task Tokens to explicitly guide the selection, weighting, and transmission of Visual Tokens. At the receiver, the framework integrates Token decoding with an explainable output mechanism, where attention heatmaps are generated to highlight critical perceptual regions under different task objectives and reveal the influence of Task Tokens on the outputs. Finally, simulation results validate the effectiveness and robustness of the proposed ET-TokenCom framework.

Feibo Jiang, Lei Mao, Li Dong +3

6G NetworksVisual TokenizationSemantic CommunicationTask-Oriented Semantic CommunicationWireless Communications
AskPDF
May 20, 2026cs.LG

TONIC: Token-Centric Semantic Communication for Task-Oriented Wireless Systems

Tokens are becoming the basic units through which foundation models represent and process information for understanding and inference. However, traditional wireless communication, centered on bit-level fidelity, faces a mismatch between what is transmitted reliably and what downstream models actually consume. This mismatch calls for a communication design that directly accounts for token-level task relevance and downstream model requirements, rather than treating all transmitted bits as equally important. In this paper, we propose TONIC, a token-centric semantic communication framework for task-oriented wireless systems. The transmitter converts each source sample into a sequence of tokens, estimates token-level task relevance, and allocates protection through utility-aware unequal error protection under a fixed channel-use budget. At the receiver, token-level confidence is used to gate unreliable decisions, turning harmful substitutions into recoverable erasures before a Transformer-based completion model restores the masked tokens for final task inference. Our framework combines transmitter-side semantic-aware protection with receiver-side confidence-aware gating in a modular and interpretable architecture, rather than relying solely on fully black-box end-to-end learning. We further establish a utility-aware Bayes-risk interpretation for the receiver-side gating rule and study its interaction with unequal protection and completion. Experimental results on image classification show that TONIC consistently outperforms separation-based schemes, the pixel-domain DeepJSCC baseline, and token-domain baselines under matched communication budgets over AWGN, Rayleigh, and Rician channels.

Sige Liu, Kezhi Wang

Department of Computer Science, Brunel University London, Uxbridge UB8 3PH, U.K.

Semantic CommunicationTask-Oriented Semantic CommunicationWireless CommunicationsWireless Resource Allocation
AskPDF
May 15, 2026cs.LG

World Model-Enabled Causal Digital Twins for Semantic Communications in Physical AI Systems

Semantic communication has emerged as a promising paradigm for enabling goal-oriented networking. However, most existing semantic communication solutions are tailored to one-shot tasks and optimize instantaneous performance. Hence, they cannot be used to support closed-loop dynamic systems with physical artificial intelligence (AI), in which the transmitted semantics affect not only the current inference outcome but also future control actions, state evolution, and ultimately long-horizon task performance. To address this gap, this paper investigates goal-oriented semantic communications for physical AI systems with closed-loop sensing-communication-inference-control. In particular, the problem of semantic communications is formulated as a long-term return-per-bit maximization under wireless bit-budget constraints while capturing both control efficiency and communication efficiency. To solve this problem, a novel causal information value (CIV) metric is introduced to evaluate the marginal contribution of each semantic token to the expected long-term return by transmission interventions. Then, a world-model-enabled causal digital twin (WM-CDT) framework is proposed to capture the dynamics of closed-loop physical AI systems and enable counterfactual reasoning for long-horizon imagined rollouts. Based on these imagined rollouts, an actor-critic policy is trained for long-horizon agent control with high data efficiency, while the semantic token selector is trained through CIV-per-bit evaluation. Extensive simulations on an AirSim-Sionna-based unmanned aerial vehicle (UAV) navigation simulator show that the proposed WM-CDT framework achieves significant improvement in return-per-kbit and navigation success rate compared to existing reinforcement learning solutions.

Lingyi Wang, Tingyu Shui, Walid Saad +1

Bradley Department of Electrical and Computer Engineering, Virginia Tech, Alexandria, VA, 22305, USA · Research and Innovation Labs, InterDigital Communications Inc., New York, NY 10120, USA

Semantic CommunicationTask-Oriented Semantic CommunicationDigital Twins
AskPDF
May 11, 2026cs.CV

ChronoSC: Task-Oriented Semantic Communication via Temporal-to-Color Encoding

Semantic communication (SC) aims to reduce transmission overhead by conveying task-relevant information rather than raw data. However, existing SC approaches for video largely focus on pixel-level reconstruction or rely on complex spatiotemporal pipelines, leading to excessive bandwidth usage and latency that are unsuitable for low-resource deployments. In this paper, we propose ChronoSC, a task-oriented semantic communication framework for Video Question Answering (VideoQA). ChronoSC introduces Chrono-Color Stacking, a lightweight and lossless projection scheme that encodes temporal video dynamics into a single static image, enabling extreme temporal compression before transmission. This compact semantic representation is transmitted using a lightweight Deep Joint Source-Channel Coding (DeepJSCC) transceiver and explicitly reconstructed at the receiver. Unlike latent-space methods, explicit visual reconstruction enables the direct reuse of pre-trained vision-language models; specifically, a pre-trained BLIP model is employed to infer answers from noisy, reconstructed chrono-images. Experiments on the CLEVRER dataset show that ChronoSC achieves up to 192 times bandwidth reduction compared to raw video transmission while maintaining high VideoQA accuracy.

Phuc H. Nguyen, Trung T. Nguyen, Quy N. Duong +1

Smart Green Transformation Center (GREEN-X), VinUniversity, Vietnam. · School of Computer Science and Statistics, Trinity College Dublin, Ireland

Semantic CommunicationVideo QATask-Oriented Semantic CommunicationVideo Compression
AskPDF
Apr 22, 2026cs.MM

Sema: Semantic Transport for Real-Time Multimodal Agents

Real-time multimodal agents transport raw audio and screenshots using networking stacks designed for human receivers, which optimize for perceptual fidelity and smooth playout. Yet agent models act as event-driven processors with no inherent sense of physical time, consuming task-relevant semantics rather than reconstructing signals in real time. This fundamental difference shifts the transport goal from the technical problem of signal fidelity (Shannon-Weaver Level A) to the semantic problem of meaning preservation (Level B). This mismatch imposes significant overhead. In visual pipelines, screenshot upload accounts for over 60% of end-to-end action latency on constrained uplinks, and in voice pipelines, conventional transport carries massive redundancy, sending 43-64x more data than needed to maintain task accuracy. We present Sema, a semantic transport system that combines discrete audio tokenizers with a hybrid screen representation (lossless accessibility-tree or OCR text, plus compact visual tokens) and bursty token delivery that eliminates jitter buffers. In simulations under emulated WAN conditions, Sema reduces uplink bandwidth by 64x for audio and 130-210x for screenshots while preserving task accuracy within 0.7 percentage points of the raw baseline.

Jiaying Meng, Bojie Li

Unaffiliated · Pine AI

Semantic CommunicationTask-Oriented Semantic CommunicationMultimodal Token CompressionMultimodal Agents
AskPDF
Jun 24, 2025cs.CR

Diffusion-aided Task-oriented Semantic Communications with Model Inversion Attack

Semantic communication enhances transmission efficiency by conveying semantic information rather than raw input symbol sequences. Task-oriented semantic communication further aims to retain only task-specific information, thereby achieving greater bandwidth savings. However, these neural-network-based communication systems are vulnerable to model inversion attacks, in which adversaries attempt to recover sensitive input information from intercepted semantic features. The key challenge is therefore to preserve privacy while maintaining task accuracy and robustness. We consider a task-confidential setting in which the adversary attempts to reconstruct the original input from intercepted features without knowing the legitimate receiver's task or model. Although PSNR and SSIM are commonly used to assess reconstruction quality, we find that an external classifier can still perform the legitimate receiver's task with nontrivial accuracy on reconstructions with low PSNR or SSIM, indicating that these reconstructions still contain task-level semantic leakage. We therefore propose DiffSem, which splits the diffusion process between controlled transmitter-side self-noising and matched receiver-side reverse denoising. Experiments on the MNIST, CIFAR-10, and CelebA datasets show that DiffSem improves the legitimate receiver's task accuracy without increasing either the transmitted feature size or information leakage.

Xuesong Wang, Mo Li, Xingyan Shi +2

School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, Guangdong 518172, China

Semantic CommunicationTask-Oriented Semantic CommunicationModel Inversion AttacksDiffusion Models
AskPDF