Organizations: Department of Engineering, King’s College London, U.K. · Department of Intelligent Systems, KTH Royal Institute of Technology, 10044 Stockholm, Sweden · Department of Automation Technology, ABB Corporate Research Sweden, 72226 Västerås, Sweden · Institute for Digital Communications, FriedrichAlexander-Universität Erlangen-Nürnberg, 91058 Erlangen, Germany
Wirelessly-connected robotic systems empower robots with real-time intelligence by leveraging remote computing resources for decision-making. However, the data exchange between robots and edge servers often overwhelms communication links, introducing latency that degrades task performance. To tackle this, goal-oriented semantic communication (GSC) has been introduced for wirelessly-connected robotic systems to extract and transmit only goal-relevant semantic representations. While this improves task effectiveness, it generally overlooks practical safety requirements. Meanwhile, existing robotics research often treats safety primarily as a control-level problem, without systematically considering safety across sensing, communication, and control in a closed-loop manner. To bridge this gap, we investigate how to enable safety-aware goal-oriented semantic (SA-GS) sensing, communication, and control co-design in wirelessly-connected robotic systems, aiming to maximize the robotic task effectiveness subject to practical safety requirements. We first introduce {an} architecture {for} wirelessly-connected robotic systems and representative use cases. We then summarize general safety requirements and effectiveness metrics across the use cases. Next, we systematically analyze the unique safety and effectiveness challenges in sensing, communication, and control. Based on these, we further present potential SA-GS research directions. Finally, an Unmanned Aerial Vehicle (UAV) target tracking case study validates that one of the presented SA-GS research directions, i.e., semantic-based C&C packet execution, could significantly improve safety rate and tracking success rate by more than 2 times and 4.5 times, respectively.
Figures & tables
Robot Embeded Sensor
Sensor Type
Data Type
Devices
Transmission Frequency
Data Load
Tactile Sensor
Haptic
Tactile Array Sensor
100-1000 Hz
Low-Medium
Torque Sensor
Force/Torque
F/T Sensor (e.g., ATI Mini40)
500-2000 Hz
Low-Medium
Joint Encoder
Joint Angles/Velocities
Optical or Magnetic Encoder
500-1000 Hz
Low
Inertial Measurement Unit (IMU)
Acceleration, Angular Velocity
MPU6050 IMU
100-1000 Hz
Low-Medium
Magnetometer
Heading
BNO055 Orientation Sensor
10-100 Hz
Low
TABLE I: Commonly-used sensors in wirelessly-connected robotic systems.
Fig. 1: Illustration of a wirelessly-connected robotic system.
Use Case
Effectiveness Metrics
Description
Robot Arm Grasping
Grasp Success Rate
The percentage of successful grasps out of the total number of grasping attempts.
Execution Time
The time for the robot arm to complete the entire grasping process.
Mean Picks Per Hour (MPPH)
The average number of successful picks a robot arm can perform in an hour.
UAV-assisted Task
Tracking Accuracy
Measures how closely the UAV maintains its states relative to targets’ states.
Tracking Success Rate
Measures how consistently the UAV can successfully track targets.
Identity F1 score (IDF1)
Evaluate performance in maintaining the correct identities of tracked objects over time.
TABLE II: Table of Effectiveness Metrics
Semantic Representations
Data Type
Robot Arm Grasping
UAV-assisted Task
Multi-robot Exploration
Safety
Effectiveness
Safety
Effectiveness
Safety
Effectiveness
Object’s attributes (e.g., type, material, size, and surface texture)
Image, Point cloud
✓
✓
o
✓
−
−
Object 6D pose
Point cloud
✓
✓
o
✓
−
−
Grasping position
Point cloud
✓
✓
−
−
−
−
Obstacles (e.g., walls, roads, and barriers)
Image, Point cloud
✓
✓
✓
o
✓
✓
Obstacle’s 6D pose
Point cloud
✓
✓
✓
o
✓
✓
TABLE III: Types of Semantic Representations.
Fig. 2: Safety and tracking success rate comparisons over various ds and de .
Traditional safety-critical control methods, such as control barrier functions, suffer from semantic blindness, exhibiting the same behavior around obstacles regardless of contextual significance. This limitation leads to the uniform treatment of all obstacles, despite their differing semantic meanings. We present Safe-SAGE (Social-Semantic Adaptive Guidance for Safe Engagement), a unified framework that bridges the gap between high-level semantic understanding and low-level safety-critical control through a Poisson safety function (PSF) modulated using a Laplace guidance field. Our approach perceives the environment by fusing multi-sensor point clouds with vision-based instance segmentation and persistent object tracking to maintain up-to-date semantics beyond the camera's field of view. A multi-layer safety filter is then used to modulate system inputs to achieve safe navigation using this semantic understanding of the environment. This safety filter consists of both a model predictive control layer and a control barrier function layer. Both layers utilize the PSF and flux modulation of the guidance field to introduce varying levels of conservatism and multi-agent passing norms for different obstacles in the environment. Our framework enables legged robots to safely navigate semantically rich, dynamic environments with context-dependent safety margins.
We investigate an integrated sensing and communication (ISAC)-enabled BS for the unmanned aerial vehicle (UAV) obstacle avoidance task, and propose a goal-oriented semantic communication (GOSC) framework for the BS to transmit sensing and command and control (C&C) signals efficiently and effectively. Our GOSC framework establishes a closed loop for sensing-C&C generation-sensing and C&C transmission: For sensing, a Kalman filter (KF) is applied to continuously predict UAV positions, mitigating the reliance of UAV position acquisition on continuous sensing signal transmission, and enhancing position estimation accuracy through sensing-prediction fusion. Based on the refined estimation position provided by the KF, we develop a Mahalanobis distance-based dynamic window approach (MD-DWA) to generate precise C&C signals under uncertainty, in which we derive the mathematical expression of the minimum Mahalanobis distance required to guarantee collision avoidance. Finally, for efficient sensing and C&C signal transmission, we propose an effectiveness-aware deep Q-network (E-DQN) to determine the transmission of sensing and C&C signals based on their value of information (VoI). The VoI of sensing signals is quantified by the reduction in uncertainty entropy of UAV's position estimation, while the VoI of C&C signals is measured by their contribution to UAV navigation improvement. Extensive simulations validate the effectiveness of our proposed GOSC framework. Compared to the conventional ISAC transmission framework that transmits sensing and C&C signals at every time slot, GOSC achieves the same 100% task success rate while reducing the number of transmitted sensing and C&C signals by 92.4% and the number of transmission time slots by 85.5%.
Wenjie Liu, Yansha Deng, Henk Wymeersch
Department of Engineering, King’s College London, Strand, London WC2R 2LS, U.K. · Department of Electrical Engineering, Chalmers University of Technology, Gothenburg, Sweden.
Collaborative robotics is a representative task-oriented 6G use-case, where communication quality should be reflected in mission execution, environment understanding, and closed-loop operation rather than packet delivery alone. This demo paper presents a robot-edge semantic communication (SemCom) testbed integrating robot-side visual compression, edge-side semantic mapping, and dashboard-based mission interaction. A mobile robot equipped with RGB-D sensing and LiDAR runs ROS 2, while a Jetson Orin edge node performs reconstruction, RTAB-Map mapping, semantic object handling, and browserbased visualization. As an initial proof of concept, RGB frames are encoded on the robot into VQ-VAE tokens using an ONNX Runtime encoder and reconstructed on the edge using a PyTorch decoder. A 320 X 240 image is represented by an 80 X 60 token grid with a packed payload of 5400 bytes, corresponding to a 42.67X reduction relative to model-input RGB bytes. The reconstructed visual stream is further associated with depth, pose, and 3D mapping information to generate a semantic map for downstream robotic applications. The demo exposes the full path from semantic visual transport to object-level map interaction, and provides a practical platform for future task-aware 6G networking studies at the intersection of SemCom, embodied AI, and physical AI-enabled robotics. A video of the demo is available at https://tinyurl.com/Tos09
Peizheng Li, Xinyi Lin, Sajida Gufran +1
Bristol Research and Innovation Laboratory, Toshiba Europe Ltd., U.K.