Leveraging Distributed Sensing and Hyperdimensional Computing for Non-Line-of-Sight Passive Object Sensing
Authors: Placido Mursia, Marco Rossanese, Francesco Devoti, Adrian Lendinez Ibanez, Lanfranco Zanzi, Xi Li, Xavier Costa-Perez
Organizations: NEC Laboratories Europe GmbH, Heidelberg, Germany. · Univeristy of Bedfordshire, Luton, United Kingdom. · NEC Laboratories Europe GmbH, Heidelberg, Germany, i2CAT Foundation, Barcelona, Spain, and ICREA Foundation, Barcelona, Spain.
In smart factory environments, passive entities such as humans and movable obstacles may interfere with autonomous robot operations, posing significant safety risks. Detecting potential conflicts between robots and entities outside their field-of-view (FoV) is therefore of paramount importance. However, conventional passive object detection systems rely on sensors such as LiDAR, radar, or cameras, which operate under direct line-of-sight (LoS) and are limited by obstacles and occlusions. To overcome these limitations, integrated sensing and communication (ISAC) enables environmental perception via the communication infrastructure thanks to cooperation among distributed wireless devices. In this paper, we propose a framework for passive entity sensing beyond the robot's FoV that relies solely on distributed channel measurements from the communication infrastructure to perform risk-aware classification. These measurements are embedded into high-dimensional representations using hyperdimensional computing (HDC), enabling robust and scalable learning without explicit channel reconstruction. Numerical simulations based on a commercial ray tracer and ROS-based validation in realistic smart factory scenarios demonstrate improved safety and enhanced path planning with up to 89% accuracy under severe occlusions, while maintaining low computational complexity.
Figures & tables
Fig. 1 : Considered smart factory scenario where humans and robots operate concurrently.
Fig. 2 : Proposed system architecture combining distributed wireless sensing and HDC to complement onboard robot sensors, allowing them to preemptively perform risk assessment of passive entities beyond their FoV , and update their path.
Fig. 3 : Group-level encoding for a given target position.
Fig. 4 : Class accumulation with iterative retraining tuning.
Fig. 5 : Simulated warehouse environment layouts.
Fig. 6 : Performance comparison for RX 1, RX 2, and Joint RXs configurations.
Fig. 7 : Similarity difference maps (Danger - Safe) for RX 1, RX 2, and Joint RXs. Red/blue areas indicate danger/safe predictions; dashed rectangle delimits the two regions.
Fig. 8 : Comparison between the HDC encodings and DNN baseline in the joint scenario.
Operation
D = 100
D = 1,000
D = 10,000
MAP
Binary
MAP
Binary
MAP
Binary
Bundle
43.0
4.3
425
26.4
4260
248
Bind
43.0
2.8
426
14.7
4260
135
Permute
9.2
3.4
76
18.3
750
169
Similarity
39.0
4.4
378
24.0
3740
209
TABLE I : MAP vs. Binary HDC execution time ( μs )
Fig. 9 : Resulting costmap for a given navigation goal.
Future integrated sensing and communication (ISAC) architectures separate the sensing entity (SE) that acquires measurements from the sensing function (SF) that performs inference, creating a need for compact, task-oriented feedback on the SE-SF interface. Forwarding the raw channel frequency response or full per-link delay-Doppler-azimuth-elevation (DDAE) tensor is prohibitively expensive, while peak-only reporting discards target-discriminative structure under clutter. We propose a learning-based coarse-to-fine sensing pipeline with candidate-latent feedback for single-target estimation. At the SE, a lightweight convolutional scorer produces a dense delay-Doppler proposal map from pilot-based OFDM channel estimates, and a learned encoder constructs K compact C-dimensional candidate tokens by fusing per-candidate azimuth-elevation patches, normalized position, and confidence cues. The latents are uniformly quantized post-training to b bits and transmitted under a finite budget B_fb = bKC + 18K + 16 bits to the SF, which performs cross-candidate refinement, reranking, and joint four-parameter estimation. On a ray-traced urban scene with static and dynamic clutter, three operating points in the (K, C, b) design space achieve 96.33-98.88% detection at 107-806 bytes per coherent processing interval, compression ratios of 1.2-9.2 x 10^4 over the 8-bit DDAE magnitude tensor, reducing the SE-SF interface from multi-Gbit/s to sub-Mbit/s rates. Cross-scene evaluation on an independent campus-scale environment achieves 98.79-99.50% detection and at-or-better angular accuracy without retraining, indicating that the learned representation captures target-relevant structure that transports across scenes of comparable or lower clutter density.
We investigate an integrated sensing and communication (ISAC)-enabled BS for the unmanned aerial vehicle (UAV) obstacle avoidance task, and propose a goal-oriented semantic communication (GOSC) framework for the BS to transmit sensing and command and control (C&C) signals efficiently and effectively. Our GOSC framework establishes a closed loop for sensing-C&C generation-sensing and C&C transmission: For sensing, a Kalman filter (KF) is applied to continuously predict UAV positions, mitigating the reliance of UAV position acquisition on continuous sensing signal transmission, and enhancing position estimation accuracy through sensing-prediction fusion. Based on the refined estimation position provided by the KF, we develop a Mahalanobis distance-based dynamic window approach (MD-DWA) to generate precise C&C signals under uncertainty, in which we derive the mathematical expression of the minimum Mahalanobis distance required to guarantee collision avoidance. Finally, for efficient sensing and C&C signal transmission, we propose an effectiveness-aware deep Q-network (E-DQN) to determine the transmission of sensing and C&C signals based on their value of information (VoI). The VoI of sensing signals is quantified by the reduction in uncertainty entropy of UAV's position estimation, while the VoI of C&C signals is measured by their contribution to UAV navigation improvement. Extensive simulations validate the effectiveness of our proposed GOSC framework. Compared to the conventional ISAC transmission framework that transmits sensing and C&C signals at every time slot, GOSC achieves the same 100% task success rate while reducing the number of transmitted sensing and C&C signals by 92.4% and the number of transmission time slots by 85.5%.
Wenjie Liu, Yansha Deng, Henk Wymeersch
Department of Engineering, King’s College London, Strand, London WC2R 2LS, U.K. · Department of Electrical Engineering, Chalmers University of Technology, Gothenburg, Sweden.
Robust 3D environmental perception is critical for applications such as autonomous driving and robot navigation. However, optical sensors such as cameras and LiDAR often fail under adverse conditions, including smoke, fog, and non-ideal lighting. Although specialized radar systems can operate in these environments, their reliance on bespoke hardware and licensed spectrum limits scalability and cost-effectiveness. This paper introduces Rascene, an integrated sensing and communication (ISAC) framework that leverages ubiquitous mmWave OFDM communication signals for 3D scene imaging. To overcome the sparse and multipath-ambiguous nature of individual radio frames, Rascene performs multi-frame, spatially adaptive fusion with confidence-weighted forward projection, enabling the recovery of geometric consensus across arbitrary poses. Experimental results demonstrate that our method reconstructs 3D scenes with high precision, offering a new pathway toward low-cost, scalable, and robust 3D perception.
Kunzhe Song, Geo Jie Zhou, Xiaoming Liu +1
Department of Computer Science and Engineering, Michigan State University