Edge Radar Material Classification Under Geometry Shifts

Jannik Hohmann; Dong Wang; Andreas Nüchter

cs.ROMar 24, 2026

Edge Radar Material Classification Under Geometry Shifts

Authors: Jannik Hohmann, Dong Wang, Andreas Nüchter

Organizations: Department of Informatics XVII (Robotics), Julius-Maximilians-University Würzburg, 97074 Würzburg, Germany. · Zentrum für Telematik e.V., Würzburg · U2IS, ENSTA, Institut Polytechnique de Paris, France.

Abstract

Material awareness can improve robotic navigation and interaction, particularly in conditions where cameras and LiDAR degrade. We present a lightweight mmWave radar material classification pipeline designed for ultra-low-power edge devices (TI IWRL6432), using compact range-bin intensity descriptors and a Multilayer Perceptron (MLP) for real-time inference. While the classifier reaches a macro-F1 of 94.2% under the nominal training geometry, we observe a pronounced performance drop under realistic geometry shifts, including sensor height changes and small tilt angles. These perturbations induce systematic intensity scaling and angle-dependent radar cross section (RCS) effects, pushing features out of distribution and reducing macro-F1 to around 68.5%. We analyze these failure modes and outline practical directions for improving robustness with normalization, geometry augmentation, and motion-aware features.

Figures & tables

Fig. 1 : Default experimental setup used for baseline data acquisition. The radar is mounted in a nadir-facing configuration at $H=45$ cm and $\theta=90^{\circ}$ relative to the material surface.

Fig. 2 : Confusion matrix of the MLP classifier under nominal conditions ( $H=45$ cm, $\theta=90^{\circ}$ ). Diagonal entries indicate per-class recall.

Fig. 3 : Confidence (maximum Softmax probability) distribution under nominal geometry.

Fig. 4 : Confusion matrix under height variations (35 cm and 55 cm). Off-diagonal density increases due to distance-dependent power changes.

Fig. 5 : Confidence distribution under geometry shifts (tilt). Increased low-confidence predictions indicate reduced certainty under OOD geometry.

Fig. 6 : Confusion matrix for “similar but unknown” data at nominal geometry ( $H=45$ cm, $\theta=90^{\circ}$ ). Increased confusion indicates session-level sensitivity despite unchanged geometry.

Fig. 7 : Confidence distribution for “similar but unknown” data at nominal geometry. The probability mass shifts toward lower confidence compared to the baseline.

Explore similar work

VLMaterial: Vision-Language Model-Based Camera-Radar Fusion for Physics-Grounded Material Identification
Apr 13, 2026Jiangyou Zhu, He ChenCamera-Radar Fusion Materials Property Prediction
Department of Inforation Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China
RadarTwin: Scene-Specific mmWave Radar Simulation and Learning for Mobile Indoor Perception
Jun 24, 2026Emily Bejerano, Federico Tondolo, Devang Gupta +5Sim-to-Real Transfer Millimeter-Wave Radar
Columbia University New York, NY, USA · University of California, Merced Merced, CA, USA
Weather-Robust Scene Semantics with Vision-Aligned 4D Radar
May 8, 2026Kali Hamilton, Christoffer HeckmanCross-Modal Learning VLM Robustness
Autonomous Robotics and Perception Group at the University of Colorado Boulder, Boulder, CO 80309, USA.

Apr 13, 2026J/K move · Enter open · S save

VLMaterial: Vision-Language Model-Based Camera-Radar Fusion for Physics-Grounded Material Identification

Jiangyou Zhu, He Chen

Department of Inforation Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China

Camera-Radar Fusion Materials Property Prediction mmWave Radar Sensing Uncertainty-Aware Multimodal Fusion

Open paper AskPDF arXiv

Accurate material recognition is a fundamental capability for intelligent perception systems to interact safely and effectively with the physical world. For instance, distinguishing visually similar objects like glass and plastic cups is critical for safety but challenging for vision-based methods due to specular reflections, transparency, and visual deception. While millimeter-wave (mmWave) radar offers robust material sensing regardless of lighting, existing camera-radar fusion methods are limited to closed-set categories and lack semantic interpretability. In this paper, we introduce VLMaterial, a training-free framework that fuses vision-language models (VLMs) with domain-specific radar knowledge for physics-grounded material identification. First, we propose a dual-pipeline architecture: an optical pipeline uses the segment anything model and VLM for material candidate proposals, while an electromagnetic characterization pipeline extracts the intrinsic dielectric constant from radar signals via an effective peak reflection cell area (PRCA) method and weighted vector synthesis. Second, we employ a context-augmented generation (CAG) strategy to equip the VLM with radar-specific physical knowledge, enabling it to interpret electromagnetic parameters as stable references. Third, an adaptive fusion mechanism is introduced to intelligently integrate outputs from both sensors by resolving cross-modal conflicts based on uncertainty estimation. We evaluated VLMaterial in over 120 real-world experiments involving 41 diverse everyday objects and 4 typical visually deceptive counterfeits across varying environments. Experimental results demonstrate that VLMaterial achieves a recognition accuracy of 96.08%, delivering performance on par with state-of-the-art closed-set benchmarks while eliminating the need for extensive task-specific data collection and training.

Abstract

Explore similar work

VLMaterial: Vision-Language Model-Based Camera-Radar Fusion for Physics-Grounded Material Identification

RadarTwin: Scene-Specific mmWave Radar Simulation and Learning for Mobile Indoor Perception

Weather-Robust Scene Semantics with Vision-Aligned 4D Radar