cs.CVOct 24, 2026

FreqPKC: Frequency-Aware PeakConv for Radar Semantic Segmentation

Authors: Yangyong Deng, Pengyu Zhang, Shaoqiu Song, Xiaotao Huang, Jian Wang

Organizations: College of Electronic Science and Technology, National University of Defense Technology, Changsha, Hunan 410073, China.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Automotive millimeter-wave (mmWave) radar has attracted increasing attention for semantic segmentation in autonomous driving, owing to its robustness under adverse weather and lighting conditions. Recent works have introduced radar-specific neural operators such as PeakConv (PKC), inspired by constant false alarm rate (CFAR) detection, to better model the peak-dominated characteristics of radar images. However, existing methods typically treat the guard bandwidth as learnable parameters, without explicitly characterizing the frequency response of the PKC operator. In this work, we revisit PKC from a frequency-domain perspective and propose Frequency-Aware PeakConv (FreqPKC), which explicitly models and exploits its underlying spectral characteristics. By analyzing the frequency response of PKC, FreqPKC enables the network to capture a broader range of frequency components and adaptively adjust its effective bandwidth. With the proposed Frequency-Aware (FA) module, the network can infer appropriate guard bandwidths to balance high- and low-frequency information: it enhances high-frequency components in target regions for more precise segmentation while suppressing noise through low-frequency filtering in background areas. Building on FreqPKC, we further develop FreqPKC-Net, a radar semantic segmentation (RSS) framework that integrates frequency-guided adaptive modulation with cross-view fusion for multi-view inputs, and can also be adapted to single-view data. Experiments on the public datasets CARRADA, CARRADA-RAC, and a self-built dataset named RS-SAR demonstrate that FreqPKC-Net consistently outperforms existing SoTA methods on both range-Doppler and range-angle views.

Figures & tables

Fig. 1 : Data processing and spectral analysis visualization. (a) Optical camera image of a driving scene. (b) Corresponding Range-Angle (RA) radar map. (c) Class-wise normalized mean power spectra computed over the entire CARRADA-RAC dataset, showing the spectral separability between different semantic categories (Car, Pedestrian, Cyclist, and Background). The yellow band highlights the mid-frequency region ρ∈[1/8,3/8]\rho\in[1/8,3/8] where target-background discrimination is most pronounced. (d) Measured radial power spectra of the PKC operator under white-noise excitation for varying guard bandwidths bGb_{G} , validating the bandpass nature of PKC and the monotonic decrease of the passband center ρpeak\rho_{\rm peak} with bGb_{G} .
Fig. 2 : Schematic diagram of the Peak Receptive Field (PRF). For a center unit xcx_{c} , the reference cells {xr(i)}i=1Nr\{x_{r}^{(i)}\}_{i=1}^{N_{r}} are sampled outside the guard region defined by guard bandwidth bG\mathbf{b}_{G} and reference bandwidth bR\mathbf{b}_{R} .
Fig. 3 : Overview of the Frequency-Aware PeakConv (FreqPKC). The input features are fed into the Frequency-Aware (FA) module for frequency decoupling, producing multiple sub-band features. Each sub-band is weighted and modulated by learnable weights (Multiply) and the modulated features are fused (Add); the Modulation Module then outputs the guard bandwidth offsets BG†∈R4×H×WB^{G\dagger}\in\mathbb{R}^{4\times H\times W} that control the PKC operation, with b1G†b_{1}^{G\dagger} and b2G†b_{2}^{G\dagger} representing the offsets at two spatial locations, each specifying bandwidth adjustments in four directions (up, down, left, right).
Fig. 4 : Schematic of the Frequency Decoupling process.
Fig. 5 : The Rotary Scan SAR imaging system, consisting of an array radar subsystem with MIMO antennas, a rotating arm subsystem, and an unmanned ground vehicle (UGV). The LiDAR provides auxiliary data to facilitate the annotation of the radar images.
Fig. 6 : Output power distribution of FreqPKC and standard convolution layers across three radar views (RD, RA, AD) under white-noise input. The power ratio in each normalized band [fl,fh)[f_{l},f_{h}) reflects the layer’s frequency selectivity. Bold lines: group averages; Δ\Delta : FreqPKC average minus Conv average.
ModelRD ViewRA View
mIoUmDicemIoUmDice
PKCIn-Net (w/o FA)60.172.647.358.7
w/o frequency decomposition64.176.049.260.9
w/o band-level modulation62.974.749.761.4
w/o adaptive BG†B^{G\dagger}63.575.349.561.1
FreqPKC-Net (full FA)65.677.252.364.2
TABLE I : Component-wise ablation of the FA module on the CARRADA-RAC dataset.
Frameworks#Params @FramesRD ViewRA View
mIoUmDicemIoUmDice
RAMP-CNN [ 45 ]106.4M@956.6%68.5%27.9%30.5%
MVA-Net [ 22 ]4.8M@353.5%65.3%37.1%44.8%
TMVA-Net [ 22 ]5.6M@556.1%68.0%37.7%46.2%
TransRadar [ 49 ]4.9M@556.6%68.9%37.9%46.8%
PKCOn-Net [ 26 ]6.3M@159.5%71.3%39.5%48.3%
TABLE II : Quantitative comparison with SoTA methods on the CARRADA dataset. The best results are highlighted in bold . The second best results are underlined .
Frameworks#Params @FramesRD ViewRA View
mIoUmDicemIoUmDice
RAMP-CNN [ 45 ]106.4M@956.5%67.5%37.9%40.5%
MVA-Net [ 22 ]4.8M@353.6%66.3%40.1%53.9%
TMVA-Net [ 22 ]5.6M@556.3%68.1%46.6%57.9%
PKCOn-Net [ 26 ]6.3M@159.9%71.4%45.5%57.3%
PKCIn-Net [ 26 ]6.3M@560.1%72.6%47.3%58.7%
TABLE III : Quantitative comparison with SoTA methods on the CARRADA-RAC dataset. The best results are highlighted in bold . The second best results are underlined .
Fig. 7 : Qualitative segmentation results comparing FreqPKC-Net with selected baseline methods on the CARRADA-RAC dataset. One representative frame is shown from the test split. The top row displays the camera image and RD view results, while the bottom row shows the RA view results. (a) Input RD/RA tensor, (b) Ground truth mask, (c) PKCIn-Net, (d) AdaPKC θ -Net, (e) AdaPKC ξ -Net FiT , (f) FreqPKC-Net (Ours). Different colors represent different object categories. Black: background, Red : car, Yellow : pedestrian, Cyan : cyclist.
Frameworks#Params @FramesmIoUmDice
MVA-Net [ 22 ]1.30M@339.72%45.19%
TMVA-Net [ 22 ]1.20M@539.44%44.61%
PKCIn-Net [ 26 ]1.25M@539.80%45.13%
AdaPKC ξ -Net [ 31 ]1.25M@540.23%45.94%
AdaPKC θ -Net [ 31 ]1.25M@540.96%47.04%
FreqPKC-Net (Ours)1.26M@542.16%49.10%
TABLE IV : Quantitative comparison with SoTA methods on the RS-SAR dataset. The best results are highlighted in bold . The second best results are underlined .
More in this topic

Semantic Segmentation

Semantic segmentation assigns a class label to every image pixel, enabling detailed scene understanding across domains and sensors.

See all 414 papers

Explore similar work

CardsList
Oct 19, 2026

Real-Time Radar Semantic and Panoptic Scene Understanding

Robust scene understanding under varying environmental conditions is essential for safe autonomous driving. While cameras and LiDAR degrade in adverse weather and poor lighting, Frequency-Modulated Continuous-Wave (FMCW) spinning radar provides dependable 360° perception at extended ranges with minimal compute cost. However, real-time FMCW radar scene understanding is challenging due to complex signal returns, limited temporal resolution, and scarce annotated datasets. We introduce StreamRPS, a novel framework for streaming FMCW radar panoptic segmentation, comprising an automated label-generation pipeline via LiDAR scene completion and a real-time streaming network for semantic and panoptic segmentation. Our encoder-decoder streaming architecture comprises a radar-specific hierarchical Transformer encoder and task-specific decoders for semantic or panoptic prediction. Our encoder introduces a stem to capture range-dependent signatures, anisotropic patch embeddings to preserve angular resolution, and axial polar attention to separately model the periodic azimuth and bounded range axes. We further introduce azimuth roll rotation augmentation to simulate ego-vehicle rotation during training. Comprehensive experiments on the SAX and BOREAS datasets validate both our auto-labeling methodology and streaming segmentation network against established baselines. We outperform the strongest baseline by at least 1.4pp while processing sectors at over 400 FPS in a 4-sector streaming setting. We additionally provide manual annotations for 100 SAX radar scans, enabling independent evaluation of the entire framework.

Santosh Parit, Julia Hindel, Rohit Mohan +4

University of Freiburg · University of Oxford

Autonomous Driving PerceptionPanoptic SegmentationSemantic SegmentationmmWave Radar Sensing
AskPDF
Apr 6, 2026

RAVEN: Radar Adaptive Vision Encoders for Efficient Chirp-wise Object Detection and Segmentation

This paper presents RAVEN, a computationally efficient deep learning architecture for FMCW radar perception. The method processes raw ADC data in a chirp-wise streaming manner, preserves MIMO structure through independent receiver state-space encoders, and uses a learnable cross-antenna mixing module to recover compact virtual-array features. It also introduces an early-exit mechanism so the model can make decisions using only a subset of chirps when the latent state has stabilized. Across automotive radar benchmarks, the approach reports strong object detection and BEV free-space segmentation performance while substantially reducing computation and end-to-end latency compared with conventional frame-based radar pipelines.

Anuvab Sen, Mir Sayeed Mohammad, Saibal Mukhopadhyay

Georgia Institute of Technology, Atlanta, Georgia, USA

mmWave Radar SensingEfficient Object DetectionObject Detection
AskPDF
Jun 30, 2026

Learning Structurally Consistent Representations for Multi-View Radar Semantic Segmentation

Radar sensors provide reliable perception under adverse weather and lighting conditions, but their sparse, noisy, and weakly semantic measurements make dense semantic segmentation challenging. Most existing radar segmentation methods rely on grid-based encodings and pairwise interactions, which struggle to capture the higher-order relational structure formed by multiple radar returns from the same physical object. We introduce a unified higher-order structural alignment framework for multi-view radar segmentation. The proposed method refines radar feature representations using learnable hypergraphs to capture higher-order dependencies among spatially related responses. To ensure consistency across heterogeneous radar projections, we further align view-specific features using Unbalanced Optimal Transport (UOT), enabling correspondence-free alignment under varying measurement densities and partial observations. An adaptive attention mechanism then fuses complementary radar views while emphasising structurally informative responses under sparsity and noise. The resulting architecture learns structurally consistent representations across Range Angle (RA), Range Doppler (RD), and Angle Doppler (AD) views and is trained using supervised segmentation together with cross-view consistency regularisation. Experiments on the CARRADA and RADIal benchmarks demonstrate consistent improvements over strong radar-specific baselines, achieving 63.8% mIoU on CARRADA and 83.4% mIoU on RADIal, improving the previous best methods by +1.7 and +2.3 mIoU, respectively. These results highlight the importance of higher-order relational modelling for robust radar perception.

Ali Zia, Muhammad Umer Ramzan, Abdelwahed Khamis +2

School of Computing, Engineering and Mathematical Sciences, La Trobe University, Melbourne, Australia · La Trobe Institute for Sustainable Agriculture & Food (LISAF), Australia · School of Engineering and Applied Sciences, GIFT University, Pakistan +1

Multi-View ConsistencyMulti-View LearningmmWave Radar SensingSemantic Segmentation
AskPDF