Learning Nonlinear Waveform Fusion Across Scales for Integrated Sensing and Communication

Jiayu Liu; Kai Wan; Shuo Shao; Giuseppe Caire

cs.AIOct 23, 2026

Learning Nonlinear Waveform Fusion Across Scales for Integrated Sensing and Communication

Authors: Jiayu Liu, Kai Wan, Shuo Shao, Giuseppe Caire

Abstract

Fixed source superposition and frequency partitioning constrain how communication content interacts with a probing reference in integrated sensing and communication (ISAC). We learn nonlinear waveform fusion across scales for image transmission and localization of multiple targets in range and angle. Reception uses the information available at each end. Adaptive temporal feature weighting and spectral updates combine image features and a deterministic linearly frequency-modulated reference in a shared complex state, generating a waveform with fixed total energy over the array frame. At the user equipment (UE), the reference and channel state information condition a neural demixer for image reconstruction. At the base station, the realized transmit pulse defines a coherent dictionary for sequential localization with learned local correction, physical refinement, and residual updates. We compare the proposed waveforms with superposition in the time domain and learned spectral linear fusion on 512 independent test images, using a shared fixed UE receiver and a baseline sensing receiver using coherent peak search. Averaged across eleven task weights, the proposed waveforms reduce normalized localization error by 29.0% and 27.0%, respectively, while increasing mean peak signal-to-noise ratio by 1.40 and 0.82~dB. With the proposed waveforms fixed, learned local correction reduces mean normalized localization error by 12.2% relative to the same sequential receiver without learned correction.

Figures & tables

Fig. 1: System overview: (1) image encoding, fusion of image and LFM features, and beamforming; (2) image recovery using the received frame, LFM reference, and CSI; and (3) coherent range and angle estimation from echoes using the transmitted waveform.

Fig. 2: Waveform generation across scales. (a) Joint state evolution over eight stages, with spectral increments injected before stages 1, 3, 5, and 7 and a final projection to

\mathbf{S}_{\mathrm{raw}}

. (b) Updates of the source branches and temporal interaction within one stage.

Fig. 3: Spectral interaction within band

b

. Initial source spectra provide candidate features and adaptive weights. Mask

M_{b}

, output projection, inverse DFT, and scaling by

\gamma_{b}\kappa_{b}

produce the increment

\Delta\mathbf{H}_{b}

. Here,

\mathcal{N}_{\mathrm{R}}=\mathcal{N}

Fig. 4: Temporal interaction at stage

i

. Normalized joint and source features generate candidates modulated by context and weights that vary across positions. Output projection and residual scaling update the joint state. Here,

\mathcal{N}_{\mathrm{R}}=\mathcal{N}

Parameter	Value
Carrier / sampling rate	2.45 GHz / 150 MHz
LFM bandwidth	150 MHz
Tx / Rx elements; spacing	$8/8$ ; $\lambda/2$
Frame pulses / pulse samples	${\mathsf{P}}=16$ , ${\mathsf{L}}=256$
Spatial modes / observed pulses	${\mathsf{M}}=4$ , ${\mathsf{P}}_{\mathrm{obs}}=1$
Training target count; velocity	${\mathsf{K}}=3$ ; static

TABLE I: Main system and scene parameters.

Configuration	Waveform construction	UE configuration
Proposed	Temporal/spectral updates of a shared state	Shared UE using reference and CSI
TD superposition [ 23 ]	Fixed source combination, $\rho=0.5$	Same UE processing
Learned spectral linear	Complex source mixing in eight bands; 512 real parameters	Same UE processing
Temporal nonlinear	Adaptive multiscale temporal features; no spectral path	Same UE processing
Fixed FDM with adapted UE [ 24 ]	Fixed complementary subbands and source filters	Matched subband UE, adapted for a fixed Tx
Optimized FDM [ 24 ]	Four communication modes; sensing powers and phases learned on fixed subcarriers	Matched subband UE, jointly trained

TABLE II: Waveform and UE configurations. The main waveform comparisons use the baseline sensing DSP with each configuration’s complete realized waveform. Citations identify the underlying principles of superposition or frequency separation.

Fig. 5: Tradeoffs on the development set between PSNR and (a) range MAE, (b) angle MAE, and (c) normalized localization error. Curves span eleven task weights; open circles and triangles mark

w=0

and

w=1

, respectively. All use the baseline DSP receiver. The three methods other than FDM share the UE receiver; TD uses

\rho=0.5

, and FDM uses UEs matched to their respective configurations.

Fig. 6: Image reconstruction on the development set: (a) PSNR and (b) SSIM versus receive SNR at

w=0.5

, with the common UE receiver. Shading indicates 95% bootstrap intervals using images as clusters, conditional on the trained models.

Fig. 7: Original images and Proposed reconstructions for fixed development images 0–2 at $w=0.5$ . Reconstruction columns use 0 and 20 dB receive SNR; labels give PSNR (dB)/SSIM.

Fig. 8: Localization on the development set: (a) range MAE, (b) angle MAE, and (c) normalized localization error versus reference sensing SNR at

w=0.5

and

{\mathsf{K}}=3

. DSP denotes baseline coherent peak search; neural assistance denotes sequential recovery with local correction. Colored bands are conditional 95% bootstrap intervals; gray shading marks the region of high SNR.

Fig. 9: Ablation of local correction with the proposed waveforms fixed: (a) range MAE, (b) angle MAE, and (c) normalized localization error. Both sequential variants share quadratic initialization and one physical refinement per target addition. Points average 512 development images at 10 dB reference sensing SNR; the corrected curve also averages three seeds for receiver training.

Fig. 10: Fixed development example: scene/image 0, pulse 0, $w=0.5$ , 10 dB reference sensing SNR, receiver seed 42. Columns compare Proposed, TD superposition, and learned spectral linear. Top: coherent scores normalized by their peak, with ground truth and estimates from baseline DSP and recovery with neural assistance. Bottom: a common zoom around the farthest target.

Fig. 11: LFM reference and realized proposed waveform (image 0, pulse 0): (a) I/Q from antenna 0 normalized by each waveform’s RMS; (b) aperiodic autocorrelation and a zoom around zero delay; (c) ambiguity magnitude. Correlations are summed coherently across antennas and normalized by total pulse energy. Ambiguity modulates the undelayed signal by $e^{-j2\pi\nu\ell/f_{\mathrm{s}}}$ ; $\nu$ is Doppler frequency and $T_{p}={\mathsf{L}}/f_{\mathrm{s}}$ is pulse duration. LFM has no spatial mapping. Localization uses the model with circular blocks in Section II-C .

Integrated Sensing and Communication

Integrated sensing and communication combines wireless data links with radar-like perception using shared signals, spectrum, and hardware.

See all 48 papers

Sep 27, 2026

Short-Length Code Designs for Integrated Sensing and Communications: A Deep Learning Approach

Integrated sensing and communication (ISAC) enables joint communication and sensing using a shared waveform, but its signal design is challenging due to the inherent trade-off between the two objectives, particularly in the short blocklength regime. This paper proposes an autoencoder (AE)-based framework for ISAC waveform design in noncoherent settings. We derive a modified Cramér-Rao bound for multi-target delay estimation and analyze the maximum-likelihood decoding rule for noncoherent communication under correlated fading. These results reveal structural connections and trade-offs between communication and sensing objectives in waveform design. Based on this analysis, the AE learns waveform representations that jointly optimize both functionalities, with a tunable parameter controlling the trade-off. Simulation results show that the proposed design outperforms conventional schemes in both communication reliability and sensing accuracy, especially under short blocklength and fading conditions.

Muah Kim, Shuangyang Li, Tayyebeh Jahani-Nezhad +2

Faculty of Electrical and Computer Engineering, Dresden University of Technology, Dresden 01069, Germany · Faculty of Electrical Engineering and Computer Science, Technical University of Berlin, Berlin 10587, Germany

Integrated Sensing and Communication Wireless Communications

Ask PDF

Jun 20, 2026

Resolving Multi-Target Association in OFDM-based ISAC via Vision-aided Multi-Modal Learning

Orthogonal frequency division multiplexing (OFDM)-based integrated sensing and communication (ISAC) systems commonly extract target parameters by peak-searching a delay-Doppler map (DDM) constructed from reflected pilots. In multi-target scenarios, this results in ambiguity: the DDM does not reveal which physical target produced which peak, and two targets within the same delay-Doppler resolution cell cannot be separated. We propose a vision-assisted OFDM-ISAC framework that resolves both limitations by fusing wireless and visual modalities. The transmitter encodes an onboard street-view image with deep joint source-channel coding (DeepJSCC) and transmits it over the same OFDM waveform used for sensing; the receiver reconstructs the image, runs a fine-tuned YOLOv5 detector and fuses the resulting per-target features (bounding-box coordinates and class labels) with the DDM and transmitter-receiver geometry through a learned multi-modal network. To stabilize training of the high dimensional delay and Doppler classifiers, we introduce a Kullback Leibler loss against triangular soft labels centered on the ground-truth bin. On a Blender-rendered vehicular testbed, the proposed framework achieves a 16 cm localization root mean square error (RMSE) and a 10.8 ns delay RMSE. An ablation study confirms that removing the visual modality causes a 60x degradation in localization. These results highlight the potential of vision to overcome the data-association and resolution limits of single-modality ISAC.

Meng Hua, Chenghong Bian, Deniz Gunduz

Department of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, U.K. · Department of Electrical and Computer Engineering, Hong Kong University of Science and Technology, Hong Kong.

Integrated Sensing and Communication Wireless Communications Multimodal Sensor Fusion

Ask PDF

Aug 4, 2026

Task-Oriented Candidate-Latent Feedback for Coarse-to-Fine Sensing in Distributed OFDM-ISAC Networks

Future integrated sensing and communication (ISAC) architectures separate the sensing entity (SE) that acquires measurements from the sensing function (SF) that performs inference, creating a need for compact, task-oriented feedback on the SE-SF interface. Forwarding the raw channel frequency response or full per-link delay-Doppler-azimuth-elevation (DDAE) tensor is prohibitively expensive, while peak-only reporting discards target-discriminative structure under clutter. We propose a learning-based coarse-to-fine sensing pipeline with candidate-latent feedback for single-target estimation. At the SE, a lightweight convolutional scorer produces a dense delay-Doppler proposal map from pilot-based OFDM channel estimates, and a learned encoder constructs K compact C-dimensional candidate tokens by fusing per-candidate azimuth-elevation patches, normalized position, and confidence cues. The latents are uniformly quantized post-training to b bits and transmitted under a finite budget B_fb = bKC + 18K + 16 bits to the SF, which performs cross-candidate refinement, reranking, and joint four-parameter estimation. On a ray-traced urban scene with static and dynamic clutter, three operating points in the (K, C, b) design space achieve 96.33-98.88% detection at 107-806 bytes per coherent processing interval, compression ratios of 1.2-9.2 x 10^4 over the 8-bit DDAE magnitude tensor, reducing the SE-SF interface from multi-Gbit/s to sub-Mbit/s rates. Cross-scene evaluation on an independent campus-scale environment achieves 98.79-99.50% detection and at-or-better angular accuracy without retraining, indicating that the learned representation captures target-relevant structure that transports across scenes of comparable or lower clutter density.

Shiv Shankar, Radha Krishna Ganti, J Klutto Milleth

Indian Institute of Technology (IIT), Madras, Chennai India · Centre of Excellence in Wireless Technology (CEWiT), Chennai India

Integrated Sensing and Communication

Ask PDF

Abstract

Integrated Sensing and Communication

Explore similar work

Short-Length Code Designs for Integrated Sensing and Communications: A Deep Learning Approach

Resolving Multi-Target Association in OFDM-based ISAC via Vision-aided Multi-Modal Learning

Task-Oriented Candidate-Latent Feedback for Coarse-to-Fine Sensing in Distributed OFDM-ISAC Networks