Separating a radio-frequency signal of interest from a co-channel interference is mostly addressed with specialized separators, each trained for one pair of signal and interference types and therefore relying on a label that a heavily jammed receiver cannot identify. We show that this reliance is not benign: applied to an interference it was not trained on, such a separator usually fails to recover the signal, and in most of the cases we measure it leaves the signal worse than no processing at all. We then train a single separator over every pair, giving it no indication at test time of which one it faces, on two architectures with no component in common: the convolutional WaveNet of the ICASSP 2024 RF Challenge, and a band-split transformer that we adapt from music source separation. Both behave the same way. At the size of one specialized model, the unified separator stays close to the whole models it replaces, keeping the bit error rate close on almost every pair, better on one and worse on another. On an interference absent from every training set, it does as well as the best specialized model, without anyone having to choose which one to deploy.
Figures & tables
Figure 1: Models applied to a mixture of a QPSK signal of interest and an interference absent from every training set. The proposed unified band-split transformer reaches out-of-domain performance as good as the best specialized model, without having to choose which one to deploy. A poor choice degrades the signal instead: three of the eight specialists are worse than the raw signal at one or more points, marked × .
Figure 2: The proposed band-split separator. The mixture y is analyzed by a short-time Fourier transform (STFT), its bins are split into K=16 uniform bands and the result is processed by N=12 blocks that alternate attention along time with attention across bands. The head produces a complex ratio mask M of unbounded magnitude, applied bin by bin to the spectrogram Y of the mixture; the inverse transform returns the estimated signal of interest s^ . Sec. 2.1 details each stage.
Published [ 7 ]
This paper
WaveNet
WaveNet
Band-split transformer
Spec.
Unified
Spec.
Unified
Unified mid
Unified small
Unified cond.
Pair
ber ↓
mse ↓
ber ↓
mse ↓
ber ↓
mse ↓
ber ↓
mse ↓
ber ↓
mse ↓
ber ↓
mse ↓
ber ↓
mse ↓
QPSK+CS2
−18.01
−24.93
−19.66
−26.37
−18.06
−21.49
−19.63± 0.01
−22.17± 0.10
−19.50
−20.04
−19.23
−16.67
−17.99± 0.03
−22.07± 0.12
QPSK+CS3
−1.83
−4.59
−1.53
−4.31
−1.23
−3.69
−0.77± 0.02
−2.55± 0.02
−0.61
−2.21
−0.56
−2.14
−1.57± 0.20
−1.80± 1.02
QPSK+CS5G1
−21.43
−29.79
−16.08
−18.76
−30.00
−33.91
−24.58± 1.66
−23.04± 2.02
−26.68
−23.11
−6.42
−8.13
−29.31± 0.61
−29.91± 0.23
Table 1: Source separation performance on TestSet1. ber ↓ is the BER score of Sec. 3.1 , the SINR in dB at which the BER curve crosses 10−2 ; -- a curve that never reaches it; a −30.0 entry marks the left edge of the grid, so the true BER score is lower. mse ↓ is the official MSE score in dB. Spec. is one model per pair, Unified one model for the eight, mid and small the same at D=256 , N=8 and D=144 , N=6 ; ours use D=384 , N=12 unless stated. Unified and Unified cond.: mean and standard deviation over three seeds; every other column is a single run. Unified cond. is given the interference and SOI labels, and is therefore excluded from the comparison. Bold : best of the six other columns, separately for ber and mse .
Figure 3: Evaluation of the specialists and unified model on different interference. Rows: interference actually mixed with the SOI. Columns: model deployed, the four specialists of the SOI and, last column, the unified model. Cell: BER score in dB, -- when the curve never reaches 10−2 . Color: BER score minus that of the row’s own specialist, in dB, clipped at 4 . Top: WaveNet. Bottom: band-split transformer. Left: QPSK SOI. Right: OFDM SOI. Only the interference is mismatched: the SOI is assumed correctly identified.
Single-channel radio-frequency (RF) signal separation is important for interference mitigation when a signal of interest overlaps with a co-channel interferer in time and frequency. Classical matched filtering and LMMSE receivers are limited under structured non-Gaussian interference, while diffusion and score-based methods often require iterative sampling or optimization. We propose FlowSep-RF, a rectified flow matching framework that learns a clean-endpoint transport from the received mixture to the signal of interest. The model combines time-domain IQ samples with frequency-domain features and recovers the separated waveform using one endpoint velocity evaluation. We evaluate FlowSep-RF on QPSK mixtures with OFDM-BPSK, OFDM-QPSK, and RFChallenge CommSignal2 interference against classical, discriminative, and generative baselines. FlowSep-RF achieves the lowest average BER and MSE among evaluated baselines across all three settings with one-step inference.
Mohamed Kamel, Tamer Nadeem
Department of Computer Science, Virginia Commonwealth University, Richmond, VA, USA
We study a problem of signal separation: estimating a signal of interest (SOI) contaminated by an unknown non-Gaussian background/interference. Given the training data consisting of examples of SOI and interference, we show how to build a fully data-driven signal separator. To that end we learn a good discrete tokenizer for SOI and then train an end-to-end transformer on a cross-entropy loss. Training with a cross-entropy shows substantial improvements over the conventional mean-squared error (MSE). Our tokenizer is a modification of Google's SoundStream, which incorporates additional transformer layers and switches from VQVAE to finite-scalar quantization (FSQ). Across real and synthetic mixtures from the MIT RF Challenge dataset, our method achieves competitive performance, including a 122x reduction in bit-error rate (BER) over prior state-of-the-art techniques for separating a QPSK signal from 5G interference. The learned representation adapts to the interference type without side information and shows zero-shot generalization to unseen mixtures at inference time, underscoring its potential beyond RF. Although we instantiate our approach on radio-frequency mixtures, we expect the same architecture to apply to gravitational-wave data (e.g., LIGO strain) and other scientific sensing problems that require data-driven modeling of background and noise.
AI-enhanced interference rejection in radio frequency (RF) transmissions has recently attracted interest because deep learning approaches trained on both the signal of interest (SOI) and the signal mixture (SOI plus interference) can outperform traditional approaches which only consider the SOI. The goal is to detect, demodulate, and decode signals over a range of signal-to-interference-plus-noise (SINR) levels without having a detailed, design-level knowledge of the interfering signal or the propagation conditions. Our present AI interference suppression results are based on Autoregressive Transformer Decoder models which exhibit orders of magnitude faster throughput at inference time than WaveNet models developed in earlier work. As a specific example, we investigate an analog FM "Walkie Talkie" radio signal of interest in the presence of an Orthogonal Frequency-Division Multiplexing (OFDM) interferer. This type of interferer is near-ubiquitous in the current RF landscape. Our results clearly show the benefits of transformer-based interference mitigation in tactical settings. We show that unintelligible transmissions become intelligible via metrics such as Perceptual Evaluation of Speech Quality (PESQ), while overall latency is kept to a minimum using readily available lightweight GPUs such as a Jetson AGX Orin. We believe these same techniques can also be applied to a broader set of national security scenarios, as well as having commercial applications.