MERLIN: Building Low-SNR Robust Multimodal LLMs for Electromagnetic Signals

Junyu Shen; Zhendong She; Chenghanyu Zhang; Yuchuang Sun; Luqing Luo; Dingwei Tan; Zonghao Guo; Bo Guo

cs.CVMar 9, 2026

MERLIN: Building Low-SNR Robust Multimodal LLMs for Electromagnetic Signals

Authors: Junyu Shen, Zhendong She, Chenghanyu Zhang, Yuchuang Sun, Luqing Luo, Dingwei Tan, Zonghao Guo, Bo Guo,

Organizations: Tsinghua University · Tianjin University · Beijing University of Posts and Telecommunications · Institute of Microelectronics of the Chinese Academy of Sciences ·

Abstract

The paradigm of Multimodal Large Language Models (MLLMs) offers a promising blueprint for advancing the electromagnetic (EM) domain. However, prevailing approaches often deviate from the native MLLM paradigm, instead using task-specific or pipelined architectures that lead to fundamental limitations in model performance and generalization. Fully realizing the MLLM potential in EM domain requires overcoming three main challenges: (1) Data. The scarcity of high-quality datasets with paired EM signals and descriptive text annotations used for MLLMs pre-training; (2) Benchmark. The absence of comprehensive benchmarks to systematically evaluate and compare the performance of models on EM signal-to-text tasks; (3) Model. A critical fragility in low Signal-to-Noise Ratio (SNR) environments, where critical signal features can be obscured, leading to significant performance degradation. To address these challenges, we introduce a tripartite contribution to establish a foundation for MLLMs in the EM domain. First, to overcome data scarcity, we construct and release EM-100k, a large-scale dataset comprising over 100,000 EM signal-text pairs. Second, to enable rigorous and standardized evaluation, we propose EM-Bench, the most comprehensive benchmark featuring diverse downstream tasks spanning from perception to reasoning. Finally, to tackle the core modeling challenge, we present MERLIN, a novel training framework designed not only to align low-level signal representations with high-level semantic text, but also to explicitly enhance model robustness and performance in challenging low-SNR environments. Comprehensive experiments validate our method, showing that MERLIN is state-of-the-art in the EM-Bench and exhibits remarkable robustness in low-SNR settings.

Figures & tables

Subset	Signal Type	Mission Type	Data Source	Number of Samples
MOD	–	14 types of modulation	Real-world	588,000
PE	LFM/Barker/Frank/Stepped/Rectangular	6 types of parameters	Simulation	7,310,600
PI	BPSK/QPSK/16-QAM/64-QAM/256-QAM	8 types of protocols	Simulation	911,895
SD	–	9 types of Jamming	Simulation	162,000
RJR	LFM/Barker/Noise/Frank/Sin/Rectangular	12 types of Jamming	Simulation	4,918,700
CJR	BPSK/QPSK/16-QAM/64-QAM/GFSK	9 types of Jamming	Simulation	4,984,875

Table 1: Composition of the large-scale data corpus used for creating EM-134K and EM-Bench. This table details the 10 subsets, totaling over 35 million signal samples, that form our foundational data pool.

Figure 1: The hierarchical evaluation framework of EM-Bench, which systematically assesses the perception and reasoning capabilities of MLLMs on electromagnetic IQ signals across 3 levels and 14 sub-tasks.

Figure 2: The two-stage construction pipeline for the EM-134K fine-tuning dataset and the EM-Bench evaluation benchmark. Stage 1 focuses on building a large-scale, expert-validated IQ data corpus. Stage 2 details the distinct generation processes: a direct, large-scale formatting for EM-134K, and a rigorously expert-validated pathway for the high-quality EM-Bench.

Figure 3: The architecture and training framework of MERLIN. (1) The baseline model architecture consists of a Signal Encoder, a Projector, and a LLM. (2) The knowledge distillation framework enhances low-SNR robustness by using a frozen high-SNR teacher model to guide a student model. (3) The Denoising Subspace Module (DSM) facilitates effective distillation by projecting noisy signal features into a clean, noise-invariant feature space.

Panel of Figure 4 — Figure 4: Motivational analysis demonstrating that low-SNR degradation is a feature-collapse problem

Model	Perception Tasks											Reasoning Tasks
	MOD	PE (Radar)					PI	CJR	RJR	SD	Avg.	Anti-CJ	Anti-RJ	CLS	RJS
	MOD	BW	DC	NP	PRF	PW	PI	CJR	RJR	SD	Avg.	Rouge-L/BLEU	Rouge/BLEU	Rouge/BLEU	Rouge/BLEU
GPT-5	28.00	12.67	0.00	52.67	14.00	6.67	30.00	19.67	26.00	42.33	23.20	0.01 / 0.00	0.00 / 0.00	0.03 / 0.00	0.01 / 0.00
Claude-4-Sonnet	30.00	24.00	38.00	52.00	25.17	31.00	29.67	25.67	30.67	44.33	32.35	0.11 / 0.00	0.10 / 0.00	0.10 / 0.00	0.12 / 0.01
DeepSeek-v3.2-exp	18.00	9.33	9.67	19.67	13.67	17.00	8.33	5.67	11.00	18.33	13.07	0.05 / 0.00	0.06 / 0.00	0.09 / 0.00	0.09 / 0.00
Qwen3-Next-80b-A3B	17.00	7.33	14.67	29.33	15.33	16.33	10.00	9.33	13.67	24.00	15.70	0.01 / 0.00	0.01 / 0.00	0.03 / 0.00	0.01 / 0.00

Table 2: Main results on EM-Bench. We compare MERLIN with leading proprietary and open-source LLMs. All baseline LLMs process EM signals in a textualized format. The best results are highlighted in bold . ”PE” denotes Parameter Estimation tasks. Its sub-tasks denote BW : Bandwidth, DC : Duty Cycle, NP : Num. Pulses, PRF : Pulse Rep. Freq, PW : Pulse Width. MERLIN achieves state-of-the-art performance across both perception and reasoning tasks.

Method	Components			Perf. (%)
Method	$\mathcal{L}_{MLP}$	DSM	$\mathcal{L}_{Logits}$	Low-SNR	Overall
Stage-1	$\times$	$\times$	$\times$	59.7	71.8
+ Stage-2	$\times$	$\times$	$\times$	62.9	77.6
+ Feature KD	✓	$\times$	$\times$	64.2	77.9
+ DSM	✓	✓	$\times$	64.4	78.0
MERLIN	✓	✓	✓	65.1	78.6

Table 3: Ablation of MERLIN’s distillation components. We show the impact of incrementally adding each component on model performance, starting from the pre-trained baseline.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Configuration	Parameter Value
Dataset (Stage 1)	EM-100K (100K instruction pairs)
Dataset (Stage 2)	Synthetic low-SNR variants (SNR = $-$ 20dB to 20dB)
Batch Size	256
LR: Signal Encoder (SIT)	$5.0\times 10^{-5}$ (cosine decay)
LR: Projector, LLM	$5.0\times 10^{-5}$
Optimizer	AdamW ( $\beta_{1}=0.9,\beta_{2}=0.999,\epsilon=1\text{e-}8$ )

Appendix

Table 4: Training configuration details for MERLIN framework. Hyperparameters used during the two-stage training. SIT = Signal Integration Transformer encoder.

L1-Task	L2-Task	L3-Task	Abbr.	Annotation Format	Number of Samples	Answer Type
Perception	Signal Characterization	Modulation Classification	MOD	VQA	300	Single Choice(A/B/C/D/E)
		Duty Cycle Estimation	PE.DC	VQA	300	Single Choice(A/B/C/D/E)
		Pulse Repetition Frequency Estimation	PE.PRF	VQA	300	Single Choice(A/B/C/D/E)
		Bandwidth Estimation	PE.BW	VQA	300	Single Choice(A/B/C/D/E)
		Pulse Width Estimation	PE.PW	VQA	300	Single Choice(A/B/C/D/E)
		Pulse Number Estimation	PE.NoP	VQA	300	Single Choice(A/B/C/D/E)

Appendix

Table 5: Overview of the EM-Bench evaluation framework. The benchmark features a three-level hierarchy spanning Perception and Reasoning, consisting of 14 distinct L3-Tasks. Each sub-task contains 300 samples, totaling 4,200 VQA pairs. The table lists the specific abbreviations, data quantities, and answer types (Single Choice vs. Open-ended Strategy) for each task.

Panel of Figure 5 — Figure 5: Detailed performance comparison across 10 sub-tasks. The red line denotes MERLIN and the gray line denotes the Stage-1 baseline. MERLIN demonstrates significant improvements in complex parameter estimation tasks (e.g., Duty Cycle, Pulse Width) across all SNR levels, while maintaining robust performance in classification tasks.

Multimodal Large Language Models

Multimodal large language models combine language models with image, video, audio, and other inputs for grounded understanding and generation.

See all 1,091 papers

PReD: An LLM-based Foundation Multimodal Model for Electromagnetic Perception, Recognition, and Decision
Mar 30, 2026Zehua Han, Jing Xiao, Yiqi Duan +13Wireless Foundation Models Multimodal Foundation Models
Beihang University · Civil Aviation University of China · Beijing Institute of Technology +10
EMRB: A Multi-Level Benchmark for Evaluating LLM Reasoning over Raw Electromagnetic Signals
Aug 25, 2026Mingxu Zhang, Ying Sun, Yuhan Li +4LLM Evaluation Scientific Reasoning in Language Models
The Hong Kong University of Science and Technology (Guangzhou) · The 63rd Research Institute, National University of Defense Technology · Nanjing University of Aeronautics and Astronautics
EmoMM: Benchmarking and Steering MLLM for Multimodal Emotion Recognition under Conflict and Missingness
May 1, 2026Yueru Sun, Yimeng Zhang, Haoyu Gu +5Missing-Modality Learning Multimodal Large Language Models
School of Future Technology, South China University of Technology, Guangzhou, China

Mar 30, 2026J/K move · Enter open · S save

PReD: An LLM-based Foundation Multimodal Model for Electromagnetic Perception, Recognition, and Decision

Zehua Han, Jing Xiao, Yiqi Duan, Mengyu Xiang+12

Beihang University · Civil Aviation University of China · Beijing Institute of Technology+10

Wireless Foundation Models Multimodal Foundation Models Multimodal Pretraining

Open paper AskPDF arXiv

Multimodal Large Language Models have demonstrated powerful cross-modal understanding and reasoning capabilities in general domains. However, in the electromagnetic (EM) domain, they still face challenges such as data scarcity and insufficient integration of domain knowledge. This paper proposes PReD, the first foundation model for the EM domain that covers the intelligent closed-loop of "perception, recognition, decision-making." We constructed a high-quality multitask EM dataset, PReD-1.3M, and an evaluation benchmark, PReD-Bench. The dataset encompasses multi-perspective representations such as raw time-domain waveform, frequency-domain spectrograms, and constellation diagrams, covering typical features of communication and radar signals. It supports a range of core tasks, including signal detection, modulation recognition, parameter estimation, protocol recognition, radio frequency fingerprint recognition, and anti-jamming decision-making. PReD adopts a multi-stage training strategy that unifies multiple tasks for EM signals. It achieves closed-loop optimization from end-to-end signal understanding to language-driven reasoning and decision-making, significantly enhancing EM domain expertise while maintaining general multimodal capabilities. Experimental results show that PReD achieves state-of-the-art performance on PReD-Bench constructed from both open-source and self-collected signal datasets. These results collectively validate the feasibility and potential of vision-aligned foundation models in advancing the understanding and reasoning of EM signals.

Abstract

Multimodal Large Language Models

Explore similar work

PReD: An LLM-based Foundation Multimodal Model for Electromagnetic Perception, Recognition, and Decision

EMRB: A Multi-Level Benchmark for Evaluating LLM Reasoning over Raw Electromagnetic Signals

EmoMM: Benchmarking and Steering MLLM for Multimodal Emotion Recognition under Conflict and Missingness