Small Language Model enabled Autonomous agent for Language-Conditioned Cognitive Radar

Minhaj Uddin Ahmad; Zakia Zaman; Shunqiao Sun; Mizanur Rahman

eess.SPAug 12, 2026

Small Language Model enabled Autonomous agent for Language-Conditioned Cognitive Radar

Authors: Minhaj Uddin Ahmad, Zakia Zaman, Shunqiao Sun, Mizanur Rahman

Organizations: Department of Civil, Construction and Environmental Engineering, University of Alabama · Department of Electrical and Computer Engineering, University of Alabama

Abstract

Modern radar systems require adapting their processing strategies in response to changing interference, clutter, and data availability. This paper introduces a framework for a small language model (SLM)-driven autonomous agent designed for language-conditioned cognitive radar, functioning as an intelligent controller for a suite of array signal processing tools. Given a natural-language command, the agent extracts radar-operation-related cues, selects an appropriate sequence of signal-processing methods, configures parameters, and invokes executable tools for numerical computation. Experiments with a synthetic uniform linear array (ULA) radar demonstrate that, given a natural-language command, the agent performs meaningful algorithm selection across diverse scenarios for sidelobe control, jammer suppression, multiple-null beamforming, coherent-source handling, and low-snapshot direction-of-arrival (DOA) estimation. Ablation results show that radar-specific prompting and physics-grounded tool execution are both required for reliable decisions and hallucination-free numerical results.

Figures & tables

Fig. 1 : Framework for language-conditioned cognitive radar processing loop.

Task category	Example user command	Expected tool choice
Sidelobe control	“Scan toward $30^{\circ}$ with sidelobes below $-30$ dB.”	DAS with Chebyshev/Taylor taper.
Single jammer	“Track a target at $10^{\circ}$ ; a strong jammer appears at $-20^{\circ}$ .”	MVDR if training data are available; MPDR otherwise
Multiple jammers	“Track the target at $10^{\circ}$ and suppress jammers at $-30^{\circ}$ and $25^{\circ}$ .”	LCMV with one distortionless constraint and two null constraints
Low-snapshot DOA	“Only 10 snapshots are available; estimate two closely spaced DOAs.”	ESPRIT or OMP; avoid conventional MUSIC when $L$ is too small
Coherent-source DOA	“Two coherent sources are present; estimate their DOAs.”	MUSIC with forward spatial smoothing or an appropriate sparse method
Underspecified prompt	“There is strong interference; choose a robust processing strategy.”	Ask for missing information or make explicit, physically reasonable assumptions

Table 1: Natural-language benchmark tasks categories.

Method	Alg. acc.	Tool valid	Param. valid	Halluc.-free	Radar Metric	E2E success
Rule-based controller	$15.0\%$ [8.1, 26.1]	$20.0\%$ [11.9, 31.6]	$16.7\%$ [9.3, 28.0]	$100.0\%$ [94.0, 100.0]	$31.7\%$ [21.4, 44.2]	$15.0\%$ [8.1, 26.1]
SLM w/o radar prompt + tools	$67.8\%$ [60.6, 74.2]	$96.7\%$ [92.9, 98.5]	$93.3\%$ [88.7, 96.2]	$90.6\%$ [85.4, 94.1]	$73.9\%$ [67.1, 79.7]	$52.8\%$ [45.5, 60.0]
SLM w/ radar prompt, no tools	$96.1\%$ [92.1, 98.1]	–	–	$52.8\%$ [45.5, 60.0]	–	$52.2\%$ [44.9, 59.4]
Full SLM agent	$91.1\%$ [86.1, 94.4]	$100.0\%$ [97.9, 100.0]	$98.3\%$ [95.2, 99.4]	$98.9\%$ [96.0, 99.7]	$88.3\%$ [82.7, 92.3]	$86.1\%$ [80.3, 90.4]

Table 2: Agent-level quantitative evaluation. All values reported with 95% Wilson confidence intervals.

Taper	Steer angle	Requested SLL	Achieved SLL	3-dB BW
Uniform	$0^{\circ}$	–	$-13.15$ dB	$6.348^{\circ}$
Dolph-Chebyshev	$0^{\circ}$	$-30$ dB	$-30.00$ dB	$7.966^{\circ}$
Taylor	$0^{\circ}$	$-30$ dB	$-30.06$ dB	$8.054^{\circ}$
Dolph-Chebyshev	$30^{\circ}$	$-30$ dB	$-30.00$ dB	$9.212^{\circ}$
Dolph-Chebyshev	$30^{\circ}$	$-40$ dB	$-40.00$ dB	$10.384^{\circ}$

Table 3: Sidelobe-control performance for beamformers.

Fig. 2 : Beamforming results from the physics tools. Left: DAS beampatterns with uniform and Dolph–Chebyshev tapering, illustrating the trade-off between mainlobe width and sidelobe suppression. Right: adaptive beampattern for a target at

10^{\circ}

and jammer at

-20^{\circ}

, showing a deep null at the jammer while preserving the target response.

Scenario	Selected tool	Target	Jammer(s)	Null depth	Target gain
Single jammer, training available	MVDR	$10^{\circ}$	$-20^{\circ}$	$-81.77$ dB	$0.00$ dB
Single jammer, no training	MPDR	$10^{\circ}$	$-20^{\circ}$	$-55.29$ dB	$-0.01$ dB
Two jammers	LCMV	$10^{\circ}$	$-30^{\circ},+25^{\circ}$	$-120.00$ dB	$0.00$ dB
Steering mismatch	MVDR (Loaded)	$10^{\circ}$	$-20^{\circ}$	$-85.52$ dB	$0.00$ dB

Table 4: Adaptive beamforming in jamming conditions.

$N$	$L$	SNR	True DOAs	Selected tool	Estimated DOAs / RMSE
$8$	$10$	$10$ dB	$[10^{\circ},15^{\circ}]$	ESPRIT	$[9.570^{\circ},14.876^{\circ}]$ / RMSE $=0.316^{\circ}$
$16$	$10$	$10$ dB	$[30^{\circ},32^{\circ}]$	ESPRIT	$[29.794^{\circ},32.764^{\circ}]$ / RMSE $=0.559^{\circ}$
$16$	$1$	$10$ dB	$[10^{\circ},15^{\circ}]$	OMP	$[6.500^{\circ},13.750^{\circ}]$ / RMSE $=2.628^{\circ}$
$16$	$200$	$10$ dB	$[10^{\circ},15^{\circ}]$	MUSIC	$[10.000^{\circ},15.000^{\circ}]$ / RMSE $=0.000^{\circ}$
$16$	$50$	$0$ dB	$[10^{\circ},15^{\circ}]$	ESPRIT	$[10.116^{\circ},15.284^{\circ}]$ / RMSE $=0.217^{\circ}$
$16$	$100$	$10$ dB	$[10^{\circ},15^{\circ}]$	FSS-MUSIC	$[10.000^{\circ},15.000^{\circ}]$ / RMSE $=0.000^{\circ}$

Table 5: DOA-estimation results under limited snapshots and closely spaced sources.

Scenario	Executed	Count	What’s going on
Algorithm mis-selection (wrong tool dispatched)
Low-snapshot DOA	MUSIC	8/180	Model recites instruction that MUSIC needs well-conditioned $R$ , but still dispatches estimate_doa_music . A better-engineered system prompt to embed the relationship between low snapshot and the “well-condition” of $R$ matrix may help.
Sidelobe shaping	MVDR	4/180	Phrasings like “reflections from elsewhere are negligible” are read as an adaptive-null problem; the model calls MVDR instead of selecting a low-sidelobe taper on DAS.
Sidelobe shaping	LCMV	3/180	Phrasings like “junk everywhere off-axis” are read as needing many explicit nulls; the model invents a list of null angles (e.g. [15, 30, 45, 60, 75] ) rather than picking a tapered DAS that suppresses all sidelobes.
Coherent-source DOA	ESPRIT	1/180	Coherent sources break ESPRIT’s rotational-invariance assumption; the system prompt routes the scenario to MUSIC with spatial smoothing, but the model dispatches ESPRIT anyway.
Right tool, wrong result (parameter or radar check fails)

Table 6: Failure modes observed in the benchmark.

Task	Model latency	Tool time	Total time	No. tool calls
Sidelobe control	$10.16$ s	$0.393$ s	$10.55$ s	$2$
Single-jammer MVDR	$9.78$ s	$0.357$ s	$10.14$ s	$2$
Multiple-jammer LCMV	$8.52$ s	$0.465$ s	$8.99$ s	$2$
Low-snapshot DOA	$13.21$ s	$0.240$ s	$13.45$ s	$3$

Table 7: Runtime of the local SLM-agent system.

Language Model-Based Control

Language model-based control uses language models to interpret goals, choose actions, and steer AI systems or physical processes.

See all 180 papers

Teaching Foundation Models to Read mmWave: Pose-Guided Kinematic Representation for Human Behavior Understanding
Aug 4, 2026Duo Zhang, Zhehui Yin, Zhiyun Yao +8Representation Learning Human Activity Recognition
1Peking University · 2ETH Zurich · 3Institut Polytechnique de Paris
SCORE-LM: State-Space Radar Representations with Language Models for Fault Diagnosis
Sep 30, 2026Mainak Mallick, Seung-Kyum ChoiFault Detection Fault Diagnosis
Georgia Institute of Technology, Atlanta, GA, USA
Weather-Robust Scene Semantics with Vision-Aligned 4D Radar
May 8, 2026Kali Hamilton, Christoffer HeckmanCross-Modal Learning VLM Robustness
Autonomous Robotics and Perception Group at the University of Colorado Boulder, Boulder, CO 80309, USA.

Aug 4, 2026J/K move · Enter open · S save

Teaching Foundation Models to Read mmWave: Pose-Guided Kinematic Representation for Human Behavior Understanding

Duo Zhang, Zhehui Yin, Zhiyun Yao, Haotong Qin+7

1Peking University · 2ETH Zurich · 3Institut Polytechnique de Paris

Representation Learning Human Activity Recognition Millimeter-Wave Radar mmWave Radar Sensing

Open paper AskPDF arXiv

Large language model agents need to perceive human behavior in physical environments. Millimeter-wave (mmWave) radar provides a privacy-friendly and contactless sensing modality, but radar observations are difficult to align with language. Existing radar-language methods often rely on synthetic data or lack explicit supervision for human body structure and motion. We present mmMind, a radar-language model that uses synchronized 3D pose as training-only supervision. A spatio-temporal radar encoder is pretrained to capture body configuration and motion dynamics, after which the pose head is removed so that inference requires radar alone. The learned radar representations are then aligned with an LLM for behavior captioning and spatio-temporal question answering. We also introduce mmMind-Bench, a real-world mmWave-language benchmark containing 17.9 hours of recordings from 23 participants across seven indoor environments. Experiments on captioning, question answering, and unseen-action generalization show that mmMind consistently outperforms existing radar-language baselines, while ablations confirm the importance of pose-guided pretraining.

Abstract

Language Model-Based Control

Explore similar work

Teaching Foundation Models to Read mmWave: Pose-Guided Kinematic Representation for Human Behavior Understanding

SCORE-LM: State-Space Radar Representations with Language Models for Fault Diagnosis

Weather-Robust Scene Semantics with Vision-Aligned 4D Radar