AERIS: Offline Policy Improvement for Multi-UAV Integrated Sensing and Communication

Ziyuan Wang; Yifan Sui; Wei Wei; Wenjie Xin; Zekai Zhang; Xiangwang Hou; Xiao-Ping; Zhang

cs.NIAug 26, 2026

AERIS: Offline Policy Improvement for Multi-UAV Integrated Sensing and Communication

Authors: Ziyuan Wang, Yifan Sui, Wei Wei, Wenjie Xin, Zekai Zhang, Xiangwang Hou, Xiao-Ping, Zhang

Organizations: Steven · Tsinghua University, Beijing, China · Shanghai Jiao Tong University, China · The University of Hong Kong, Hong Kong ·

Abstract

Unmanned aerial vehicle (UAV)-enabled integrated sensing and communication (ISAC) is a promising 6G paradigm, but dynamic multi-UAV ISAC control must jointly balance communication quality, sensing reliability, and flight safety under stochastic mobility. Existing optimization methods often require repeated global non-convex solving, while online reinforcement learning (RL) depends on risky trial-and-error flights that may cause sensing loss or collision-risk events. This paper proposes AERIS, an offline policy improvement framework for multi-UAV ISAC. AERIS learns from fixed flight logs under centralized training and decentralized execution, so each UAV acts from local histories while training uses logged global information to assess team-level effects. We further design STAR-CRDT, an offline multi-agent RL algorithm that performs support-aware local action rectification and distills only trusted improvements into the decentralized actor. We prove an offline-support policy improvement guarantee. Experiments show that STAR-CRDT improves the main ISAC objective return by 29.3% over the strongest baseline. It further improves communication sum rate, sensing pass rate, and sensing margin by 3.4%, 4.8%, and 69.1%, while reducing collision-risk events by 54.2%. On unseen real-road maps built from OpenStreetMap data, STAR-CRDT still obtains the best return.

Figures & tables

Panel of Figure 1 — Fig. 1: Motivation diagnostics. Online trial actions can create sensing and safety risk. Generic offline baselines face an imitation and out-of-distribution (OOD) dilemma under fixed joint-action support.

Fig. 2: Dynamic multi-UAV ISAC control under local execution.

Fig. 3: Workflow of STAR-CRDT. The actor executes locally, while the centralized critic selects and filters local corrections during offline training.

Fig. 4: Training and ablation results under random mobility. The first three panels show TD3 behavior-policy training, offline policy evaluation curves, and best-checkpoint return comparison. The fourth panel compares CRDT and STAR-CRDT in joint-action OOD ratio and ISAC violation cost. STAR-CRDT reduces violations while keeping critic-corrected actions near useful support.

Fig. 5: Random-mobility offline evaluation. The compact five-panel row reports return, communication sum rate, sensing margin, sensing pass rate, and collision-risk count. STAR-CRDT obtains the best overall tradeoff.

Fig. 6: System-scale evaluation under random mobility. The compact five-panel row follows Fig. 5 and reports return, communication sum rate, sensing margin, sensing pass rate, and collision-risk count. STAR-CRDT remains strongest when the trained shared actor is evaluated with more UAVs.

System parameters
Service area $L_{x}\times L_{y}$	$500\times 500$ m 2
Episode horizon $T$	200 slots
Main topology $(M,K,L)$	$(3,6,3)$
Scale sweep $M$	3 to 6 UAVs
UAV altitude $[z_{\min},z_{\max}]$	$[100,150]$ m
UAV speed $V_{\max}$	5 m/slot

TABLE I: Key Parameters

Scene	Metric	$M=3$									$M=6$
		STAR	TD3+BC	CRDT	DT	CRR	OMIGA	BC	TD3	OMAR	STAR	TD3+BC	CRDT	DT	CRR	OMIGA	BC	TD3	OMAR
S1	Ret.	648.3	570.9	576.8	512.3	516.5	384.9	395.8	410.0	-675.0	1288.1	942.3	272.7	372.1	329.4	387.6	319.9	351.8	-1569.2
	Comm.	3.63	3.27	3.37	3.29	3.49	3.25	3.29	3.24	3.37	6.76	5.74	5.65	5.55	5.82	5.74	5.63	5.64	5.37
	Pass	0.707	0.715	0.707	0.689	0.685	0.661	0.659	0.668	0.310	0.753	0.749	0.677	0.681	0.678	0.673	0.674	0.676	0.396
	Margin	0.0366	0.0341	0.0267	0.0237	0.0213	0.0194	0.0211	0.0181	-0.0574	0.0572	0.0585	0.0311	0.0351	0.0341	0.0339	0.0316	0.0318	-0.0401
	Coll. risk	0.028	0.063	0.073	0.058	0.080	0.126	0.118	0.116	0.106	0.333	0.424	0.766	0.655	0.742	0.632	0.700	0.677	0.878

TABLE II: Zero-shot road-map deployment evaluation. Metrics are shown by row for each road scene. Best values are bold red, second-best values are underlined bold blue, and lower is better only for Coll. risk.

Fig. 7: Representative STAR-CRDT rollout under road-map mobility. The left and right panels show top-down and 3-D views, respectively.

Integrated Sensing and Communication

Integrated sensing and communication combines wireless data links with radar-like perception using shared signals, spectrum, and hardware.

See all 47 papers

CRB-Driven Beamforming and Trajectory Optimization for UAV-assisted ISAC System
Jul 21, 2026Yi Yang, Qianqian Zhang, Huaxia WangTrajectory Optimization Integrated Sensing and Communication
Department of Electrical and Computer Engineering, Rowan University, NJ, USA
An Embodied Multiagent Framework Based on Token Communications for Cooperative ISAC
Oct 8, 2026Jiahe Guo, Jun Du, Chunxiao Jiang +1Integrated Sensing and Communication Multi-Agent Communication
Department of Electronic Engineering, Tsinghua University, Beijing 100084, China · State Key Laboratory of Space Network and Communications, Tsinghua University, Beijing 100084, China · Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing 100084, China
JCAS-MARL: Joint Communication and Sensing UAV Networks via Resource-Constrained Multi-Agent Reinforcement Learning
Mar 13, 2026Islam Guven, Mehmet ParlakIntegrated Sensing and Communication Wireless Resource Allocation
ICTEAM, Université catholique de Louvain Ottignies-Louvain-la-Neuve, Belgium

Jul 21, 2026J/K move · Enter open · S save

CRB-Driven Beamforming and Trajectory Optimization for UAV-assisted ISAC System

Yi Yang, Qianqian Zhang, Huaxia Wang

Department of Electrical and Computer Engineering, Rowan University, NJ, USA

Trajectory Optimization Integrated Sensing and Communication Wireless Communications UAV Trajectory Optimization

Open paper AskPDF arXiv

In this paper, we study an unmanned aerial vehicle (UAV)-assisted integrated sensing and communication (ISAC) system, where a UAV enhances the sensing capability of a base station (BS) towards a target while ensuring reliable communication towards a downlink user. This architecture is practically attractive for future wireless networks due to the UAV's controllable mobility and adaptive sensing coverage in wireless environments. The sensing performance is characterized by the average Cramér-Rao bound (CRB), which quantifies the minimum variance of the unbiased angle-of-arrival estimation. To enhance the sensing performance, the UAV trajectory and beamforming parameters are jointly optimized under power and mobility constraints, while satisfying communication requirements to the downlink user. To address the resulting non-convex problem, we employ null-space projection for beamforming design and adopt deep reinforcement learning for the trajectory optimization over a discrete-time scale. In each time slot, beamforming is optimized based on the channel state information to improve CRB performance while mitigating interference between the BS and the communication user. Simulation results demonstrate that the proposed method significantly reduces the time-averaged CRB by over 10%, compared with the ISAC system without UAV assistance, and also achieves a higher sensing accuracy than both the fixed-UAV-trajectory and the maximum-ratio-transmission-based beamforming benchmarks.

Abstract

Integrated Sensing and Communication

Explore similar work

CRB-Driven Beamforming and Trajectory Optimization for UAV-assisted ISAC System

An Embodied Multiagent Framework Based on Token Communications for Cooperative ISAC

JCAS-MARL: Joint Communication and Sensing UAV Networks via Resource-Constrained Multi-Agent Reinforcement Learning