Aerial Multi-Functional RIS in Fluid Antennas-Aided Full-Duplex Networks: A Self-Optimized Hybrid Deep Reinforcement Learning Approach

Li-Hsiang Shen; Yu-Quan Zheng

cs.ITApr 15, 2026

Aerial Multi-Functional RIS in Fluid Antennas-Aided Full-Duplex Networks: A Self-Optimized Hybrid Deep Reinforcement Learning Approach

Authors: Li-Hsiang Shen, Yu-Quan Zheng

Organizations: Department of Communication Engineering, National Central University, Taoyuan 320317, Taiwan

Abstract

To address high data traffic demands of sixth-generation (6G) networks, this paper proposes a novel architecture that integrates autonomous aerial vehicles (AAVs) and multi-functional reconfigurable intelligent surfaces (MF-RISs) as AM-RIS in fluid antenna (FA)-assisted full-duplex (FD) networks. The AM-RIS provides hybrid functionalities, including signal reflection, amplification, and energy harvesting (EH), potentially improving both signal coverage and sustainability. Meanwhile, FA facilitates fine-grained spatial adaptability at FD-enabled base station (BS), which complements residual self-interference (SI) suppression. We aim at maximizing the overall energy efficiency (EE) by jointly optimizing transmit DL beamforming at BS, UL user power, configuration of AM-RIS, and positions of the FA and AM-RIS. Owing to the hybrid continuous-discrete parameters and high dimensionality of the intractable problem, we have conceived a self-optimized multi-agent hybrid deep reinforcement learning (DRL) framework (SOHRL), which integrates multi-agent deep Q-networks (DQN) and multi-agent proximal policy optimization (PPO), respectively handling discrete and continuous actions. To enhance self-adaptability, an attention-driven state representation and meta-level hyperparameter optimization are incorporated, enabling multi-agents to autonomously adjust learning hyperparameters. Simulation results validate the effectiveness of the proposed AM-RIS-enabled FA-aided FD networks empowered by SOHRL algorithm. The results reveal that SOHRL outperforms benchmarks of the case without attention mechanism and conventional hybrid/multi-agent/standalone DRL. Moreover, AM-RIS in FD achieves the highest EE compared to half-duplex, conventional rigid antenna arrays, partial EH, and conventional RIS without amplification, highlighting its potential as a compelling solution for EE-aware wireless networks.

Figures & tables

Fig. 1: The proposed architecture of multi-AM-RISs for the FA-aided FD network, where FD-BS serves HD-DL/HD-UL users. The AM-RIS is implemented on AAV via MF-RIS. Note that simultaneous UL/DL transmissions occur in the cell. The DL users receive inter-DL and AM-RIS-induced reflected UL user interferences, whereas the BS Rx receives direct and AM-RIS-induced SI from DL Tx antennas. Both DL users and BS Rx receive AM-RIS amplified noise.

Fig. 2: The proposed SOHRL algorithm integrating MADQN and MAPPO with attention-driven states and self-optimization scheme.

Parameter	Value
Network parameters	$I=4$ , $M=32$ , $K_{\rm D}=K_{\rm U}=4$ , $N_{\rm T}=N_{\rm R}=16$ , $h_{0}=-20$ dB, $\kappa_{0}=2.2$ , $\beta_{0}=3$ dB, $P_{\rm BS}=40$ W, $P_{\max}=100$ W, $P_{\rm UL}=1$ W, $\sigma_{k}^{2}=\bar{\sigma}^{2}=\bar{\sigma}_{k}^{2}=-80$ dBm, $R_{\rm th}^{\rm DL}=R_{\rm th}^{\rm UL}=1$ bps/Hz,
AAV parameters [ 41 , 42 ]	$W_{i}^{\rm AAV}=20$ Newton, $P_{\rm BP}=79$ W, $P_{\rm IP}=88$ W, $z_{i}\in[50,500]$ m, $d_{\rm min}=6$ m, $v_{\rm max}=15$ m/s, $a_{\rm max}=1$ m/s 2 , $\Omega_{B}=300$ rad/s, $\Omega_{R}=0.4$ m, $v_{R}=4$ m/s, $\zeta_{D}=0.3$ , $\zeta_{A}=1.225$ kg/m 3 , $\zeta_{S}=0.05$ , $\zeta_{R}=0.5$ m 2 , $\zeta_{C}=0.1$ , $b_{1}=12.08$ , $b_{2}=0.11$ , $d_{{\rm th},1}=0.05$ m, $d_{{\rm th},2}=0.1$ m.
MF-RIS parameters [ 9 ]	$W_{i}^{\rm MRIS}=10$ Newton, $P_{\rm ph}=5$ mW, $P_{\rm hc}=50$ mW, $P_{\rm am}=100$ mW, $\beta_{\max}=3$ , $\varsigma_{\rm PA}=1.1$ , $Z_{1}=24$ mW, $c_{1}=150$ , $c_{2}=0.014$
DRL parameters	$\varpi=5$ , $\|\mathcal{D}\|=128$ , $U=64$ , $\varsigma_{\rm th}\in[0.05,0.4]$ , $\gamma_{\text{d}}\in[0.1,0.99]$ , $\eta_{\text{d}}\in[10^{-4},10^{-2}]$ , $\tau_{Q}\in[0.7,0.95]$ , $\gamma_{\text{c}}\in[0.1,0.99]$ , $\epsilon\in[0.1,0.3]$ , $\eta_{\text{c},1}\in[10^{-4},10^{-2}]$ , $\eta_{\text{c},2}\in[10^{-4},10^{-2}]$ , $\rho_{1}=\rho_{2}=\rho_{3}=5\times 10^{-4}$ , $\rho_{4}=\rho_{5}=\rho_{6}=3\times 10^{-5}$ , $\lambda_{c}=3\times 10^{-4}$ , $\varepsilon_{\xi}=10^{-6}$ , $N_{H}=4$ , $T_{\xi}=5$ , $d_{K}=64$ , $T_{up}=10$

TABLE I: Simulation parameters

Fig. 3: Convergence of EE performance of proposed SOHRL scheme compared to the benchmarks of SOHRL without attention-drive state, hybrid DRL, MAPPO, centralized PPO and DQN methods.

Fig. 4: Convergence of EE performance of SOHRL with different portions of hyperparameters to be optimized.

Fig. 5: Convergence of EE performance of SOHRL with different numbers of the multi-head attention $N_{H}$ and dropout rates $D_{r}$ in deep neural networks.

Panel of Figure 7 — Fig. 6: Convergence of different exploration methods in MADQN in terms of (a) EE and its corresponding (b) exploration rate.

Panel of Figure 8 — Fig. 7: EE performance with different deployment heights of AM-RISs and various numbers of users w.r.t. (a) EE, (b) rate, and (c) power consumption.

Fig. 8: EE performance under different numbers of AM-RISs and the maximum velocity values of AAVs.

Fig. 9: Comparison the Energy Efficency performance with different numbers of MF-RIS elements.

Fig. 10: EE performance of different configurations of optimized FA, partial FA ( $50\%$ ) and conventional rigid array in FD and HD-based DL/UL under various numbers of users.

Fig. 11: Comparison of four schemes under different fluid-antenna array sizes

Fig. 12: EE performance of different FA array sizes of $L\in\{0.5,1\}$ m and resolutions of $\{0.02,0.03,0.05\}$ m.

Fig. 13: Comparison of four schemes under different fluid-antenna array sizes

Algorithm	Computational Complexity
C-DQN	$\mathcal{O}\left(N_{B}N_{L}N_{W}^{2}+N_{B}N_{W}\cdot\|\mathcal{A}^{p}_{\mathcal{Q}}\|\right)$
C-PPO	$\mathcal{O}\left(N_{B}N_{L}N_{W}^{2}+N_{B}N_{W}\cdot\|\mathcal{A}^{p}_{\mathcal{C}}\|\right)$
MAPPO	$\mathcal{O}\left(N_{B}N_{L}N_{W}^{2}+N_{B}N_{W}\cdot\|\mathcal{A}^{p}_{i,\mathcal{C}}\|\right)$
Hybrid DRL	$\mathcal{O}\Big(2N_{B}N_{L}N_{W}^{2}+N_{B}N_{W}\cdot(\|\mathcal{A}^{\text{c}}\|+\|\mathcal{A}^{\text{d}}\|)\Big)$
Proposed SOHRL w/o Hyperparameter Optimization	$\mathcal{O}\Big(2N_{B}N_{L}N_{W}^{2}+N_{B}N_{W}\cdot(\|\mathcal{A}^{\text{c}}_{i}\|+\|\mathcal{A}^{\text{d}}_{i}\|)$ $+N_{B}\cdot(4\|\mathcal{S}_{i}\|N_{W}^{2}N_{H})\Big)$
Proposed SOHRL w/o Attention	$\mathcal{O}\Big(3N_{B}N_{L}N_{W}^{2}+N_{B}N_{W}\cdot(\|\mathcal{A}^{\text{c}}_{i}\|+\|\mathcal{A}^{\text{d}}_{i}\|+\|\mathcal{G}\|)\Big)$

TABLE II: Computational Complexity

Hybrid Action Space RL

See all 8 papers

Trustworthy AI-Driven Dynamic Hybrid RIS: Joint Optimization and Reward Poisoning-Resilient Control in Cognitive MISO Networks
Mar 27, 2026Deemah H. Tashman, Soumaya CherkaouiWireless Communications Reconfigurable Intelligent Surfaces
LINCS Laboratory, Department of Computer and Software Engineering, Polytechnique Montréal, Montréal, QC, Canada, H3T 1J4
Multi-Agent DRL for QoS and Energy Optimization in RIS-Enabled Open-RAN Industrial 6G TN/NTN Networks
May 31, 2026Marwan Dhuheir, Thang X. Vu, Symeon Chatzinotas6G Networks Reconfigurable Intelligent Surfaces
The Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, Luxembourg.
Low-Altitude Fluid Antenna Network with Multi-Agent Reinforcement Learning
Aug 28, 2026Tong Zhang, Yanfei Su, Shuai Wang +3Wireless Communications Wireless Resource Allocation
Guangdong Provincial Key Laboratory of Aerospace Communication and Networking Technology, Harbin Institute of Technology, Shenzhen, 518055, China · Harbin Institute of Technology, Shenzhen, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China +3

Mar 27, 2026J/K move · Enter open · S save

Trustworthy AI-Driven Dynamic Hybrid RIS: Joint Optimization and Reward Poisoning-Resilient Control in Cognitive MISO Networks

Deemah H. Tashman, Soumaya Cherkaoui

LINCS Laboratory, Department of Computer and Software Engineering, Polytechnique Montréal, Montréal, QC, Canada, H3T 1J4

Wireless Communications Reconfigurable Intelligent Surfaces Wireless Resource Allocation Adversarial Attacks RL Control

Open paper AskPDF arXiv

Cognitive radio networks (CRNs) are a key mechanism for alleviating spectrum scarcity by enabling secondary users (SUs) to opportunistically access licensed frequency bands without harmful interference to primary users (PUs). To address unreliable direct SU links and energy constraints common in next-generation wireless networks, this work introduces an adaptive, energy-aware hybrid reconfigurable intelligent surface (RIS) for underlay multiple-input single-output (MISO) CRNs. Distinct from prior approaches relying on static RIS architectures, our proposed RIS dynamically alternates between passive and active operation modes in real time according to harvested energy availability. We also model our scenario under practical hardware impairments and cascaded fading channels. We formulate and solve a joint transmit beamforming and RIS phase optimization problem via the soft actor-critic (SAC) deep reinforcement learning (DRL) method, leveraging its robustness in continuous and highly dynamic environments. Notably, we conduct the first systematic study of reward poisoning attacks on DRL agents in RIS-enhanced CRNs, and propose a lightweight, real-time defense based on reward clipping and statistical anomaly filtering. Numerical results demonstrate that the SAC-based approach consistently outperforms established DRL baselines, and that the dynamic hybrid RIS strikes a superior trade-off between throughput and energy consumption compared to fully passive and fully active alternatives. We further show the effectiveness of our defense in maintaining SU performance even under adversarial conditions. Our results advance the practical and secure deployment of RIS-assisted CRNs, and highlight crucial design insights for energy-constrained wireless systems.

Abstract

Hybrid Action Space RL

Explore similar work

Trustworthy AI-Driven Dynamic Hybrid RIS: Joint Optimization and Reward Poisoning-Resilient Control in Cognitive MISO Networks

Multi-Agent DRL for QoS and Energy Optimization in RIS-Enabled Open-RAN Industrial 6G TN/NTN Networks

Low-Altitude Fluid Antenna Network with Multi-Agent Reinforcement Learning