Aerial Multi-Functional RIS in Fluid Antennas-Aided Full-Duplex Networks: A Self-Optimized Hybrid Deep Reinforcement Learning Approach

Li-Hsiang Shen; Yu-Quan Zheng

cs.ITApr 15, 2026

Aerial Multi-Functional RIS in Fluid Antennas-Aided Full-Duplex Networks: A Self-Optimized Hybrid Deep Reinforcement Learning Approach

Authors: Li-Hsiang Shen, Yu-Quan Zheng

Organizations: Department of Communication Engineering, National Central University, Taoyuan 320317, Taiwan

Abstract

To address high data traffic demands of sixth-generation (6G) networks, this paper proposes a novel architecture that integrates autonomous aerial vehicles (AAVs) and multi-functional reconfigurable intelligent surfaces (MF-RISs) as AM-RIS in fluid antenna (FA)-assisted full-duplex (FD) networks. The AM-RIS provides hybrid functionalities, including signal reflection, amplification, and energy harvesting (EH), potentially improving both signal coverage and sustainability. Meanwhile, FA facilitates fine-grained spatial adaptability at FD-enabled base station (BS), which complements residual self-interference (SI) suppression. We aim at maximizing the overall energy efficiency (EE) by jointly optimizing transmit DL beamforming at BS, UL user power, configuration of AM-RIS, and positions of the FA and AM-RIS. Owing to the hybrid continuous-discrete parameters and high dimensionality of the intractable problem, we have conceived a self-optimized multi-agent hybrid deep reinforcement learning (DRL) framework (SOHRL), which integrates multi-agent deep Q-networks (DQN) and multi-agent proximal policy optimization (PPO), respectively handling discrete and continuous actions. To enhance self-adaptability, an attention-driven state representation and meta-level hyperparameter optimization are incorporated, enabling multi-agents to autonomously adjust learning hyperparameters. Simulation results validate the effectiveness of the proposed AM-RIS-enabled FA-aided FD networks empowered by SOHRL algorithm. The results reveal that SOHRL outperforms benchmarks of the case without attention mechanism and conventional hybrid/multi-agent/standalone DRL. Moreover, AM-RIS in FD achieves the highest EE compared to half-duplex, conventional rigid antenna arrays, partial EH, and conventional RIS without amplification, highlighting its potential as a compelling solution for EE-aware wireless networks.

Figures & tables

Fig. 1: The proposed architecture of multi-AM-RISs for the FA-aided FD network, where FD-BS serves HD-DL/HD-UL users. The AM-RIS is implemented on AAV via MF-RIS. Note that simultaneous UL/DL transmissions occur in the cell. The DL users receive inter-DL and AM-RIS-induced reflected UL user interferences, whereas the BS Rx receives direct and AM-RIS-induced SI from DL Tx antennas. Both DL users and BS Rx receive AM-RIS amplified noise.

Fig. 2: The proposed SOHRL algorithm integrating MADQN and MAPPO with attention-driven states and self-optimization scheme.

Parameter	Value
Network parameters	$I=4$ , $M=32$ , $K_{\rm D}=K_{\rm U}=4$ , $N_{\rm T}=N_{\rm R}=16$ , $h_{0}=-20$ dB, $\kappa_{0}=2.2$ , $\beta_{0}=3$ dB, $P_{\rm BS}=40$ W, $P_{\max}=100$ W, $P_{\rm UL}=1$ W, $\sigma_{k}^{2}=\bar{\sigma}^{2}=\bar{\sigma}_{k}^{2}=-80$ dBm, $R_{\rm th}^{\rm DL}=R_{\rm th}^{\rm UL}=1$ bps/Hz,
AAV parameters [ 41 , 42 ]	$W_{i}^{\rm AAV}=20$ Newton, $P_{\rm BP}=79$ W, $P_{\rm IP}=88$ W, $z_{i}\in[50,500]$ m, $d_{\rm min}=6$ m, $v_{\rm max}=15$ m/s, $a_{\rm max}=1$ m/s 2 , $\Omega_{B}=300$ rad/s, $\Omega_{R}=0.4$ m, $v_{R}=4$ m/s, $\zeta_{D}=0.3$ , $\zeta_{A}=1.225$ kg/m 3 , $\zeta_{S}=0.05$ , $\zeta_{R}=0.5$ m 2 , $\zeta_{C}=0.1$ , $b_{1}=12.08$ , $b_{2}=0.11$ , $d_{{\rm th},1}=0.05$ m, $d_{{\rm th},2}=0.1$ m.
MF-RIS parameters [ 9 ]	$W_{i}^{\rm MRIS}=10$ Newton, $P_{\rm ph}=5$ mW, $P_{\rm hc}=50$ mW, $P_{\rm am}=100$ mW, $\beta_{\max}=3$ , $\varsigma_{\rm PA}=1.1$ , $Z_{1}=24$ mW, $c_{1}=150$ , $c_{2}=0.014$
DRL parameters	$\varpi=5$ , $\|\mathcal{D}\|=128$ , $U=64$ , $\varsigma_{\rm th}\in[0.05,0.4]$ , $\gamma_{\text{d}}\in[0.1,0.99]$ , $\eta_{\text{d}}\in[10^{-4},10^{-2}]$ , $\tau_{Q}\in[0.7,0.95]$ , $\gamma_{\text{c}}\in[0.1,0.99]$ , $\epsilon\in[0.1,0.3]$ , $\eta_{\text{c},1}\in[10^{-4},10^{-2}]$ , $\eta_{\text{c},2}\in[10^{-4},10^{-2}]$ , $\rho_{1}=\rho_{2}=\rho_{3}=5\times 10^{-4}$ , $\rho_{4}=\rho_{5}=\rho_{6}=3\times 10^{-5}$ , $\lambda_{c}=3\times 10^{-4}$ , $\varepsilon_{\xi}=10^{-6}$ , $N_{H}=4$ , $T_{\xi}=5$ , $d_{K}=64$ , $T_{up}=10$

TABLE I: Simulation parameters

Fig. 3: Convergence of EE performance of proposed SOHRL scheme compared to the benchmarks of SOHRL without attention-drive state, hybrid DRL, MAPPO, centralized PPO and DQN methods.

Fig. 4: Convergence of EE performance of SOHRL with different portions of hyperparameters to be optimized.

Fig. 5: Convergence of EE performance of SOHRL with different numbers of the multi-head attention $N_{H}$ and dropout rates $D_{r}$ in deep neural networks.

Panel of Figure 7 — Fig. 6: Convergence of different exploration methods in MADQN in terms of (a) EE and its corresponding (b) exploration rate.

Panel of Figure 8 — Fig. 7: EE performance with different deployment heights of AM-RISs and various numbers of users w.r.t. (a) EE, (b) rate, and (c) power consumption.

Fig. 8: EE performance under different numbers of AM-RISs and the maximum velocity values of AAVs.

Fig. 9: Comparison the Energy Efficency performance with different numbers of MF-RIS elements.

Fig. 10: EE performance of different configurations of optimized FA, partial FA ( $50\%$ ) and conventional rigid array in FD and HD-based DL/UL under various numbers of users.

Fig. 11: Comparison of four schemes under different fluid-antenna array sizes

Fig. 12: EE performance of different FA array sizes of $L\in\{0.5,1\}$ m and resolutions of $\{0.02,0.03,0.05\}$ m.

Fig. 13: Comparison of four schemes under different fluid-antenna array sizes

Algorithm	Computational Complexity
C-DQN	$\mathcal{O}\left(N_{B}N_{L}N_{W}^{2}+N_{B}N_{W}\cdot\|\mathcal{A}^{p}_{\mathcal{Q}}\|\right)$
C-PPO	$\mathcal{O}\left(N_{B}N_{L}N_{W}^{2}+N_{B}N_{W}\cdot\|\mathcal{A}^{p}_{\mathcal{C}}\|\right)$
MAPPO	$\mathcal{O}\left(N_{B}N_{L}N_{W}^{2}+N_{B}N_{W}\cdot\|\mathcal{A}^{p}_{i,\mathcal{C}}\|\right)$
Hybrid DRL	$\mathcal{O}\Big(2N_{B}N_{L}N_{W}^{2}+N_{B}N_{W}\cdot(\|\mathcal{A}^{\text{c}}\|+\|\mathcal{A}^{\text{d}}\|)\Big)$
Proposed SOHRL w/o Hyperparameter Optimization	$\mathcal{O}\Big(2N_{B}N_{L}N_{W}^{2}+N_{B}N_{W}\cdot(\|\mathcal{A}^{\text{c}}_{i}\|+\|\mathcal{A}^{\text{d}}_{i}\|)$ $+N_{B}\cdot(4\|\mathcal{S}_{i}\|N_{W}^{2}N_{H})\Big)$
Proposed SOHRL w/o Attention	$\mathcal{O}\Big(3N_{B}N_{L}N_{W}^{2}+N_{B}N_{W}\cdot(\|\mathcal{A}^{\text{c}}_{i}\|+\|\mathcal{A}^{\text{d}}_{i}\|+\|\mathcal{G}\|)\Big)$

TABLE II: Computational Complexity

Hybrid Action Space RL

See all 8 papers

Mar 27, 2026

Trustworthy AI-Driven Dynamic Hybrid RIS: Joint Optimization and Reward Poisoning-Resilient Control in Cognitive MISO Networks

Cognitive radio networks (CRNs) are a key mechanism for alleviating spectrum scarcity by enabling secondary users (SUs) to opportunistically access licensed frequency bands without harmful interference to primary users (PUs). To address unreliable direct SU links and energy constraints common in next-generation wireless networks, this work introduces an adaptive, energy-aware hybrid reconfigurable intelligent surface (RIS) for underlay multiple-input single-output (MISO) CRNs. Distinct from prior approaches relying on static RIS architectures, our proposed RIS dynamically alternates between passive and active operation modes in real time according to harvested energy availability. We also model our scenario under practical hardware impairments and cascaded fading channels. We formulate and solve a joint transmit beamforming and RIS phase optimization problem via the soft actor-critic (SAC) deep reinforcement learning (DRL) method, leveraging its robustness in continuous and highly dynamic environments. Notably, we conduct the first systematic study of reward poisoning attacks on DRL agents in RIS-enhanced CRNs, and propose a lightweight, real-time defense based on reward clipping and statistical anomaly filtering. Numerical results demonstrate that the SAC-based approach consistently outperforms established DRL baselines, and that the dynamic hybrid RIS strikes a superior trade-off between throughput and energy consumption compared to fully passive and fully active alternatives. We further show the effectiveness of our defense in maintaining SU performance even under adversarial conditions. Our results advance the practical and secure deployment of RIS-assisted CRNs, and highlight crucial design insights for energy-constrained wireless systems.

Deemah H. Tashman, Soumaya Cherkaoui

LINCS Laboratory, Department of Computer and Software Engineering, Polytechnique Montréal, Montréal, QC, Canada, H3T 1J4

Wireless Communications Reconfigurable Intelligent Surfaces Wireless Resource Allocation Adversarial Attacks RL Control

Ask PDF

May 31, 2026

Multi-Agent DRL for QoS and Energy Optimization in RIS-Enabled Open-RAN Industrial 6G TN/NTN Networks

Industrial 6G networks require ultra-reliable, low-latency, and energy-efficient connectivity in dynamic and blockage-prone environments, where conventional terrestrial deployments often fail to ensure stable coverage. Hence, in this paper, we propose a RIS-enabled Open-RAN framework for integrated terrestrial/non-terrestrial (TN/NTN) industrial 6G networks, in which UAVs-mounted reconfigurable intelligent surfaces (RISs) cooperate with ground radio units and a high-altitude platform (HAP) to enhance connectivity for dense industrial IoT devices. Owing to the high dimensionality and strong coupling among decision variables, conventional optimization techniques become computationally intractable. To overcome this limitation, the joint optimization problem of data rates, latency, and energy consumptions is formulated as a decentralized partially observable Markov decision process (Dec-POMDP) and solved using a multi-agent deep reinforcement learning framework. Simulation results show improvements of up to 75% in data rate, 25% latency reduction, and 16% energy savings compared with state-of-the-art learning-based and non-RIS baselines, demonstrating the effectiveness of RIS-assisted Open-RAN intelligence for industrial 6G networks.

Marwan Dhuheir, Thang X. Vu, Symeon Chatzinotas

The Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, Luxembourg.

6G Networks Reconfigurable Intelligent Surfaces Wireless Communications Open Radio Access Network Wireless Resource Allocation Multi-Objective Optimization

Ask PDF

Aug 28, 2026

Low-Altitude Fluid Antenna Network with Multi-Agent Reinforcement Learning

Low-altitude wireless networks (LAWNs) integrate terrestrial and aerial platforms to provide ubiquitous communication, sensing, and localization services for unmanned aerial vehicles (UAVs) and electric vertical takeoff and landing (eVTOL) aircraft. However, dynamic air-ground and air-air channels, abrupt blockages, and heterogeneous interference hinder the realization of this goal. Nevertheless, fluid antenna (FA), a cutting-edge multiple-input multiple-output (MIMO) technique, overcomes these challenges by reconfiguring antenna positions to unlock additional spatial degrees-of-freedom. In this paper, towards bringing low-altitude FA networks into reality, we study the fast and high-performance FA reconfiguration for low-altitude FA networks with multi-agent reinforcement learning (MARL). Specifically, we present an electromagnetic digital twin (EM-DT)-assisted MARL framework. To fill the sim-to-real gap, we introduce a two-stage transfer learning framework. Our case study shows that joint FA positions and beamforming optimization can enhance the system sum-rate by 118.5%, compared to the fixed position baseline. This gain comes from the dynamic millisecond timescale reconfiguration of FA arrays and the adaptive steering of beams toward aerial users with mobility.

Tong Zhang, Yanfei Su, Shuai Wang +3

Guangdong Provincial Key Laboratory of Aerospace Communication and Networking Technology, Harbin Institute of Technology, Shenzhen, 518055, China · Harbin Institute of Technology, Shenzhen, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China +3

Wireless Communications Wireless Resource Allocation Multi-Agent Reinforcement Learning

Ask PDF

Abstract

Hybrid Action Space RL

Explore similar work

Trustworthy AI-Driven Dynamic Hybrid RIS: Joint Optimization and Reward Poisoning-Resilient Control in Cognitive MISO Networks

Multi-Agent DRL for QoS and Energy Optimization in RIS-Enabled Open-RAN Industrial 6G TN/NTN Networks

Low-Altitude Fluid Antenna Network with Multi-Agent Reinforcement Learning