Reflection-Driven Self-Improving Language Model Agents for Adaptive RAN Slicing

Yuanhao Li; Haozhe Wang; Geyong Min; Nektarios Georgalas; Wang Miao

cs.AIMar 11, 2026

Reflection-Driven Self-Improving Language Model Agents for Adaptive RAN Slicing

Authors: Yuanhao Li, Haozhe Wang, Geyong Min, Nektarios Georgalas, Wang Miao

Organizations: Department of Computer Science, Faculty of Environment, Science and Economy, University of Exeter, Exeter, EX4 4QF, UK. · British Telecom, UK.

Abstract

Adaptive control in AI-native networks remains challenging because tasks such as dynamic Radio Access Network (RAN) slicing require persistent trade-off decisions among spectrum efficiency, service quality, and reconfiguration stability. Conventional Reinforcement Learning (RL) compresses these competing objectives into handcrafted scalar rewards, requiring exhaustive tuning for a balanced trade-off. Language model (LM) agents provide a new path toward adaptive control because they can semantically represent complex preference relations, but prompt-based adaptation struggles to retain long-horizon interaction history due to finite context limits. We propose a Reflection-Driven Self-Improvement (ReDSI) framework that replaces scalar rewards with semantic feedback for continuous policy adaptation. This framework is built upon a Small Language Model (SLM) agent for fast decision-making, coupled with a bi-perspective reflection to transform interaction history into preference data, subsequently distilling long-horizon decision experience into the SLM parameters through preference-based fine-tuning. Evaluated on a dynamic RAN slicing task, our method achieves an optimal trade-off across multiple competing objectives, outperforming standard RL baselines and existing LM-based agents by demonstrating substantially greater sample efficiency. These results highlight a viable paradigm of self-improving LM agents for reward-free multi-objective continuous control in AI-native networks.

Figures & tables

Fig. 1 : Overview of ReDSI. (a) During online policy inference, the Actor allocates PRBs based on state feedback, while its interactions with the RAN environment are recorded as trajectory

H

. The O-RAN elements illustrate a conceptual deployment mapping. (b) During offline parallel reflection, Reflector workers use the full trajectory

H

to label each decision as positive (

+

) or negative (

-

). They generate a candidate action for each negatively labeled decision, forming the reflection-labeled trajectory

H^{\prime}

. (c) During offline policy refinement, each RfR iteration generates rollout outputs

R

from original negative-example prompts, combines

H^{\prime}

and

R

into preference dataset

\mathcal{D}

, and updates the Actor through KTO. The same

H^{\prime}

is reused without additional environment interaction. Updated parameters support subsequent RfR iterations and online control.

Metric	GBR Traffic	Non-GBR Traffic
Active UE count	20	4
Transmission duration	Exp(mean = 15 sec)	Exp(mean = 15 sec)
Idle duration	Exp(mean = 15 sec)	Exp(mean = 15 sec)
Bit rate	0.5 Mb/s	2 Mb/s
Packet Size	512 bytes	512 bytes
QoS Requirement	Delay $<$ 10 ms	Delay $<$ 50 ms

TABLE I : Traffic Model Parameters

Parameter	Value
Transmission power	30 dBm
Base station antenna gain	0 dB
Base station antenna pattern	Antenna Model in 3GPP TR 38.901
Noise figure	5 dB
Carrier frequency	2120 MHz
Propagation model	Urban Propagation Loss Model

TABLE II : Radio Channel Parameters

Panel of Figure 4 — Fig. 2 : Performance comparison of RL baselines, Reflexion, and the proposed ReDSI framework across multiple objectives

Alg.	Env. Steps	Avg. SE	Reconf.	PQoS	Ep. Return
ReDSI	300	5.35 $\pm$ 0.07	21.09 $\bm{\pm}$ 6.59	8.56 $\pm$ 3.76	25702.2
Reflexion	300	5.29 $\pm$ 0.04	29.45 $\pm$ 9.26	8.63 $\pm$ 5.25	25314.69
DQN	4.8e5	5.21 $\pm$ 0.26	46.20 $\pm$ 9.60	15.91 $\pm$ 4.31	22519.1
PPO	4.8e5	3.58 $\pm$ 0.09	119.6 $\pm$ 21.08	1.10 $\bm{\pm}$ 1.04	18532.3
SAC	4.8e5	5.74 $\bm{\pm}$ 0.09	44.77 $\pm$ 16.2	59.96 $\pm$ 19.9	11704.3

TABLE III : Performance comparison (mean

\pm

standard deviation). ‘Env. Steps’ reports training/adaptation environment steps per run. ‘Ep. Return’ sums per-slice rewards over three slices and 300 steps. The best and second-best results are marked in bold and underline , respectively.

Panel of Figure 6 — Fig. 4 : KTO training dynamics of ReDSI using one reflection-labeled trajectory. (a)–(f) Chosen and rejected scores across six RfR iterations. (g) Scores gradually approaching zero, indicating diminishing returns from repeated refinement on the same trajectory.

Self-Improving Agents

Self-improving agents use experience and feedback to update their models, memories, skills, or workflows for better future task performance.

See all 169 papers

Toward Autonomous O-RAN: A Multi-Scale Agentic AI Framework for Real-Time Network Control and Management
Feb 15, 2026Hojjat Navidan, Mohammad Cheraghinia, Jaron Fontaine +5Multi-Agent Orchestration Language Model-Based Control
IDLab, Department of Information Technology at Ghent University - imec, Ghent, Belgium · Department of Electrical and Computer Engineering, Princeton University, USA
AI-Native Network Controller: A Modular Framework for Safe Agentic Control of Multi-Domain Network Infrastructure
Apr 20, 2026Merim DzaferagicAI Agent Safety
School of Computer Science and Statistics, Trinity College Dublin, Ireland
X-RIC: An Agentic RAN-Intelligent Controller Benchmark for Enhancing Reinforcement Learning
Oct 15, 2026Adhwaa Al-Chaab, Ayman Younis, Dario PompiliNetwork Slicing Agentic RL
†Department of Electrical and Computer Engineering, Rutgers University–New Brunswick, NJ, USA

Feb 15, 2026J/K move · Enter open · S save

Toward Autonomous O-RAN: A Multi-Scale Agentic AI Framework for Real-Time Network Control and Management

Hojjat Navidan, Mohammad Cheraghinia, Jaron Fontaine, Mohamed Seif+4

IDLab, Department of Information Technology at Ghent University - imec, Ghent, Belgium · Department of Electrical and Computer Engineering, Princeton University, USA

Multi-Agent Orchestration Language Model-Based Control Network Slicing Wireless Communications Open Radio Access Network Intent-Based Networking+1

Open paper AskPDF arXiv

Open Radio Access Networks (O-RAN) promise flexible 6G network access through disaggregated, software-driven components and open interfaces, but this programmability also increases operational complexity. Multiple control loops coexist across the service management layer and RAN Intelligent Controller (RIC), while independently developed control applications can interact in unintended ways. In parallel, recent advances in generative Artificial Intelligence (AI) are enabling a shift from isolated AI models toward agentic AI systems that can interpret goals, coordinate multiple models and control functions, and adapt their behavior over time. This article proposes a multi-scale agentic AI framework for O-RAN that organizes RAN intelligence as a coordinated hierarchy across the Non-Real-Time (Non-RT), Near-Real-Time (Near-RT), and Real-Time (RT) control loops: (i) A Large Language Model (LLM) agent in the Non-RT RIC translates operator intent into policies and governs model lifecycles. (ii) Small Language Model (SLM) agents in the Near-RT RIC execute low-latency optimization and can activate, tune, or disable existing control applications; and (iii) Wireless Physical-layer Foundation Model (WPFM) agents near the distributed unit provide fast inference close to the air interface. We describe how these agents cooperate through standardized O-RAN interfaces and telemetry. Using a proof-of-concept implementation built on open-source models, software, and datasets, we demonstrate the proposed agentic approach in two representative scenarios: robust operation under non-stationary conditions and intent-driven slice resource control.

Abstract

Self-Improving Agents

Explore similar work

Toward Autonomous O-RAN: A Multi-Scale Agentic AI Framework for Real-Time Network Control and Management

AI-Native Network Controller: A Modular Framework for Safe Agentic Control of Multi-Domain Network Infrastructure

X-RIC: An Agentic RAN-Intelligent Controller Benchmark for Enhancing Reinforcement Learning