RAGAM-AI: A Hybrid Reinforcement Learning based Agentic approach powered by Generative AI for Edge Resource Management
Organizations: Iquadrat Informatica S.L., Barcelona, Spain · Universitat Politècnica de Catalunya, Barcelona, Spain · Department of Informatics of the University of Western Macedonia, Greece · University of Patras, Patras, Greece ·
Abstract
Resource management for distributed edge networks hosting different services has been handled efficiently using Reinforcement learning (RL). Reward generalization in complex workflows is still an open issue within the RL realm. Therefore, this work proposes RAGAM-AI, a hybrid framework that combines the capabilities of RL algorithms and Generative-AI (Gen-AI) for powering an Agentic AI workflow that optimizes resource allocation at the Multi-access Edge Computing (MEC) units. This framework further resolves the reward generalization issue by utilizing Large Language Model (LLM) for feedback. A Multi- Agent Soft-Actor Critic (MASAC) using MathΣtral as a reward oracle is implemented for managing resource allocations at the MEC. RAGAM-AI interprets system state information and evaluates multi-objective trade-offs guiding the resource orchestration loop over distributed edge server agents. The complexity of the framework is further brought down by incorporating surrogate reward model. The overall performance is evaluated against the single agent SAC, REINFORCE and random policies for benchmarking. On using RAGAM-AI, lower allocation deviations for CPU, memory and bandwidth resources are reported while minimizing the latency and data rate costs. A scalable AI as a feedback (AIAF) solution optimizing network resources with significant performance improvement is thus presented.
Figures & tables
| Reference | Application | Highlights | Metric evaluated | |||
|---|---|---|---|---|---|---|
| Resource allocation | Task offloading | Expandability | Heterogeneity | |||
| [ 16 ] | Network slicing | ● | ○ | Part I | Part II | Processing ratio |
| [ 28 ] | IoT devices | ● | ● | ○ | ○ | Task completion time, energy consumption |
| [ 29 ] | UAV, D2D | Part III | Part IV | ○ | Part V | Task offloading ratio, computational latency |
| [ 30 ] | Distributed systems | ● | ● | Part VI | ○ | System gain, delay, energy |
| [ 31 ] | Ultra-dense low latency networks | Part VII | ○ | Part VIII | ○ | Resource utilization, load reduction, energy |
| Reference | Identified limitations | Improvement using RAGAM-AI |
|---|---|---|
| [ 28 ] - [ 32 ] | • Scalability concerns • Static or semi-static network assumptions • Security and privacy Concerns • High computational complexity • Parameter sensitivity | • Model-free optimization • Improved handling of high-dimensional spaces • Automatic policy learning |
| [ 16 ] , [ 33 ] - [ 40 ] | • Information exchange during centralized training • Training instabilities from hierarchical layers • Hybrid-action spaces • Synchronization issues • High algorithmic complexity • Deployment and Integration Challenges | • Sequential decision-making • Adaptation to changing environment • Elimination of labelled datasets • Focus on long-term objectives |
| [ 41 ] - [ 44 ] | • Model maintenance and updates • Computational overhead at the edge • Stationarity considerations | • Distributed learning • Improved reasoning • Handling complex state information • Avoiding reward bias • Including AI as feedback |
| Notation | Description |
|---|---|
| MEC Servers | |
| CPU request | |
| Memory request | |
| Bandwidth request | |
| Latency | |
| Latency limit |
| Parameter | |||||
|---|---|---|---|---|---|
| Value | 20 MHz | 7dB | 46 dBm | 125 dB | 5.5 dBi |
| Setting | Parameter | Value |
| MASAC | Actor learning rate | 0.001 |
| Critic learning rate | 0.0001 | |
| Hidden layers | 8 | |
| Neurons per layer | 64 | |
| Math tral | Temperature | 0.2 |
| Token | 60 |
| Data | |||
| Input States | 50 | ||
| Query runs | 5 | ||
| Reward stability | |||
| Mean | Standard deviation | Confidence | |
| State 1 | 0.88 | 0.06 | 0.07 |
| State 2 | 0.68 | 0.08 | 0.1 |
| Metric | SAC | REINFORCE | Random |
|---|---|---|---|
| CPU deviation | 17.33% | 27.29% | 47.36% |
| Memory deviation | 16.38% | 17.50% | 56.09% |
| Bandwidth deviation | 19.27% | 23.01% | 56.52% |
| Latency cost | 2e-4 | 2.4e-4 | 1.42e-3 |
| Data rate cost | 2e+6 | 3e+6 | 7e+6 |
Edge Computing
Edge computing places computation and AI near data sources to reduce latency, bandwidth use, and reliance on centralized clouds.
See all 215 papers