NetRL: Powering Network Decisions via NetGRPO-Driven Reinforcement Learning

Lingfeng Peng; Xiaohui Xie; Yang Bao; Yong Cui

cs.NIOct 12, 2026

NetRL: Powering Network Decisions via NetGRPO-Driven Reinforcement Learning

Authors: Lingfeng Peng, Xiaohui Xie, Yang Bao, Yong Cui

Organizations: Tsinghua University

Abstract

Network decision-making tasks, including adaptive bitrate streaming, load balancing, and congestion control, require policies to process heterogeneous states and actions while balancing near- and long-term consequences. Existing solutions generalize poorly across network tasks, require substantial task-specific redesign, and lack reliable evaluation of long- horizon action consequences. Methods that learn through online interaction additionally face unstable optimization. We present NetRL, a two-stage LLM-based reinforcement learning framework that combines supervised fine-tuning on demonstration trajectories with online policy optimization. A universal mixed-embedding interface combines textual task descriptions with precise numerical states, while a network action head produces valid decisions. At its core, NetGRPO evaluates candidate actions over multiple future trajectories, directs deeper search toward uncertain alternatives through confidence-aware budgeting, and performs group-relative, clipped, and KL-regularized updates. NetRL achieves 23.6% higher QoE than NetLLM, 12.2% lower job completion time than Genet, and 24.1% higher network utility than Orca, yielding state-of-the-art performance among the evaluated methods. Confidence-aware budgeting reduces end-to-end training time by 27.1% without reducing validation QoE. NetRL also converges stably and generalizes to unseen network traces without retraining. Code and model weights are available at the repository linked in this paper.

Figures & tables

Fig. 1: Long-horizon sensitivity in ABR. Starting from the same state, we take different bitrate actions at step 0 and then follow the same policy in all subsequent steps, resulting in substantially different cumulative QoE outcomes.

Requirement	NetLLM	Confucius	NetRL
Common Interface Architecture	✗	✗	✓
Multi-Path Action Evaluation	✗	✗	✓
Stable Online Policy Updates	✗	✗	✓
Closed-Loop Network Control	✓	✗	✓

TABLE I: Comparison of LLM-based network decision approaches.

Fig. 2: An overview of the NetRL framework.

Fig. 3: The mixed-embedding interface. A textual prompt describing the task objective and feature semantics is processed by the native tokenizer, while numerical state features are projected into the LLM hidden space through MLP encoders. The text and numerical embeddings are concatenated and jointly processed by the LLM.

Fig. 4: The NetGRPO algorithm. All candidate actions are expanded from the root state. Descendant trajectories are generated through pruned beam rollout, and their cumulative returns are aggregated to estimate the long-term value of each root action. The resulting action group is used for group-relative policy optimization.

Panel of Figure 6 — Fig. 5: ABR performance.

Panel of Figure 7 — Fig. 6: LB performance.

Panel of Figure 8 — Fig. 7: CC performance.

Variant	ABR	LB	CC
NetRL (full)	0.89	2.65	0.36
w/o LLM Pre-training	0.67	3.57	0.28
w/o Mixed-Embedding	0.70	4.83	0.27
w/o Network Action Head	0.76	4.56	0.32
w/o SFT	0.65	2.91	0.29
w/o RL (SFT only)	0.82	3.83	0.33

TABLE II: Ablation results (ABR: QoE

\uparrow

; LB: JCT

\downarrow

; CC: utility

\uparrow

Fig. 8: ABR training dynamics for NetLLM, NetRL SFT, and NetGRPO.

Fig. 9: QoE and inference latency of candidate foundation models on ABR. Marker size indicates GPU memory usage.

Panel of Figure 12 — Fig. 10: ABR performance on two independent out-of-distribution trace sets without retraining.

RL Control

Reinforcement learning control uses reward-driven learning to make decisions in dynamic systems, from robots and vehicles to energy and infrastructure.

See all 261 papers

From Prior-Guided Heuristics to Deployable Agents: Accelerating Demonstration-Driven Reinforcement Learning for Deadline-Constrained Network Control
Sep 3, 2026Vincenzo Norman Vitale, Mohammad Solki, Antonia Maria Tulino +2Learning from Demonstration Multi-Agent Reinforcement Learning
DIETI, University of Naples Federico II, Via Claudio 21, Naples, 80125, Italy · New York University, 6 MetroTech Center, Brooklyn, New York, 11201, USA · University of Southern California, 3740 McClintock Ave., Los Angeles, 90089, California, USA +2
Selecting Offline Reinforcement Learning Algorithms for Stochastic Network Control
Mar 4, 2026Nicolas Helson, Pegah Alizadeh, Anastasios GiovanidisConservative RL Offline RL
Ericsson Research, Massy, France
Learning from the Near Future: Temporal Self-Distillation for RLVR
Apr 22, 2026Chuanyu Qin, Chenxu Yang, Qingyi Si +6Reinforcement Learning with Verifiable Rewards Self-Distillation
Institute of Information Engineering, CAS · School of Cyber Security, UCAS

Sep 3, 2026J/K move · Enter open · S save

From Prior-Guided Heuristics to Deployable Agents: Accelerating Demonstration-Driven Reinforcement Learning for Deadline-Constrained Network Control

Vincenzo Norman Vitale, Mohammad Solki, Antonia Maria Tulino, Andreas F. Molisch+1

DIETI, University of Naples Federico II, Via Claudio 21, Naples, 80125, Italy · New York University, 6 MetroTech Center, Brooklyn, New York, 11201, USA · University of Southern California, 3740 McClintock Ave., Los Angeles, 90089, California, USA+2

Learning from Demonstration Multi-Agent Reinforcement Learning Offline-To-Online RL

Open paper AskPDF arXiv

Timely delivery of delay-sensitive information over dynamic, heterogeneous networks is essential for NextG interactive applications, yet providing strict End-to-End (E2E) peak latency guarantees remains an open challenge. Two obstacles limit the adoption of learning-based network control in this setting: traditional volume-based routing metrics, while highly effective for general traffic management, are not designed to capture traffic urgency; and Deep Reinforcement Learning (DRL) controllers trained from scratch suffer from sample inefficiency, long training times, and early-stage exploration volatility. This paper introduces a deployment-focused network control framework that addresses both obstacles. First, we present Effective Congestion (EC), a deadline-aware metric family that quantifies interface congestion by packet urgency and proactively filters non-viable traffic, coupled with a Uniform Path Grouping (UPG) distribution heuristic promoting robust load-balancing; the resulting policies are embedded into Multi-Agent Deep Reinforcement Learning Effective Congestion (

p^*

) (MADRL EC (

p^*

)), a hybrid architecture combining a distributed scheduler with a centralized RL-based router. Second, we introduce a unified training objective that generalizes existing policy-learning paradigms---behavioral cloning, offline Reinforcement Learning (RL), online RL, and offline-to-online schemes---as special cases, combining a live-reward term, a pre-collected-reward term, and a policy-imitation term. From this objective, we derive the Model-Guided Annealed Reinforcement Learning (MGA-RL) protocol, instantiated on a Deep Deterministic Policy Gradient (DDPG) backbone: a deployment-oriented, demonstration-driven training approach that generalizes conventional Offline-to-Online (O2O) schemes, in which trajectories from a lightweight [...]

Abstract

RL Control

Explore similar work

From Prior-Guided Heuristics to Deployable Agents: Accelerating Demonstration-Driven Reinforcement Learning for Deadline-Constrained Network Control

Selecting Offline Reinforcement Learning Algorithms for Stochastic Network Control

Learning from the Near Future: Temporal Self-Distillation for RLVR