cs.CROct 11, 2026

AutoSec-RAN: Specification-Grounded LLM Fine-Tuning for Automated Security Analysis of O-RAN

Authors: Anoop Mishra, Joshua Moore, Aly Sabri Abdalla, Vuk Marojevic

Organizations: Dept. of Electrical and Computer Engineering, Mississippi State University, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

This paper presents AutoSec-RAN, a specification-grounded large language model (LLM) framework for automated security triage of Open Radio Access Network (O-RAN) testing. Unlike approaches relying on static rule sets or privileged access to raw packet captures, AutoSec-RAN fine-tunes Mistral-7B-Instruct through two Low-Rank Adaptation (LoRA) stages over a curated corpus of 69 O-RAN, 3GPP, and adjacent standards documents, then selectively augments inference with retrieval-augmented generation (RAG) over the same specification index. At inference, the model receives only an investigator-style question and a scenario metadata block, simulating an automated analyst operating without raw packet evidence. AutoSec-RAN is validated on a purpose-built 5G/O-RAN testbed across labeled fronthaul distributed denial of service replay scenarios. A Gemma-4-31B judge LLM scores each candidate's answer on faithfulness, correctness, and actionability relative to a reference authored from full packet capture and log evidence. AutoSec-RAN achieves a mean judge overall score of 0.713 across all 313 evaluation items (0.755 restricted to the five supervised taxonomy classes), clearing the 0.70 reporting threshold, and a blended score of 0.778 (0.808 supervised-only) when cosine similarity over Beijing Academy of Artificial Intelligence (BAAI) General Embedding (BGE) is incorporated as a secondary semantic signal. The framework performs strongest on mitigation and threat-model reasoning (with a mean overall of 0.849 and 0.848, respectively) and identifies a measurable gap on the held-out attack-direction class (0.515), a controlled result that exposes a specific training data requirement. These results establish specification-grounded LoRA fine-tuning combined with RAG retrieval as a viable foundation for automated security validation of AI-native O-RAN.

Figures & tables

Fig. 1 : Specification-grounded LLM framework for automated O-RAN security triage. The model receives only a structured question and a seven-field metadata block at inference time.
Fig. 2 : AutoSec-RAN pipeline architecture. The offline training pipeline (left) curates 69 security-relevant O-RAN, 3GPP, and related standards, indexes them into fronthaul and 5G-core FAISS vector stores using parent–child chunking with BGE-large-en-v1.5 embeddings, generates 5,448 analyst-style question-answer pairs with Gemma-4-31B, and fine-tunes Mistral-7B-Instruct via two-stage LoRA. The inference/evaluation pipeline (right) performs RAG-based retrieval from the same FAISS indexes, serves the Stage 1.2 LoRA adapter on a 4-bit NF4-quantized Mistral-7B backbone with bf16 compute, and evaluates outputs using a Gemma-4-31B judge for faithfulness, correctness, and actionability.
Taxonomy ClassnnGemma-4-31BQwen2.5-32BQwen2.5-72B
  MeanPass @ 0.70MeanPass @ 0.70MeanPass @ 0.70
Mitigation560.8490.8750.8441.0000.8680.982
Threat Model410.8480.7560.8120.8540.8190.854
Spec Grounding590.7580.5250.7820.8810.8040.847
Detection Signals580.7120.5340.7540.7590.7910.862
Scenario Severity Expectation440.6040.3410.7120.6590.7120.500
TABLE I : Per-Class Evaluation Results
Fig. 3 : Mean judge overall score per taxonomy class. The sample size per class is annotated within each bar.
Fig. 4 : Aggregate score components for the full evaluation set (313 items) and the supervised-class-only subset (258 items, excluding the held-out Attack Direction Reasoning class). Each group shows the mean overall, normalised cosine, and blended score.
Fig. 5 : Per-class pass rate at the judge. Classes are sorted in ascending order of pass rate. Mitigation and Threat Model are the only classes that exceed 70%; Attack Direction Reasoning, excluded from specification training by design, achieves the lowest pass rate at 20.0%.
Fig. 6 : Joint distribution of mean overall score and pass rate across taxonomy classes. Bubble area is proportional to the number of evaluation items in each class; the shaded region identifies the target quadrant where both score quality and consistency are acceptable.
Fig. 7 : Judge sub-score breakdown for two case study evaluation items from the C-plane downlink RU–DU traffic-injection scenario. Faithfulness, Correctness, and Actionability contribute to the weighted Overall score, while Coverage, shown in hatched gray, is reported only as a diagnostic metric. Diamond markers indicate the final weighted Overall score for each item.
More in this topic

Cybersecurity

Cybersecurity develops methods to protect digital and cyber-physical systems from attacks, vulnerabilities, data loss, and misuse.

See all 355 papers

Explore similar work

CardsList
May 22, 2026

Advanced AI Service Provisioning in O-RAN through LLM Engine Integration

The Open Radio Access Network (O-RAN) architecture allows AI to be embedded directly into the RAN through modular xApps and rApps, yet creating these applications collecting data, training models, writing code, and deploying them safely remains slow and largely manual. Large Language Models (LLMs) offer strong reasoning and code-generation capabilities but are unsuited for the fast, deterministic inference required in real-time RAN control. We present a proof-of-concept Dual-Brain architecture that combines both strengths: an LLM-based orchestrator translates operator intents into data-collection policies and deployment code, while an automated ML engine, NeuralSmith, trains lightweight classifiers on demand via an API. We describe the architecture and provisioning workflow, share practical insights from a containerized O-RAN 5G~SA testbed, and discuss open research directions.

Seyed Bagher Hashemi Natanzi, Pranshav Gajjar, Bo Tang +1

Department of Electrical and Computer Engineering, Worcester Polytechnic Institute, Worcester, MA 01609 USA · Department of Electrical and Computer Engineering, North Carolina State University, Raleigh, NC 27695 USA

LLM Agent OrchestrationWireless CommunicationsOpen Radio Access Network
AskPDF
Feb 27, 2026

Agentic AI-RAN: Enabling Intent-Driven, Explainable and Self-Evolving Open RAN Intelligence

Open RAN (O-RAN) exposes rich control and telemetry interfaces across the Non-RT RIC, Near-RT RIC, and distributed units, but also makes it harder to operate multi-tenant, multi-objective RANs in a safe and auditable manner. In parallel, agentic AI systems with explicit planning, tool use, memory, and self-management offer a natural way to structure long-lived control loops. This article surveys how such agentic controllers can be brought into O-RAN: we review the O-RAN architecture, contrast agentic controllers with conventional ML/RL xApps, and organise the task landscape around three clusters: network slice life-cycle, radio resource management (RRM) closed loops, and cross-cutting security, privacy, and compliance. We then introduce a small set of agentic primitives (Plan-Act-Observe-Reflect, skills as tool use, memory and evidence, and self-management gates) and show, in a multi-cell O-RAN simulation, how they improve slice life-cycle and RRM performance compared to conventional baselines and ablations that remove individual primitives. Security, privacy, and compliance are discussed as architectural constraints and open challenges for standards-aligned deployments. This framework achieves an average 8.83% reduction in resource usage across three classic network slices.

Zhizhou He, Yang Luo, Xinkai Liu +4

5G/6GIC, Institute for Communication Systems (ICS), University of Surrey, Guildford, UK · 6G Research Center, Khalifa University, Abu Dhabi, UAE

Network SlicingWireless CommunicationsOpen Radio Access NetworkWireless Resource AllocationAgentic AISelf-Evolving Agents
AskPDF
Jun 4, 2026

DAST: A VLM-LLM Framework for Cross-Interface Anomaly Detection in O-RAN

O-RAN enables a disaggregated baseband stack with programmable functions that communicate over standardized open interfaces. The same openness that enables multi-vendor composition also expands the attack surface across logically decoupled tiers that make up the compute continuum. Among these threats, Denial-of-Service and performance-degradation attacks, which account for the majority of catalogued O-RAN threats, are particularly difficult to detect. Traditional Time-Series Anomaly Detection (TSAD) methods fail in this new regime where labelled baselines are scarce, threats evolve faster than detectors can be retrained, and the high-dimensional multivariate telemetry overwhelms monolithic inference models. To address these challenges, we present DAST, a zero-shot multi-agent framework for cross-interface anomaly detection in O-RAN that chains a three-stage VLM →\rightarrow LLM →\rightarrow VLM pipeline. DAST converts multivariate KPI streams into visual representations, scores textual per-interface descriptions against O-RAN domain knowledge, and verifies suspects on high-resolution heatmaps to output the problematic interfaces, the anomalous time intervals, an indicative O-RAN WG11-aligned operational impact rating and the decision rationale. We evaluate DAST on real network traces collected from an O-RAN testbed under representative performance degradation scenarios, achieving 0.910 F1-Score and 0.843 Accuracy, outperforming state-of-the-art TSAD baselines.

Francesco Spinelli, Esteban Municio, Pau Baguer +2

i2CAT Foundation · NEC Laboratories Europe · ICREA

Wireless CommunicationsOpen Radio Access NetworkMultimodal Anomaly DetectionZero-Shot Anomaly DetectionTime-Series Anomaly DetectionAnomaly Detection
AskPDF