cs.AIOct 9, 2026

A Local Agentic AI Framework to Generate Edge Case Scenarios for Safety Assurance

Authors: Victoria J. Hodge, Colin Paterson, Rob Alexander

Organizations: Department of Computer Science, University of York, York, YO10 5DD, UK

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Generating edge case scenarios is crucial for training, testing and safety assuring autonomous systems. This identifies hazardous failures to increase system reliability and robustness, ensures the system can still function safely on the boundaries of its operational design domain and learns the expected system behaviour for these edge cases. The rapid advancement in the generation and analysis capabilities of language models (LMs) has expanded their applicability and recent work has investigated LM scenario generation. However, many recent approaches rely on human-in-the-loop or rule-based strategies. This restricts the LM's ability to generate diverse and edge case scenarios; the long tail of the distribution. Our work uses local LMs in a laptop-based agentic AI framework for adaptive scenario generation. This is cheap, fully-controlled, ensures data privacy and security. It uses think, act, observe, update, and continue loops with lightweight JSON encoding to generate scenarios. We use the system requirements, system description and environment description as knowledge bases in contextual prompts for an LM to ensure scenario realism. A second, independent LM critiques each scenario using domain specific criteria so it can be refined. The refined scenarios are run in simulation and evaluation metrics collected. These metric scores are analysed and similar scenarios are created for edge cases (exploitation) or novel scenarios are generated to explore the scenario space. Post-hoc clustering using metric scores and thematic-type analysis of the rich-text scenario descriptions produced by the LM pinpoints functional gaps and their triggering conditions. This creates traceable evidence for the safety assurance case. Our work highlights the potential of a flexible and adaptive local agentic AI framework for automating edge case generation and producing traceable safety assurance evidence.

Figures & tables

Figure 1: The overall architecture and process flow of our proposed agentic AI framework. Icons from https://thenounproject.com/ , the Noun Project
Figure 2: Overview of the first part of the agentic AI scenario generation framework. Part 1 produces the JSON-encoded scenarios to run in simulation in part 2 which is shown in Figure 3 . The domain ontology allows us to map the sections of the operational deign domain description to sections of the LM prompt and is described in section 4.2 .
Figure 3: Overview of the agentic AI simulation, evaluation and decision logic to assess the scenarios and to decide whether to exploit an existing scenario or explore novel scenarios by looping back to step 1 in Figure 2 .
Figure 4: An example scenario laid out in the Sorter arena in Unity-3D. The blue cube in the centre is the robotic agent. The red crosses are random obstacles and the coloured cubes are fixed obstacles.
Figure 5: High-level goal structuring notation for the robotic-agent navigating the circular arena. From the claim in G1, we must demonstrate that the risk of colliding with obstacles is as low as reasonably practicable. The GSN decomposes through levels includingt the GSN shown in Figure 6
Figure 6: Lower level goal structuring notation fragment for the robotic-agent’s navigation. This paper focuses on generating the evidence for Sn1.4 - Sn1.7. After generating our edge case scenarios and analysing the evaluation metrics and simulation logs, we can instantiate the artefacts for Sn1.4 { eval metrics }, Sn1.5 { simulation failures & collisions log ( EsimE_{sim} ) }, Sn1.6 { identified SOTIF area 3 →\rightarrow area 2 edge cases } & Sn1.7 { hazard discovery report }
Figure 7: Eight scenarios generated by the Agentic AI and laid out in the Sorter arena in Unity-3D.
BaselineDescription
Random_1Random asset placement generated using the Python random number generator with 1-20 tiles and 1-10 obstacles. Tile values are generated as unique values (used at most once). If tile or obstacle locations clash then a new location is generated.
Random_5Random asset placement generated using the Python random number generator with 1-20 tiles and 5 -10 obstacles. Tile values are generated as unique values (used at most once). If tile or obstacle locations clash then a new location is generated.
Table 1: The baseline scenario generation approaches to allow us to evaluate the merits of our agentic approach.
MetricDescription
Success scorehow many times it successfully completed the task (over 50 episodes)
Timeout scorehow many times the task timed out (over 50 episodes)
Wrong tile scorehow many times it visited the wrong tile (over 50 episodes)
Avg path lengththe average number of steps taken (over 50 episodes)
Ideal path lengththe ideal number of steps (shortest path)
Regret(avgPathLength / idealPathLength)
Table 2: The set of evaluation metrics exported by Unity-3D for use in the agentic AI closed loop reflection analyses.
Figure 10
Figure 11
Figure 12
Figure 14: Chart showing the average path length across all scenarios for each scenario generation method. The path length is 0 if it visited the wrong tile.
Figure 14
Figure 17: Spatial maps of obstacle placements across all scenarios for each scenario generation method.
Figure 18: Chart showing how many times each number of obstacles was selected across all scenarios for each scenario generation method.
Figure 19: Chart showing how many times each number (count) of tiles was selected across all scenarios for each scenario generation method.
Figure 20: Chart showing how many times each tile position was selected across all scenarios for each scenario generation method.
Figure 21: Chart showing how many times each tile value was selected across all scenarios for each scenario generation method.
Figure 22: Chart showing the ideal path length across all scenarios for each scenario generation method.
Table 21
Table 22
Table 23
Table 24
JSON Parameter / Asset ConfigurationISO 21448 Triggering Condition (TC)Exposed Functional Gaps (FG)Resulting Unsafe Behaviour
Dense clusters of obstacles near the robot’s starting pointAlgorithmic search space saturation: Complex spatial topologyPlanning oscillations ( FGplanFG_{plan} ): Local planner path deadlock or trajectory freezing.Timeouts: Robot stuck, path oscillation, or trajectory abandonment
Tile access blockades (clearance <wrobot<w_{robot} )Geometric Constraint Boundary: Access to tile blocked requiring robot to reversePerception Insufficiency ( SWperSW_{per} ): LiDAR point-cloud sparsity (no backward facing LiDAR)Collisions with wrong tile or arena walls, or obstacle contact.
Table 3: Hazard Discovery Report: formal mapping table to demonstrate how the generated spatial layouts for θJSON1\theta_{JSON_{1}} and θJSON2\theta_{JSON_{2}} map to triggering conditions and functional gaps exposed by the simulation’s evaluation metrics. We assume that timeouts are a safety risk as the robot is stuck and may oscillate leading to collisions.
More in this topic

AI Assurance

Artificial intelligence assurance develops evidence and controls to evaluate, govern, and certify AI systems across their lifecycle and deployment contexts.

See all 77 papers

Explore similar work

CardsList
Jun 2, 2026

EvoDrive: Pareto Evolution for Safety-Critical Autonomous Driving via Self-Improving LLM Agents

Generating safety-critical scenarios is essential for validating and improving autonomous driving systems, yet it inherently requires maximizing adversariality to expose failures while preserving realism. Existing methods usually manage this trade-off with handcrafted heuristics, confining generation to known priors and overlooking underexplored patterns. While recent open-ended agentic evolution can push this limit, unconstrained general agents lack strict simulator grounding and tend to collapse the multi-objective tension into single-scalar maximization. Here we present EvoDrive, the first automated, LLM-based agentic evolution framework for multi-objective scenario generation. EvoDrive employs a simulator-grounded actor-critic architecture where a memory-driven actor iteratively proposes improvements to the generators and critics filter out implausible candidates, and a self-evolving world evaluator routes promising proposals to optimize simulation budgets. EvoDrive further maintains a Pareto archive of evaluated candidates to preserve diverse attack-realism trade-offs and guide future evolution via simulation feedback. Benchmark results on MetaDrive and CARLA show that EvoDrive not only significantly expands the Pareto frontier across various generators, but also produces valuable scenarios for policy training.

Tong Nie, Yuewen Mei, Yihong Tang +4

The Hong Kong Polytechnic University · Tongji University · McGill University +1

Multi-Objective Evolutionary OptimizationAutonomous Driving Safety EvaluationAdversarial Scenario GenerationAutonomous DrivingSelf-Improving AgentsAutonomous Driving Simulation+1
AskPDF
May 15, 2026

PCASim: Promptable Closed-loop Adversarial Simulation for Urban Traffic Environment

Real-world autonomous driving, particularly in urban environments with numerous corner cases, requires rigorous testing to ensure product safety and robustness. However, few studies have explored integrating adversarial scenario generation with the training of safety agents in closed-loop testing, enabling efficient co-evolution and mutual enhancement of both. To address this challenge, an adversarial behavior knowledge repository is constructed by applying rule-based filtering to an open-source dataset, combined with knowledge retrieval modules tailored for simulation environments. A large language model (LLM) is employed to integrate knowledge-, data-, and adversarial-driven approaches, generating safety-critical traffic scenarios customized to user needs. Additionally, while evaluating the generated scenarios, we employ reinforcement learning models to train the behaviors of different types of vehicles, thereby enriching scenario diversity beyond existing datasets while preserving realism. Experimental results demonstrate that the proposed framework improves the accuracy of domain-specific language generation by 12%. Moreover, the success rate of newly generated scenario transformations increases by 8%, while obstacle-avoidance capability is enhanced by 30%. For the complete manuscript, please refer to: https://zhenhaooo.github.io/PCASim.github.io/

Chuancheng Zhang, Zhenhao Wang, Kaizheng Li +3

Shenzhen Research Institute of Shandong University, Shenzhen, 518057, Guangdong, China · School of Airspace Science and Engineering, Shandong University, 264209 Weihai, China. · School of Computer Science and Technology, Shandong University of Finance and Economics, 250014, Jinan, China.

Autonomous Driving Safety EvaluationSynthetic Data GenerationAdversarial Scenario GenerationAutonomous DrivingAutonomous Driving SimulationAutonomous Driving Scenario Generation
AskPDF
Jun 7, 2026

ForesightSafety-SAGE:A Fully Automated Scenario Generation and Safety Evaluation Framework for LLM Agents

Large language models (LLMs) are increasingly evolving from simple text-based interaction systems into LLM agents that can maintain memory, use tools, access external environments, and execute tasks. As their capabilities and autonomy expand, the safety risks they face also become more diverse. Existing evaluations often rely on manually written scenarios, static prompts, or final-output judgments, making it difficult to capture the diverse risks that agents may face during task execution. We introduce ForesightSafety-SAGE, a fully automated scenario generation and safety evaluation framework for LLM agents. Based on five risk dimensions,we instantiae abstract and diverse safety risks in real-world task execution into 1,072 measurable evaluation scenarios. Using the automated evaluation pipeline, 12 LLM agents are evaluated under two authority contexts. The results show that current agents still face substantial behavioral safety risks during task execution, with an average ASR of 47.1% and several models exceeding 70%. These findings demonstrate the importance of executable, process-level evaluation for understanding and improving LLM agent safety.

Lu Jia, Haibo Tong, Feifei Zhao +5

BrainCog AI Lab, Institute of Automation, Chinese Academy of Sciences · Beijing Institute of AI Safety and Governance, China · University of Chinese Academy of Sciences (UCAS), Beijing, China +2

AI Agent Safety BenchmarksLLM Agent Safety
AskPDF