Targeting the Right Situations: Mutation Testing of Fuzzy-Powered Situation Hyperspace

Nawshin Mannan Proma

cs.ROOct 10, 2026

Targeting the Right Situations: Mutation Testing of Fuzzy-Powered Situation Hyperspace

Authors: Nawshin Mannan Proma

Organizations: University of York, Heslington, United Kingdom

Abstract

Ensuring the safety of autonomous Unmanned Aerial Vehicles (UAVs) operating in underground mines is challenging due to the vast and uncertain Operational Design Domain (ODD). Traditional validation approaches, such as random testing, often result in inefficient situation coverage, allocating computational resources to low-risk scenarios while failing to expose critical edge cases. Building on prior work in risk-prioritized Situation Coverage, this paper investigates whether Fuzzy Logic-based risk estimation effectively correlates with real system failures. To this end, we conduct a mutation testing study within our SCALOFT simulation framework. A fixed set of software mutants is systematically injected and evaluated under two testing strategies: (i) uniformly random situation generation and (ii) Fuzzy-Guided risk-prioritized situation generation. Experimental results show that the Fuzzy-Guided approach achieves a higher mutation score and is more effective at exposing difficult-to-detect faults compared to random testing.

Figures & tables

Panel of Figure 1 — Figure 1: Overall Framework for Mutation Testing

Mutant ID	Type	Description
M1	AOR	Arithmetic operator modification affecting numerical computations
M2	BCR	Boundary condition change altering control thresholds
M3	ASR	State/assignment modification causing abnormal system behaviour
M4	COD	Conditional logic modification leading to incorrect decisions
M5	AOR+ASR	Combination of arithmetic and state-related faults

Table 1: Injected Mutants Used for Evaluation

Figure 2: Fuzzy-powered situation hyperspace from [ 4 ] . The color gradient represents a gradual risk transition (the yellow zone indicates high-risk situations, the cyan region corresponds to medium risk, and the blue region denotes low-risk situations).

Mutant	Random (10 Runs)	Fuzzy-Guided (10 Runs)
M1	4/10	10/10
M2	2/10	9/10
M3	1/10	10/10
M4	5/10	9/10
M5	0/10	10/10

Figure 3: Mutant detection frequency across generated scenarios. Each cell shows the number of scenarios (out of 10) in which the mutant was killed.

Panel of Figure 4 — Figure 4: Comparison of mutation scores between random and fuzzy-guided testing.

Automated Software Testing

Automated software testing uses tools and algorithms to generate, run, and evaluate tests that find defects and prevent regressions.

See all 136 papers

Revealing Safety-Critical Scenarios for UTM via Transformer
Jun 30, 2026Huaze Tang, Bill Zeng, Chao Wang +3Reinforcement Learning Intelligent Transportation Systems
Tsinghua University · Meituan
UAMTERS: Uncertainty-Aware Mutation Analysis for DL-enabled Robotic Software
Feb 23, 2026Chengjie Lu, Jiahui Wu, Shaukat Ali +5Mutation Testing Automated Software Testing
Simula Research Laboratory and University of Oslo · Simula Research Laboratory · Danish Technological Institute +1
MulRobBench: A Decision-Level Benchmark for Safe and Security-Policy-Compliant Multimodal UAV Agents
Jul 26, 2026Belal S. Alsinglawi, Weizheng Wang, Junyi Wu +3Aerial Robotics AI Agent Security Benchmarks
College of Technological Innovation, Zayed University, Abu Dhabi, United Arab Emirates

Jun 30, 2026J/K move · Enter open · S save

Revealing Safety-Critical Scenarios for UTM via Transformer

Huaze Tang, Bill Zeng, Chao Wang, Zhenpeng Shi+2

Tsinghua University · Meituan

Reinforcement Learning Intelligent Transportation Systems Transformer-Based RL Software Vulnerability Detection

Open paper AskPDF arXiv

Unmanned Traffic Management (UTM) systems are cloud-based platforms designed to manage and coordinate multiple aerial vehicles remotely. UTM systems are safety-critical which cannot tolerate failures like crash or collision. To reveal latent vulnerabilities, there are neither optimal failure-exposing demonstrations nor clear reward signals. Additionally, UTM's self-healing capability introduces the ``long-tail effect'' of critical failures. We propose framing UTM vulnerability discovery as a sequence modeling problem amenable to transformer-based RL architectures. Our approach leverages attention mechanisms to directly model the relationship among system states, and predict optimal actions. Our framework introduces a Policy Model that generates targeted test scenarios and an Action Sampler that enforces domain constraints. We use a risk-based reward function to guide exploration. Through extensive evaluation on a 700-hour simulation study, we demonstrate an 8

\times

improvement in vulnerability discovery efficiency compared to expert-guided testing. It also discovers critical edge cases that traditional methods have missed.

Abstract

Automated Software Testing

Explore similar work

Revealing Safety-Critical Scenarios for UTM via Transformer

UAMTERS: Uncertainty-Aware Mutation Analysis for DL-enabled Robotic Software

MulRobBench: A Decision-Level Benchmark for Safe and Security-Policy-Compliant Multimodal UAV Agents