Organizations: Department of Electrical and Computer Engineering, Villanova University, Villanova, PA, USA · School of Arts and Sciences, University of Pennsylvania, Philadelphia, PA, USA
Neural-symbolic AI (NSAI) combines learned perception with programs, constraints, and compositional reasoning, but remains difficult to test because small semantics-preserving changes can alter grounded symbols and trigger discontinuous reasoning failures. Existing neural fuzzers mainly rely on neural coverage or representation-space feedback and do not provide a common testing interface for heterogeneous NSAI systems. We introduce NSAI-Fuzz, a unified semantics-aware fuzzing framework that separates task-specific valid mutation from architecture-neutral guided search and represents each candidate using five availability-aware, rank-normalized risk signals: boundary pressure, grounding uncertainty, representation drift, constraint violation, and archive novelty. We further propose Reliability-Calibrated Weighting (RCW), which learns dataset-model-level signal allocations from calibrated reliability, and Contextual Signal Gating (CSG), which adapts signal weights to the current candidate pool. We evaluate four NSAI systems across four reasoning tasks, nine compatible dataset-model cells, 20 policies, and 1,800 on-policy runs. Fixed five-signal guidance outperforms Random and Uniform in every cell, while RCW and CSG further improve over Fixed in all nine cells. Under matched mutation and query budgets, RCW and CSG achieve 16.17% and 16.06% aggregate failure discovery, compared with 6.06% for the strongest neural-fuzzing baseline. Wall-clock evaluation further shows that TensorFuzz requires 1.11x the runtime of RCW while achieving only 0.45x its failure throughput. These results show that effective and efficient fuzzing of heterogeneous NSAI systems benefits from a common neural-symbolic risk interface with adaptive signal allocation.
Figures & tables
Figure 1: Overview of NSAI-Fuzz . Task-specific semantic mutators generate valid candidates, which are mapped by a common adapter to five unified guidance signals. RCW and CSG adapt their weights to guide on-policy search across heterogeneous NSAI systems.
Figure 2: Five search policies on the Sort-of-CLEVR. All mutations preserve the oracle answer Square but induce the prediction Circle ; the bottom row visualizes enhanced pixel differences.
Dataset
Model
DeepHunter
TensorFuzz
DLFuzz
NSGen-T
CriticalFuzz
Random
Uniform
Fixed
RCW
CSG
Sort-of-CLEVR
ISED
0.18
1.91
1.31
0.39
0.14
1.127
11.693
12.675
13.832
13.526
Sort-of-CLEVR
DOLPHIN
1.35
1.60
1.50
0.59
1.33
2.108
13.723
14.335
14.914
15.557
Visual Sudoku
ISED
4.69
5.72
5.14
3.95
4.49
6.645
12.791
14.133
16.062
16.629
Visual Sudoku
DOLPHIN
0.00
0.00
0.00
0.00
0.00
1.395
4.002
4.908
5.764
5.740
Visual Sudoku
NASR
9.96
9.08
10.31
10.02
9.59
9.598
18.023
21.303
23.598
23.740
HWF-7
ISED
3.30
7.23
8.12
0.80
2.93
0.619
5.904
6.725
9.680
9.934
Table 1: Mean failure discovery rate (%) across ten paired attack seeds.
Policy
Raw failures
Exact unique
Similarity clusters
DeepHunter
14,670
5,300
5,450
TensorFuzz
27,940
5,610
6,210
DLFuzz
25,840
5,390
5,850
NSGen-transfer
24,780
5,330
5,740
CriticalFuzz
17,240
5,370
5,620
RCW
74,506
10,719
11,508
Table 2: Failure diversity across all nine dataset–model cells and ten paired attack seeds.