Authors: Mohammad Wali Ur Rahman, Martin Manuel Lopez, Lamia Tasnim Mim, Carter Farthing, Julius Battle, Kathryn Buckley, Salim Hariri
Organizations: Department of Electrical and Computer Engineering from the University of Arizona, Tucson, AZ 85721, USA. · Avirtek, Inc., Tucson, AZ 85712, USA. · Joint Interoperability Test Command (JITC), United States Department of Defense, AZ 85613, USA.
For digital infrastructure to be safe, compatible, and standards-aligned, automated communication protocol compliance verification is crucial. Nevertheless, current rule-based systems are becoming less and less effective since they are unable to identify subtle or intricate non-compliance, which attackers frequently use to establish covert communication channels in IPv6 traffic. In order to automate IPv6 compliance verification, this paper presents the Artificial Intelligence Driven Compliance Checker Engine (AICCE), a novel generative system that combines dual-architecture reasoning and retrieval-augmented generation (RAG). Specification segments pertinent to each query can be efficiently retrieved thanks to the semantic encoding of protocol standards into a high-dimensional vector space. Based on this framework, AICCE offers two complementary pipelines: (i) Explainability Mode, which uses parallel LLM agents to render decisions and settle disputes through organized discussions to improve interpretability and robustness, and (ii) Script Execution Mode, which converts clauses into Python rules that can be executed quickly for dataset-wide verification. With the debate mechanism enhancing decision reliability in complicated scenarios and the script-based pipeline lowering per-sample latency, AICCE achieves accuracy and F1-scores of up to 99% when tested on IPv6 packet samples across sixteen cutting-edge generative models. By offering a scalable, auditable, and generalizable mechanism for identifying both routine and covert non-compliance in dynamic communication environments, our results show that AICCE overcomes the blind spots of conventional rule-based compliance checking systems.
TABLE I : Qualitative Comparison of Network Protocol Validation Methods Across Key Criteria.
Fig. 1 : Basic Architecture Diagram of a) Llama, b) GPT and c) Mixtral 8x7b
Fig. 2 : Visualization of ANN search with the Hierarchical Navigable Small World (HNSW) algorithm.
Fig. 3 : AICCE Framework in Explainability Mode
Fig. 4 : AICCE Framework in Script Execution Mode
Aspect
Architecture A
Architecture B
Primary Focus
Interpretability, coverage
Speed, automation
Multi-agent Usage
per section & per packet
per section
Debate Mechanism
Yes
Yes
Granularity
Section-level judgments
Rule-level execution
Output
Rationale + verdict
Rule matrix + verdict
Exec Speed
∼2 s / packet
∼0.8 s / packet
TABLE II : Comparison of Framework Architectures
Violation Type
Difficulty
Occurrences
IPv6 Version Violation
Trivial
52
Length Field Violation
Moderate
54
IPv6 Address Violation
Trivial
54
Hop Limit Violation
Moderate
52
TCP/UDP/ICMPv6 Overlap
Challenging
49
Flow Label Violation
Moderate
54
TABLE III : Violation Types by Detection Difficulty
Provider
Model(s)
API / Access Layer
OpenAI
GPT-4o
OpenAI API [ 37 ]
o1
GPT-4-Turbo
GPT-4o-mini
GPT-3.5-Turbo
Meta
Llama 3.3-70B
Groq API [ 38 ]
TABLE IV : LLM providers, models, and access layers used in experiments
Fig. 5 : Model accuracy comparison under ablation vs. debate-enabled settings for Architecture A. Debate consistently improves accuracy across models, particularly for weaker baselines.
Fig. 6 : F1-score comparison under ablation vs. debate-enabled settings for Architecture A. Debate reduces false positives and stabilizes precision–recall balance.
Fig. 7 : Decision latency comparison under ablation vs. debate-enabled settings for Architecture A. As expected, enabling debate increases runtime, but accuracy gains justify the overhead.
Fig. 8 : Number of debate phases initiated per model in Architecture A. Stronger CoT models required fewer debates, while weaker baselines triggered debates more frequently.
Fig. 9 : Performance of models in Script Execution Mode (Architecture B). Subfigures show (a) accuracy, (b) F1-score, (c) latency, and (d) number of retries.
System
Accuracy
Precision
Recall
F1-score
Latency
Explainability
Err. Recovery
Logistic Regression ∗
0.93
0.9452
0.69
0.7977
0.00035
No
No
Random Forest ∗
0.938
0.8557
0.83
0.8426
0.00086
No
No
Extra Trees ∗
0.928
0.8333
0.80
0.8163
0.00043
No
No
XGBoost ∗
0.56
0.2987
0.89
0.4472
0.00016
No
No
LightGBM ∗
0.50
0.2656
0.85
0.4048
0.00020
No
No
Support Vector Machine ∗
0.914
0.9831
0.58
0.7296
0.00029
No
No
TABLE V : Performance Comparison of Different Compliance Checker Systems
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Method
Model
Acc
Prec
Rec
F1
Latency
Debates /Retries
Offers Explainability
Error Recovery
Classical ML ∗
Random Forest
0.938
0.8557
0.83
0.8426
0.00086
N/A
No
No
MLP Neural Net
0.93
0.8095
0.85
0.8293
0.00233
N/A
No
No
Logistic Regression
0.93
0.9452
0.69
0.7977
0.00035
N/A
No
No
Extra Trees
0.928
0.8333
0.80
0.8163
0.00043
N/A
No
No
SVM
0.914
0.9831
0.58
0.7296
0.00029
N/A
No
No
KNN
0.888
0.9583
0.46
0.6216
0.00001
N/A
No
No
Appendix
TABLE VI : Comprehensive comparison across AICCE configurations and baselines.