Argument-Aware Semantic Alignment of Normative Texts: A Toulmin-Based Neuro-Symbolic Approach
Organizations: Purdue University; CTO Cleantech Software · CTO Cleantech Software
Abstract
Semantic alignment between specialized normative texts is challenging when equivalent requirements use different terms, syntax, and levels of abstraction. Lexical overlap, distributional embeddings, and semantic similarity capture topical relatedness but often miss the argumentative structure by which normative claims are supported, qualified, and justified. This paper asks whether explicit argument structure adds information complementary to neural semantics for aligning requirements. We treat cross-standard control mapping as argument-aware semantic alignment and build a neuro-symbolic pipeline that combines neural text representations with Toulmin features. An LLM explicitation step identifies claims, grounds, warrants, qualifiers, and backing and reconstructs enthymemes. These feed an alignment model via argument-aware similarity and structural features. On a NERC-CIP to NIST-CSF mapping benchmark, argument-derived features improve alignment over a neuro-symbolic semantic baseline. Feature selection shows especially strong signal from warrant-related features, indicating that the link between a claim and its supporting reasoning is not captured by conventional similarity alone. A compact claim--grounds--warrant subset remains competitive with the full Toulmin feature set. The results give preliminary evidence that argument structure is a useful intermediate representation for aligning specialized normative texts. Cybersecurity standards are used as a controlled testbed, not as proof of domain-independent generalization. The argument graphs produced by LLM explicitation may also support later work on retrieval, reasoning, and explanation over normative text.
Figures & tables
| Split | Positive | Synthetic negative | Total |
|---|---|---|---|
| OLIR DS | 198 | 0 | 198 |
| Hard Semantic | 0 | 138 | 138 |
| Stochastic Rule Based | 0 | 60 | 60 |
| Total | 198 | 198 | 396 |
| Role | Cybersecurity Standard Schema |
|---|---|
| Claim | The security objective or control goal the requirement asserts |
| Grounds | The factual or evidentiary basis cited or implied |
| Warrant | The risk reasoning connecting grounds to the claim |
| Qualifier | Applicable scope, conditions, system boundaries |
| Backing | Regulatory authority, cited standards, references |
| Feature | Description |
|---|---|
| am_claim_cosine | Security-objective alignment between the two requirements' claims |
| am_premise_similarity | Mean similarity across matched premise spans (factual basis alignment) |
| am_warrant_similarity | Mean similarity across matched warrant spans (inferential reasoning alignment) |
| am_qualifier_similarity | Applicability-scope similarity |
| am_backing_overlap | Normalized overlap in shared normative/citation authority |
| am_grounds_warrant_coherence | Premise warrant cross-role coherence |
| Hyperparameter | Value |
|---|---|
| source | sklearn.ensemble.RandomForestClassifier |
| n_estimators | 100 |
| max_depth | None |
| class_weight | Balanced |
| max_features | Sqrt |
| random_state | 42 |
| Condition | config | cv_f1_mean | cv_f1_std | test_f1_mean | test_f1_std | n_seeds |
|---|---|---|---|---|---|---|
| n/a | NeSy Only (prior work) | 0.7473 | 0.0252 | 0.7375 | 0.0407 | 10 |
| B-Jaccard | NeSy + Full AM (12 features) | 0.7641 | 0.0253 | 0.7438 | 0.0449 | 10 |
| B-Jaccard | NeSy + Gupta Triple (claim/premise/warrant) | 0.7618 | 0.0198 | 0.7528 | 0.0404 | 10 |
| B-Jaccard | AM Only (no NeSy base) | 0.6658 | 0.0307 | 0.6389 | 0.0541 | 10 |
| B-Cosine | NeSy + Full AM (12 features) | 0.7786 | 0.0202 | 0.7895 | 0.0615 | 10 |
| B-Cosine | NeSy + Gupta Triple (claim/premise/warrant) | 0.7712 | 0.0252 | 0.787 | 0.0563 | 10 |
| Cond. | config | base | Metric | n | Base mean | Am mean | Mean delta | delta_ci95_low | delta_ci95_high | w | l | t | wilcoxon_p | cohens_dz | rank_biserial_r | sig | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| B-Jaccard | NeSy + Full AM (12 features) | NeSy_Only | Cv | 10 | 0.7473 | 0.7641 | 0.0168 | 0.0075 | 0.0281 | 8 | 2 | 0 | 0.0049 | 0.953 | 0.891 | ||||||
| B-Jaccard | NeSy + Full AM (12 features) | NeSy_Only | test | 10 | 0.7375 | 0.7438 | 0.0063 | -0.0169 | 0.0303 | 6 | 4 | 0 | 0.3125 | 0.155 | 0.2 | ||||||
| B-Jaccard | NeSy + Gupta Triple (claim/premise/warrant) | NeSy_Only | cv | 10 | 0.7473 | 0.7618 | 0.0145 | 0.003 | 0.0275 | 7 | 3 | 0 | 0.0322 | 0.688 | 0.673 | ||||||
| B-Jaccard | NeSy + Gupta Triple (claim/premise/warrant) | NeSy_Only | test | 10 | 0.7375 | 0.7528 | 0.0153 | -0.0081 | 0.0423 | 4 | 4 | 2 | 0.2734 | 0.354 | 0.278 | ||||||
| B-Jaccard | AM Only (no NeSy base) | NeSy_Only | cv | 10 | 0.7473 | 0.6658 | -0.0815 | -0.1055 | -0.059 | 0 | 10 | 0 | 1.0 | -2.048 | -1.0 | ||||||
| B-Jaccard | AM Only (no NeSy base) | NeSy_Only | test | 10 | 0.7375 | 0.6389 | -0.0986 | -0.1488 | -0.05 | 2 | 8 | 0 | 0.9971 | -1.17 | -0.891 | ||||||
| condition | cv_f1_mean | cv_f1_std | test_f1_mean | test_f1_std | n_seeds |
|---|---|---|---|---|---|
| A-Extraction | 0.5307 | 0.0332 | 0.4758 | 0.0826 | 10 |
| B-Jaccard | 0.6658 | 0.0307 | 0.6389 | 0.0541 | 10 |
| B-Cosine | 0.7736 | 0.0218 | 0.7313 | 0.0424 | 10 |
| condition | Metric | n_seeds | extraction_mean | explicitation_mean | wilcoxon_p | sig_flag |
|---|---|---|---|---|---|---|
| B-Jaccard | cv_f1 | 10 | 0.5307 | 0.6658 | 0.001 | |
| B-Jaccard | test_f1 | 10 | 0.4758 | 0.6389 | 0.001 | |
| B-Cosine | cv_f1 | 10 | 0.5307 | 0.7736 | 0.001 | |
| B-Cosine | test_f1 | 10 | 0.4758 | 0.7313 | 0.001 |
| Cond. | W | chi2 | df | p | Sig. | seeds | N | Modal top role | Modal seed count |
|---|---|---|---|---|---|---|---|---|---|
| B-Jaccard | 0.7422 | 29.6884 | 4 | 0.0 | 10 | 5 | am_warrant_similarity | 7/10 | |
| B-Cosine | 0.458 | 18.32 | 4 | 0.0011 | 10 | 5 | am_warrant_similarity | 9/10 |
| Cond. | Config. | Basel. | metric | n_seeds | baseline_mean | am_mean | mean_delta | delta_ci95_low | delta_ci95_high | Wins | losses | ties | wilcoxon_p | cohens_dz | rank_biserial_r | sig_flag |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| B-Jaccard | NeSy + Full AM (12 features) | NeSy_Only | cv | 10 | 0.7431 | 0.7717 | 0.0286 | 0.0242 | 0.0323 | 10 | 0 | 0 | 0.001 | 4.206 | 1.0 | |
| B-Jaccard | NeSy + Full AM (12 features) | NeSy_Only | test | 10 | 0.7436 | 0.7484 | 0.0048 | -0.0187 | 0.0273 | 6 | 3 | 1 | 0.3262 | 0.123 | 0.2 | |
| B-Jaccard | NeSy + Gupta Triple (claim/premise/warrant) | NeSy_Only | cv | 10 | 0.7431 | 0.7655 | 0.0224 | 0.0134 | 0.0319 | 10 | 0 | 0 | 0.001 | 1.414 | 1.0 | |
| B-Jaccard | NeSy + Gupta Triple (claim/premise/warrant) | NeSy_Only | test | 10 | 0.7436 | 0.7613 | 0.0177 | -0.0007 | 0.0377 | 6 | 4 | 0 | 0.0967 | 0.541 | 0.491 | |
| B-Jaccard | AM Only (no NeSy base) | NeSy_Only | cv | 10 | 0.7431 | 0.6588 | -0.0843 | -0.1001 | -0.0667 | 0 | 10 | 0 | 1.0 | -2.917 | -1.0 | |
| B-Jaccard | AM Only (no NeSy base) | NeSy_Only | test | 10 | 0.7436 | 0.6506 | -0.093 | -0.1499 | -0.0414 | 2 | 8 | 0 | 0.9971 | -1.01 | -0.891 |
| rank | feature | is_am_feature | mean_importance | std | n_seeds |
|---|---|---|---|---|---|
| 1 | am_warrant_similarity | True | 0.0332 | 0.02382 | 10 |
| 2 | am_grounds_warrant_coherence | True | 0.01436 | 0.00843 | 10 |
| 3 | nesy_0 | False | 0.01332 | 0.011 | 10 |
| 4 | nesy_1 | False | 0.00904 | 0.00843 | 10 |
| 5 | nesy_25 | False | 0.00873 | 0.00779 | 10 |
| 6 | nesy_9 | False | 0.00235 | 0.00598 | 10 |