ARGUS: Role-Aware Event Knowledge Graphs for U.S. Employment-Discrimination Complaints

Sriram Kannan; Swetha Saseendran; Vishnu Vardhan Reddy Kandi; Leslie Barrett; Madhavan Seshadri; Enrico Santus

cs.CLSep 24, 2026

ARGUS: Role-Aware Event Knowledge Graphs for U.S. Employment-Discrimination Complaints

Authors: Sriram Kannan, Swetha Saseendran, Vishnu Vardhan Reddy Kandi, Leslie Barrett, Madhavan Seshadri, Enrico Santus

Organizations: University of Massachusetts Amherst · Bloomberg

Abstract

U.S. employment-discrimination complaints describe complex event sequences that are not explicitly captured by lexical or embedding-based representations alone. We present ARGUS, a source-grounded pipeline that combines a 5W1H-inspired schema, legal-domain models, and LLM-based structured generation to construct document-level Event Knowledge Graphs (EKGs) from CourtListener complaints. ARGUS extracts fact-bearing statements, builds chunk-level event graphs with participant, temporal, and causal structure, and merges them into document-level representations. We evaluate graph quality through human and multi-model assessment and test downstream utility on claim classification and legal QA. The graph-structured classifier outperforms raw and linearized baselines on the held-out set, and EKG-only retrieval improves document-scoped QA, while open-retrieval gains remain limited by low first-stage candidate recall. These results suggest that EKGs are most useful for organizing and reasoning over evidence once relevant material has been retrieved.

Figures & tables

Figure 1: Multi-stage pipeline for EKG construction from federal employment-discrimination complaint narratives.

District	Docs	Chunks	Avg/Doc	Sentences	Fact Rate
EAPD	20	87	4.35	–	–
KYED	31	138	4.45	1,155	81.7%
NYSD	46	318	6.91	4,808	92.7%
PAWD	28	165	5.89	1,607	93.0%
RID	28	118	4.21	1,631	90.9%

Table 1: Dataset statistics across districts. Fact Rate is the fraction of sentences labeled Fact; sentence-level annotation is not yet complete for EAPD.

Model	Acc.	Prec.	Rec.	F1
LEGAL-BERT	0.9400	0.8889	0.6154	0.7273
LexLM	0.9300	0.8000	0.6154	0.6957
GPT-5	0.8500	0.4615	0.9231	0.6154

Table 2: Fact-extraction results on a fixed 100-sentence U.S. gold sample. LEGAL-BERT and LexLM are fine-tuned legal-domain encoders; GPT-5 is a prompted baseline. Fact is the positive class.

Metric	GPT-OSS	Qwen
Quality Score	0.8272	0.8852
Judge Score	0.6642	0.7966
Structural Score	0.9902	0.9739
Temporal Consistency	0.9118	0.7647
Events/Chunk	7.09	7.50
Temporal Edges/Chunk	2.15	3.12

Table 3: Open-source model comparison for mini-graph extraction.

Metric	Claude	Qwen
Aligned	78.30%	85.20%
Partially Aligned	17.40%	14.80%
Misaligned	4.30%	0.00%
Percent Agreement	69.60%	91.70%
Fleiss’ $\kappa$	0.143	0.669
Krippendorff’s $\alpha$	0.153	0.673

Table 4: Human evaluation of mini-graph alignment with the source text.

Case	Pre-Ev	Post-Ev	Loss	Pre-En	Post-En
Miczulski v. Alix	27	26	1	31	13
Poole v. Ampler	40	40	0	34	17
Sermarini v. Step Up	35	35	0	26	10
Bleiler v. Chester	27	27	0	24	13
Bacon v. M.A.G.	5	5	0	6	6

Table 5: Event and entity counts before and after document-level merging on five validation cases.

Case	Events	Temporal	Causal
Miczulski v. Alix Inc	21	19	6
Poole v. Ampler Pizza	0	0	0
Sermarini v. A Step Up	16	14	20
Bleiler v. Chester Co.	0	0	0
Bacon v. M.A.G. Ent.	17	16	13

Table 6: One-shot LLM baseline event-graph extraction on the five merger-validation documents.

Dimension	Win	Agree.	AC1	$p$
Event Grounding	93.3%	86.7%	0.848	0.031
Evidence Grounding	100%	100%	1.0	0.031
Edge Correctness	100%	100%	1.0	0.031
Overall	100%	100%	1.0	0.031

Table 7: Multi-model evaluation of ARGUS relative to the one-shot baseline.

Representation	Micro-F1@0.5
Raw-text LEGAL-BERT	0.5897
Linearized-EKG LEGAL-BERT	0.5833
EKG-GNN	0.6415

Table 8: SALI claim-type classification on the 22-document held-out test set.

Context condition	Mean Token-F1
Raw document	0.323
Hybrid text + EKG	0.363
EKG only	0.446

Table 9: Document-scoped FAQ answer quality over 300 questions from 10 CourtListener complaints (GPT-4o).

Retrieval method	Recall@10
BM25	0.0100
E5-large-v2	0.0100
Hybrid Graph-RAG	0.0800

Table 10: Open-retrieval performance on the 100-question Bar Exam QA subset.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: Two-stage mini-graph extractor with GEPA-driven optimization. Stage 1 extracts grounded entities and events; Stage 2 adds temporal and causal relations.

Figure 3: GEPA candidate optimization loop: seed, evaluate, reflect, mutate, and retain Pareto-optimal configurations.

Figure 4: Example of an event mini-graph extracted from a single chunk.

Model	Stability	Flip	AC1
DeepSeek	80%	20%	1.0
Qwen	100%	0%	1.0
Mistral	100%	0%	1.0

Appendix

Table 11: Intra-model stability under temperature variation.

Mode	LEGAL-BERT	LexLM
Classifier	0.8718	0.8616
Hybrid	0.8711	0.8627
LLM	0.7115	0.7186

Appendix

Table 12: U.S. holdout Fact-class F1 by fact-extraction mode.

Feature set	Silhouette
TF-IDF baseline	0.091
EKG full (2k dims)	0.363
EKG Legacy7D	0.465
EKG Core6D ( $K=4$ )	0.649

Appendix

Table 13: Clustering silhouette by feature configuration.

Run	Representation	Sizes ( $K=4$ )	Pearson	Spearman
A	SALI(23) + Core6D	111/14/6/2	$\approx-0.06$	–
B	Core6D + Hybrid PCA(16)	balanced, 30s–50s	$\approx-0.28$	$\approx-0.31$
C	A + Hybrid PCA + $k$ NN(12)	31/61/19/22	$\approx-0.29$	$\approx-0.37$

Appendix

Table 14: Cluster validation across three feature configurations. Correlations compare centroid distance with LLM-judged cluster membership; more negative values indicate stronger association between proximity to the centroid and judged membership. Validation

n=40

for Runs B and C and

n\approx 17

for Run A.

Feature	A	B	C
SALI claim tags	yes	no	yes
Core6D structure	yes	yes	yes
Hybrid PCA context	no	yes	yes
Graph neighbors ( $k$ NN)	no	no	yes

Appendix

Table 15: Features included in each clustering configuration.

Figure 5: Run C fused representation: Core6D, hybrid PCA context, a type-Jaccard graph-neighbor component, and LLM-assigned SALI tags are combined into a single vector $Z$ before $K$ -means clustering ( $K=4$ ).

Figure 6: Run C validation: centroid distance versus LLM-judged cluster membership ( $n=40$ , Pearson $r\approx-0.29$ , Spearman $\rho\approx-0.37$ ).

Question type	Raw F1	EKG F1
Temporal order	0.381	0.538
Causal/enablement	0.387	0.522
Temporal overlap	0.241	0.384
Party resolution	0.152	0.197

Appendix

Table 16: FAQ performance by question type.

Comparison	$\Delta$ F1	Bootstrap 95% CI	Wilcoxon $p$
EKG $-$ Raw	+0.123	[+0.109, +0.137]	$3.4\times 10^{-39}$
EKG $-$ Hybrid	+0.083	[+0.069, +0.096]	$2.3\times 10^{-27}$
Hybrid $-$ Raw	+0.040	[+0.030, +0.052]	$8.5\times 10^{-11}$

Appendix

Table 17: Paired significance tests for the FAQ benchmark (

n=299

). All comparisons are significant at

\alpha=0.05

; sign-flip permutation tests agree in every comparison (

p<10^{-3}

, not shown).

Question type	$n$	$\Delta$ F1 (EKG $-$ Raw)	Wilcoxon $p$
Temporal order	119	+0.157	$1.0\times 10^{-18}$
Causal/enablement	99	+0.134	$2.7\times 10^{-16}$
Temporal overlap	12	+0.143	$2.4\times 10^{-3}$
Party resolution	66	+0.045	$8.0\times 10^{-5}$
Temporal relation	3	+0.019	0.25

Appendix

Table 18: EKG-versus-raw FAQ results by question type. Categories with

n\geq 12

are significant at

\alpha=0.05

; no conclusion is drawn for the temporal-relation category (

n=3

Retrieval condition	MC accuracy
Raw document (top 3)	0.8547
Chunked text (top 3)	0.8632
EKG graph (top 3)	0.8632
Gold EKG subset ( $n=100$ )	0.8600

Appendix

Table 19: MBE multiple-choice accuracy by retrieval condition in Experiment A (

n=117

, Claude Sonnet 5).

Comparison	McNemar $p$	Bootstrap 95% CI	Wilcoxon $p$
Document vs. chunk	1.000	$[-5.13,+3.42]$ pp	0.813
Document vs. EKG	1.000	$[-5.98,+4.27]$ pp	0.820
Chunk vs. EKG	0.688	$[-4.27,+4.27]$ pp	1.000

Appendix

Table 20: Pairwise significance tests for MBE accuracy in Experiment A. No comparison is significant at

\alpha=0.05

Context condition	MCQ accuracy
No passage	0.7600
BM25-RAG (open retrieval, top 10)	0.7300
Hybrid Graph-RAG (open retrieval, top 10)	0.7200
Gold passage	0.7900
Original oracle-document Graph-RAG	0.7500
Improved oracle-document Graph-RAG	0.8200

Appendix

Table 21: Downstream Bar Exam multiple-choice accuracy in Experiment B (Claude Sonnet 4.5).

Explore similar work

Schema-Constrained Document-Level Event Argument Extraction with Lightweight LLM Fine-Tuning
Jul 18, 2026Roberto Pietrantuono, Antonio Guerriero, Pouya SattariCoreference Resolution Large Language Model Fine-Tuning
University of Naples Federico II, Naples, Italy · University of Salerno, Fisciano, Italy
LegalGraphRAG: Multi-Agent Graph Retrieval-Augmented Generation for Reliable Legal Reasoning
May 27, 2026Zerui Chen, Qinggang Zhang, Zhishang Xiang +5Legal Reasoning Tasks Graph-Llm Integrations
School of Informatics, Xiamen University · The Hong Kong Polytechnic University · Institute of Artificial Intelligence, Xiamen University
RAGU: A Multi-Step GraphRAG Engine with a Compact Domain-Adapted LLM
Jul 13, 2026Mikhail Komarov, Ivan Bondarenko, Stanislav Shtuka +5Graph-Based Retrieval Augmented Generation Knowledge Graphs
1ITMO University · 2Novosibirsk State University · 3Far Eastern Federal University +1

Jul 18, 2026, cs.CLJ/K move · Enter open · S save

Schema-Constrained Document-Level Event Argument Extraction with Lightweight LLM Fine-Tuning

Roberto Pietrantuono, Antonio Guerriero, Pouya Sattari

University of Naples Federico II, Naples, Italy · University of Salerno, Fisciano, Italy

Coreference Resolution Large Language Model Fine-Tuning Lightweight

Open paper AskPDF arXiv

Event Argument Extraction (EAE) converts documents into structured event records by identifying argument spans and assigning them schema-defined roles. Document-level EAE is challenging due to long-range dependencies between triggers and arguments, cross-sentence context, and strict role constraints, which often lead to boundary errors, uncertainty in roles, and inconsistencies with restricted schemas. In this paper, we study whether mid-sized open LLMs can perform schema-constrained EAE reliably at the document level on MAVEN-ARG. Our approach combines (i) role-set injection in prompts for schema compliance, (ii) parameter-efficient supervised fine-tuning (LoRA) using the same JSON-only interface used at inference, and (iii) deterministic decoding with post-processing that validates JSON, filters invalid roles, de-duplicates arguments, and aligns spans to the document window. Under the official MAVEN-ARG evaluator, fine-tuned mid-sized open models outperform previously reported GPT baselines across mention, entity-coreference, and event-coreference evaluations; our best model (Phi-4, 14B) reaches 42.39% F1 at the event-coreference level. Code to reproduce experiments is publicly available at https://github.com/dessertlab/EAE/.

Abstract

Explore similar work

Schema-Constrained Document-Level Event Argument Extraction with Lightweight LLM Fine-Tuning

LegalGraphRAG: Multi-Agent Graph Retrieval-Augmented Generation for Reliable Legal Reasoning

RAGU: A Multi-Step GraphRAG Engine with a Compact Domain-Adapted LLM