Event ontology is the foundational framework for defining events in a dataset. Event type induction generally involves domain experts who define guidelines for creating event types and their names based on the dataset's domain and scope. However, this process is resource-intensive in terms of annotation costs and is prone to human biases. To alleviate the need for manual event ontology creation, we propose the CLUE framework for event type induction, naming, and annotation. Our main contributions in this work are: 1) designing a novel joint embedding representation for expressing both the verb-centric view and the semantic view of events; 2) event type induction using contrastive loss guided clustering; 3) event naming and refinement. Experiments demonstrate the effectiveness of our method. Our code is available at https://github.com/PratikMaitra/CLUE.
Figures & tables
Sid
Sentence
P–O
S1
Bob set the table for dinner.
set - table
S2
Bob set a new record in tennis.
set - record
S3
Bob set the new record in the vinyl player.
set - record
S4
Bob arranged the table for supper.
arrange - table
S5
Bob achieved a new milestone in chess.
achieve - milestone
Figure 1: Polysemy of the verb set . Sentences S1 and S4 describe the same underlying event, as do S2 and S5. Meanwhile, S2 and S3 share a similar predicate–object pair but differ in meaning.
Figure 2: Overview of CLUE. The event type induction consists of 1) P-O pair extraction by a parser, 2) Joint instance creation, 3) BERT encoding of the joint instances, 4) Augmentation of the predicate verbs using MLM, 5) applying contrastive learning guided, 6) K-Means exemplar clustering. The event induction step is followed by 7) event type naming using an LLM and 8) event type refinement.
Dataset
Domain
#Docs
#Mentions
#Event Types
#Top Event Types
ACE
News
599
5,349
33
8
MAVEN
General
4,480
118,732
168
5
RAMS
News
3,993
9,124
139
12
Table 1: Statistics of studied event datasets.
Method
ACE
MAVEN
RAMS
P ↑
R ↑
F1 ↑
ARI ↑
P ↑
R ↑
F1 ↑
ARI ↑
P ↑
R ↑
F1 ↑
ARI ↑
ETypeClus
0.180
0.013
0.020
0.26
0.043
0.043
0.040
0.11
0.010
0.010
0.010
0.19
CEO
0.180
0.360
0.240
0.28
0.131
0.172
0.150
0.28
0.150
0.280
0.200
0.25
CLUE-GPT-J
0.350
0.290
0.320 ± .000
0.29 ± .000
0.170
0.200
0.180 ± .000
0.33 ± .000
0.251
0.252
0.251 ± .000
0.62 ± .000
CLUE-ICL-gold
0.459
0.383
0.417 ± .014
0.31 ± .01
0.398
0.265
0.320 ± .018
0.50 ± .01
0.260
0.260
0.260 ± .010
0.62 ± .01
Table 2: Performance comparison across datasets on sentence-level micro-F1 statistics and ARI metric (higher is better). CLUE-GPT-J utilizes GPT-J model for naming. CLUE-ICL-gold uses GPT-4 Turbo for naming. Mean and s.d. values are presented.
ACE
MAVEN
RAMS
Method
Sim dist ↑
ROUGE-L ↑
BERTScore ↑
Sim dist ↑
ROUGE-L ↑
BERTScore ↑
Sim dist ↑
ROUGE-L ↑
BERTScore ↑
most frequent
.508
.167
.869
.466
.043
.836
.448
.041
.849
tf-idf
.505
.184
.869
.464
.041
.835
.447
.038
.849
topicrank
.437
.024
.824
.380
.000
.721
.413
.006
.817
textrank
.418
.035
.813
.376
.000
.724
.399
.016
.811
keybert
.462
.072
.838
.427
.000
.795
.425
.014
.830
Table 3: Event Type Naming Quality on ACE, MAVEN, and RAMS using Sim dist, ROUGE-L, and BERTScore (higher is better). CEO baselines are from the original paper. CLUE-ICL-gold uses in-context learning for naming; CLUE-ZS names clusters zero-shot without access to any gold labels. Mean and s.d. values are presented.
Method
Prec. ↑
Rec. ↑
F1 ↑
CLUE
0.47
0.39
0.43
Cluster with P-O pairs only
0.34
0.36
0.35
Cluster with sentences only
0.40
0.35
0.37
Cluster without augmentations
0.32
0.27
0.29
Table 4: Ablation on the Representation.
Loss Function
Prec. ↑
Rec. ↑
F1 ↑
InfoNCE
0.44
0.41
0.42
Margin
0.47
0.40
0.43
Table 5: F1-scores for InfoNCE vs. Margin loss
Cluster
Total
Mean coherence
Naming quality
type
clusters
score (1–5)
Correct
Wrong
Novel
Pre-merge
50
4.06
33
11
6
Post-merge
18
4.33
12
4
2
Table 6: Case Study Results on ACE clusters before and after name-based consolidation. Event Naming Quality (total event type) is reported as counts of Correct/Wrong/Novel .
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Hyperparameter
Value
Epochs
5
Batch size
16
Learning rate
1×10−5
Weight decay
1×10−4
Margin
0.75
Appendix
Table 7: CLUE hyperparameter values used in all experiments.
Parameter
Value
LLM Model
GPT-4 Turbo
Temperature
0.3
Number of Examples
5
Example Selection
Centroid-based closest 5 sentences
Output Format
XML tags ( <EventType> )
Appendix
Table 8: Prompt settings for LLM-Based Event Naming
Figure 3: Prompt template for LLM-based event naming.
Figure 4: Prompt template for LLM-based event type definition.
Dataset
Domain
#Docs
#Mentions
#Event Types
CASIE
Cybersecurity
999
8,469
5
GENIA
Biomedical
20
6,001
13
M2E2
Multimedia
6,013
1,105
8
PHEE
Pharmacovigilance
4,827
5,019
2
WikiEvents
Wikipedia
245
3,932
50
Appendix
Table 9: Statistics of additional datasets used in our experiments.
Dataset
Prec.
Rec.
F1
ARI
CASIE
0.628
0.549
0.586
0.550
GENIA
0.610
0.180
0.278
0.579
M2E2
0.548
0.495
0.520
0.298
PHEE
0.863
0.845
0.854
0.297
WIKIEVENTS
0.730
0.235
0.355
0.490
Appendix
Table 10: Additional Dataset Evaluations on micro-F1 and ARI
Dataset
Sim_dist
ROUGE-L
BERT Score
CASIE
0.840
0.430
0.880
GENIA
0.980
0.970
0.990
M2E2
0.867
0.738
0.929
PHEE
1.000
1.000
1.000
WIKIEVENTS
0.900
0.740
0.950
Appendix
Table 11: Additional Evaluation on Event Naming
Dataset
Total Event Types
k
ACE
33
50
MAVEN
168
252
RAMS
139
209
CASIE
5
8
M2E2
8
12
PHEE
2
3
Appendix
Table 12: Number of Total Event Types and value of k (number of clusters) used for each dataset.
Model
Sim_dist ↑
ROUGE-L ↑
BERT Score ↑
GPT-5.4
0.921
0.791
0.975
Gemini 3.1 Pro
0.904
0.750
0.981
GPT-4 Turbo
0.900
0.731
0.970
Deepseek-v3.2
0.871
0.661
0.971
Qwen-3 VL 32B-Instruct
0.834
0.594
0.956
Claude Opus-4.6
0.832
0.592
0.954
Appendix
Table 13: Performance comparison across LLMs on the event naming task. Sim_dist, ROUGE-L, and BERT Score are reported (higher is better). Best results per column are in bold .
k
Factor
P
R
F1
33
1.00
0.450
0.370
0.406
42
1.25
0.501
0.401
0.445
50
1.50
0.505
0.415
0.456
66
2.00
0.483
0.400
0.438
83
2.50
0.497
0.410
0.450
Appendix
Table 14: Ablation results across different values of k on ACE dataset, which has 33 event types. The Factor column denotes k∗33 . Precision (P), Recall (R), and F1 scores are reported. Best results are in bold .
Setting
Coverage of Major Event Types
Micro-F1
t=0.55
1/8
0.04
t=0.70
5/8
0.17
t=0.85 (baseline)
8/8
0.42
Exact Match
8/8
0.37
Appendix
Table 15: Ablation over the merge threshold t on the ACE dataset, compared against exact-match merging. ACE has 8 major event types. Lowering t merges clusters more aggressively and reduces the coverage of these major event types, while higher values of t recover full coverage. Micro-F1 is aggregated over all evaluated sentences. Best result in bold .
Dataset
Unique Sentences (Filtered)
Total Token Count of Unique Sentences
k
Sentences sampled per cluster for Prompting
Total Sentences used for Prompting
Prompt Tokens used (incl. prompt text)
ACE
6,676
144,259
50
5
250
21,355
MAVEN
25,235
614,713
252
5
1,280
188,521
RAMS
7,417
991,029
209
5
1,045
282,787
Appendix
Table 16: Token and sentence usage statistics per dataset. Unique Sentences (Filtered) is the number of unique sentences remaining after filtering, and Total Token Count of Unique Sentences is the total number of tokens across them. k is the number of clusters; CLUE samples 5 sentences from each cluster. Prompt Tokens used is the total number of tokens across all naming and definition prompts, including the full prompt text. Tokens are counted as whitespace-separated words.
CLUE annotated Event Cluster Name
Ground Truth Event Type Name of majority of the sampled sentences
Human annotated Event Cluster Name
Sampled Sentences
Government: Elect
Personnel: Elect
election, democrats/justice
(1) On the other hand, I have yet to see any Democrats challenge incumbents for being fiscally irresponsible. (2) I think it’s important to recognize that African Americans, though still overwhelmingly Democratic, comprise less of a monolith than they did 20–30 years ago. (3) Did you vote in the recent US Presidential election? (4) Our alliance to them is supporting people that want to have a free democratically elected gov’t. (5) I actually can’t recall it ever taking 2 weeks to “count” these many ballots, generally elections are called within 24 hours.
Conflict: Use-Conventional-Force
Conflict: Attack
nuclear, weapon/conflict
(1) A senior lawmaker seen as close to the Kremlin ridiculed the United States Sunday for failing to find any weapons of mass destruction in Iraq. (2) A North Korean foreign ministry spokesman, quoted by the official Korean Central News Agency, made no reference to the reports that North Korea had admitted to possessing nuclear weapons. (3) Rodong called for a “fair and equal” footing in talks with the United States, blasting Washington for insisting that there would be “no security of the system nor provision of rewards” even if Pyongyang gave up its nuclear program. (4) The plant, based on Russian design and upgraded with U.S. technology, has been plagued by frequent, non-nuclear malfunctions since testing began in November 2000. (5) North Korea has never tested a nuclear weapon, though the CIA long ago estimated that it manufactured two crude devices in the late 1980s or early 1990s.
Appendix
Table 17: Example clusters showing CLUE-assigned label, ground truth event type name for the majority of the sampled sentences, human annotated cluster name, and the five sampled sentences.
Event ontology expansion aims to discover emerging event types from data and extend them to appropriate positions in the existing event ontology.. Existing methods typically cluster contextualized trigger representations and attach induced clusters to the ontology based on instance-level similarity. However, ontology expansion requires concept-level semantics that characterize event types, whereas contextualized trigger representations often conflate these semantics with surface contextual variation, leading to unstable clustering and unreliable hierarchy expansion. To address this issue, we propose ConceptE, a conceptualization-enhanced framework for event ontology expansion. ConceptE first derives concept-level semantics by prompting an LLM with the sentence and event trigger, producing a concise concept name and a natural-language description. It then jointly encodes these semantics with trigger information to build concept-enhanced representations aligned with ontology-level reasoning. This representation design supports more coherent event clustering, more reliable hierarchy expansion, and ontology-consistent type naming. Experiments on ACE, ERE, and MAVEN demonstrate that ConceptE consistently outperforms state-of-the-art approaches across all subtasks of event ontology expansion. In particular, it achieves improvements of up to 12.37% in BCubed-F1 for event clustering and 6.48% in Taxo_F1 for hierarchy expansion, demonstrating the effectiveness of the proposed ConceptE method.
Weicheng Ren, Zixuan Li, Long Bai +3
State Key Laboratory of AI Safety · Institute of Computing Technology, Chinese Academy of Sciences
Event extraction identifies the central aspects of events from text. It supports event understanding and analysis, which is crucial for tasks such as informed decision-making in emergencies. Therefore, it is necessary to develop automated event extraction approaches. However, existing datasets for algorithm development have limitations, including limited coverage of event types in closed-domain settings and a lack of large, manually verified dataset in open-domain settings. To address these limitations, we create EVENT5Ws , a large, manually annotated, and statistically verified open-domain event extraction dataset. We design a systematic annotation pipeline to create the dataset and provide empirical insights into annotation complexity. Using EVENT5Ws, we evaluate state-of-the-art pre-trained large language models and establish a benchmark for future research. We further show that models trained on EVENT5Ws generalize effectively to datasets from different geographical contexts, which demonstrates its potential for developing generalizable algorithms. Finally, we summarize the lessons learned during the dataset development and provide recommendations to support future large-scale dataset development.
Praval Sharma, Ashok Samal, Leen-Kiat Soh +1
University of Nebraska Omaha, USA · University of Nebraska–Lincoln, USA · The Citadel, USA
Event extraction is fundamental to information extraction. Prior approaches often separate event detection and argument extraction or depend on dataset-specific designs, limiting scalability and cross-domain generalization. We propose a unified generative sequence-to-sequence framework that performs event extraction subtasks jointly and supports both pipeline and end-to-end configurations. We fine-tune pretrained language models on multiple event datasets across diverse domains, enabling a single model to retain domain-specific semantics while generalizing over large and evolving label spaces. We demonstrate these capabilities through a web-based application tailored for researchers and practitioners. The platform supports document upload, schema-aware event extraction, visualization of triggers and arguments, and comparison of different extraction configurations across domains.