CLUE: A Contrastive Learning Framework for Unsupervised Event Type Induction and Annotation

Pratik Maitra; Lauren Kwon; Qi Li

cs.CLOct 8, 2026

CLUE: A Contrastive Learning Framework for Unsupervised Event Type Induction and Annotation

Authors: Pratik Maitra, Lauren Kwon, Qi Li

Organizations: Iowa State University, Ames, IA, USA

Abstract

Event ontology is the foundational framework for defining events in a dataset. Event type induction generally involves domain experts who define guidelines for creating event types and their names based on the dataset's domain and scope. However, this process is resource-intensive in terms of annotation costs and is prone to human biases. To alleviate the need for manual event ontology creation, we propose the CLUE framework for event type induction, naming, and annotation. Our main contributions in this work are: 1) designing a novel joint embedding representation for expressing both the verb-centric view and the semantic view of events; 2) event type induction using contrastive loss guided clustering; 3) event naming and refinement. Experiments demonstrate the effectiveness of our method. Our code is available at https://github.com/PratikMaitra/CLUE.

Figures & tables

Sid	Sentence	P–O
S1	Bob set the table for dinner.	set - table
S2	Bob set a new record in tennis.	set - record
S3	Bob set the new record in the vinyl player.	set - record
S4	Bob arranged the table for supper.	arrange - table
S5	Bob achieved a new milestone in chess.	achieve - milestone

Figure 1: Polysemy of the verb set . Sentences S1 and S4 describe the same underlying event, as do S2 and S5. Meanwhile, S2 and S3 share a similar predicate–object pair but differ in meaning.

Figure 2: Overview of CLUE. The event type induction consists of 1) P-O pair extraction by a parser, 2) Joint instance creation, 3) BERT encoding of the joint instances, 4) Augmentation of the predicate verbs using MLM, 5) applying contrastive learning guided, 6) K-Means exemplar clustering. The event induction step is followed by 7) event type naming using an LLM and 8) event type refinement.

Dataset	Domain	#Docs	#Mentions	#Event Types	#Top Event Types
ACE	News	599	5,349	33	8
MAVEN	General	4,480	118,732	168	5
RAMS	News	3,993	9,124	139	12

Table 1: Statistics of studied event datasets.

Method	ACE				MAVEN				RAMS
	P $\uparrow$	R $\uparrow$	F1 $\uparrow$	ARI $\uparrow$	P $\uparrow$	R $\uparrow$	F1 $\uparrow$	ARI $\uparrow$	P $\uparrow$	R $\uparrow$	F1 $\uparrow$	ARI $\uparrow$
ETypeClus	0.180	0.013	0.020	0.26	0.043	0.043	0.040	0.11	0.010	0.010	0.010	0.19
CEO	0.180	0.360	0.240	0.28	0.131	0.172	0.150	0.28	0.150	0.280	0.200	0.25
CLUE-GPT-J	0.350	0.290	0.320 $\pm$ .000	0.29 $\pm$ .000	0.170	0.200	0.180 $\pm$ .000	0.33 $\pm$ .000	0.251	0.252	0.251 $\pm$ .000	0.62 $\pm$ .000
CLUE-ICL-gold	0.459	0.383	0.417 $\pm$ .014	0.31 $\pm$ .01	0.398	0.265	0.320 $\pm$ .018	0.50 $\pm$ .01	0.260	0.260	0.260 $\pm$ .010	0.62 $\pm$ .01

Table 2: Performance comparison across datasets on sentence-level micro-F1 statistics and ARI metric (higher is better). CLUE-GPT-J utilizes GPT-J model for naming. CLUE-ICL-gold uses GPT-4 Turbo for naming. Mean and s.d. values are presented.

	ACE			MAVEN			RAMS
Method	Sim dist $\uparrow$	ROUGE-L $\uparrow$	BERTScore $\uparrow$	Sim dist $\uparrow$	ROUGE-L $\uparrow$	BERTScore $\uparrow$	Sim dist $\uparrow$	ROUGE-L $\uparrow$	BERTScore $\uparrow$
most frequent	.508	.167	.869	.466	.043	.836	.448	.041	.849
tf-idf	.505	.184	.869	.464	.041	.835	.447	.038	.849
topicrank	.437	.024	.824	.380	.000	.721	.413	.006	.817
textrank	.418	.035	.813	.376	.000	.724	.399	.016	.811
keybert	.462	.072	.838	.427	.000	.795	.425	.014	.830

Table 3: Event Type Naming Quality on ACE, MAVEN, and RAMS using Sim dist, ROUGE-L, and BERTScore (higher is better). CEO baselines are from the original paper. CLUE-ICL-gold uses in-context learning for naming; CLUE-ZS names clusters zero-shot without access to any gold labels. Mean and s.d. values are presented.

Method	Prec. $\uparrow$	Rec. $\uparrow$	F1 $\uparrow$
CLUE	0.47	0.39	0.43
Cluster with P-O pairs only	0.34	0.36	0.35
Cluster with sentences only	0.40	0.35	0.37
Cluster without augmentations	0.32	0.27	0.29

Table 4: Ablation on the Representation.

Loss Function	Prec. $\uparrow$	Rec. $\uparrow$	F1 $\uparrow$
InfoNCE	0.44	0.41	0.42
Margin	0.47	0.40	0.43

Table 5: F1-scores for InfoNCE vs. Margin loss

Cluster	Total	Mean coherence	Naming quality
type	clusters	score (1–5)	Correct	Wrong	Novel
Pre-merge	50	4.06	33	11	6
Post-merge	18	4.33	12	4	2

Table 6: Case Study Results on ACE clusters before and after name-based consolidation. Event Naming Quality (total event type) is reported as counts of Correct/Wrong/Novel .

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
Epochs	5
Batch size	16
Learning rate	$1\times 10^{-5}$
Weight decay	$1\times 10^{-4}$
Margin	0.75

Appendix

Table 7: CLUE hyperparameter values used in all experiments.

Parameter	Value
LLM Model	GPT-4 Turbo
Temperature	0.3
Number of Examples	5
Example Selection	Centroid-based closest 5 sentences
Output Format	XML tags ( <EventType> )

Appendix

Table 8: Prompt settings for LLM-Based Event Naming

Figure 3: Prompt template for LLM-based event naming.

Figure 4: Prompt template for LLM-based event type definition.

Dataset	Domain	#Docs	#Mentions	#Event Types
CASIE	Cybersecurity	999	8,469	5
GENIA	Biomedical	20	6,001	13
M2E2	Multimedia	6,013	1,105	8
PHEE	Pharmacovigilance	4,827	5,019	2
WikiEvents	Wikipedia	245	3,932	50

Appendix

Table 9: Statistics of additional datasets used in our experiments.

Dataset	Prec.	Rec.	F1	ARI
CASIE	0.628	0.549	0.586	0.550
GENIA	0.610	0.180	0.278	0.579
M2E2	0.548	0.495	0.520	0.298
PHEE	0.863	0.845	0.854	0.297
WIKIEVENTS	0.730	0.235	0.355	0.490

Appendix

Table 10: Additional Dataset Evaluations on micro-F1 and ARI

Dataset	Sim_dist	ROUGE-L	BERT Score
CASIE	0.840	0.430	0.880
GENIA	0.980	0.970	0.990
M2E2	0.867	0.738	0.929
PHEE	1.000	1.000	1.000
WIKIEVENTS	0.900	0.740	0.950

Appendix

Table 11: Additional Evaluation on Event Naming

Dataset	Total Event Types	$k$
ACE	33	50
MAVEN	168	252
RAMS	139	209
CASIE	5	8
M2E2	8	12
PHEE	2	3

Appendix

Table 12: Number of Total Event Types and value of

k

(number of clusters) used for each dataset.

Model	Sim_dist $\uparrow$	ROUGE-L $\uparrow$	BERT Score $\uparrow$
GPT-5.4	0.921	0.791	0.975
Gemini 3.1 Pro	0.904	0.750	0.981
GPT-4 Turbo	0.900	0.731	0.970
Deepseek-v3.2	0.871	0.661	0.971
Qwen-3 VL 32B-Instruct	0.834	0.594	0.956
Claude Opus-4.6	0.832	0.592	0.954

Appendix

Table 13: Performance comparison across LLMs on the event naming task. Sim_dist, ROUGE-L, and BERT Score are reported (higher is better). Best results per column are in bold .

$k$	Factor	P	R	F1
33	1.00	0.450	0.370	0.406
42	1.25	0.501	0.401	0.445
50	1.50	0.505	0.415	0.456
66	2.00	0.483	0.400	0.438
83	2.50	0.497	0.410	0.450

Appendix

Table 14: Ablation results across different values of

k

on ACE dataset, which has 33 event types. The Factor column denotes

k*33

. Precision (P), Recall (R), and F1 scores are reported. Best results are in bold .

Setting	Coverage of Major Event Types	Micro-F1
$t=0.55$	1/8	0.04
$t=0.70$	5/8	0.17
$t=0.85$ (baseline)	8/8	0.42
Exact Match	8/8	0.37

Appendix

Table 15: Ablation over the merge threshold

t

on the ACE dataset, compared against exact-match merging. ACE has 8 major event types. Lowering

t

merges clusters more aggressively and reduces the coverage of these major event types, while higher values of

t

recover full coverage. Micro-F1 is aggregated over all evaluated sentences. Best result in bold .

Dataset	Unique Sentences (Filtered)	Total Token Count of Unique Sentences	$k$	Sentences sampled per cluster for Prompting	Total Sentences used for Prompting	Prompt Tokens used (incl. prompt text)
ACE	6,676	144,259	50	5	250	21,355
MAVEN	25,235	614,713	252	5	1,280	188,521
RAMS	7,417	991,029	209	5	1,045	282,787

Appendix

Table 16: Token and sentence usage statistics per dataset. Unique Sentences (Filtered) is the number of unique sentences remaining after filtering, and Total Token Count of Unique Sentences is the total number of tokens across them.

k

is the number of clusters; CLUE samples 5 sentences from each cluster. Prompt Tokens used is the total number of tokens across all naming and definition prompts, including the full prompt text. Tokens are counted as whitespace-separated words.

CLUE annotated Event Cluster Name	Ground Truth Event Type Name of majority of the sampled sentences	Human annotated Event Cluster Name	Sampled Sentences
Government: Elect	Personnel: Elect	election, democrats/justice	(1) On the other hand, I have yet to see any Democrats challenge incumbents for being fiscally irresponsible. (2) I think it’s important to recognize that African Americans, though still overwhelmingly Democratic, comprise less of a monolith than they did 20–30 years ago. (3) Did you vote in the recent US Presidential election? (4) Our alliance to them is supporting people that want to have a free democratically elected gov’t. (5) I actually can’t recall it ever taking 2 weeks to “count” these many ballots, generally elections are called within 24 hours.
Conflict: Use-Conventional-Force	Conflict: Attack	nuclear, weapon/conflict	(1) A senior lawmaker seen as close to the Kremlin ridiculed the United States Sunday for failing to find any weapons of mass destruction in Iraq. (2) A North Korean foreign ministry spokesman, quoted by the official Korean Central News Agency, made no reference to the reports that North Korea had admitted to possessing nuclear weapons. (3) Rodong called for a “fair and equal” footing in talks with the United States, blasting Washington for insisting that there would be “no security of the system nor provision of rewards” even if Pyongyang gave up its nuclear program. (4) The plant, based on Russian design and upgraded with U.S. technology, has been plagued by frequent, non-nuclear malfunctions since testing began in November 2000. (5) North Korea has never tested a nuclear weapon, though the CIA long ago estimated that it manufactured two crude devices in the late 1980s or early 1990s.

Appendix

Table 17: Example clusters showing CLUE-assigned label, ground truth event type name for the majority of the sampled sentences, human annotated cluster name, and the five sampled sentences.

Ontology Learning

Ontology learning develops methods for acquiring, extending, and aligning formal concepts and relationships from text, data, and expert examples.

See all 30 papers

Event Ontology Expansion via LLM-Based Conceptualization
Jun 19, 2026Weicheng Ren, Zixuan Li, Long Bai +3Clustering Ontology Learning
State Key Laboratory of AI Safety · Institute of Computing Technology, Chinese Academy of Sciences
EVENT5Ws: A Large Dataset for Open-Domain Event Extraction from Documents
Apr 23, 2026Praval Sharma, Ashok Samal, Leen-Kiat Soh +1Document Information Extraction Event Extraction
University of Nebraska Omaha, USA · University of Nebraska–Lincoln, USA · The Citadel, USA
A Scalable Cross-Domain Event Extraction System via a Unified Generative Training Framework
Aug 24, 2026Siting Liang, Omar Adjali, Omair Shahzad Bhatti +1Event Extraction Document Information Extraction
German Research Center for Artificial Intelligence · Carl von Ossietzky University of Oldenburg

Jun 19, 2026J/K move · Enter open · S save

Event Ontology Expansion via LLM-Based Conceptualization

Weicheng Ren, Zixuan Li, Long Bai, Xiaolong Jin+2

State Key Laboratory of AI Safety · Institute of Computing Technology, Chinese Academy of Sciences

Clustering Ontology Learning

Open paper AskPDF arXiv

Event ontology expansion aims to discover emerging event types from data and extend them to appropriate positions in the existing event ontology.. Existing methods typically cluster contextualized trigger representations and attach induced clusters to the ontology based on instance-level similarity. However, ontology expansion requires concept-level semantics that characterize event types, whereas contextualized trigger representations often conflate these semantics with surface contextual variation, leading to unstable clustering and unreliable hierarchy expansion. To address this issue, we propose ConceptE, a conceptualization-enhanced framework for event ontology expansion. ConceptE first derives concept-level semantics by prompting an LLM with the sentence and event trigger, producing a concise concept name and a natural-language description. It then jointly encodes these semantics with trigger information to build concept-enhanced representations aligned with ontology-level reasoning. This representation design supports more coherent event clustering, more reliable hierarchy expansion, and ontology-consistent type naming. Experiments on ACE, ERE, and MAVEN demonstrate that ConceptE consistently outperforms state-of-the-art approaches across all subtasks of event ontology expansion. In particular, it achieves improvements of up to 12.37% in BCubed-F1 for event clustering and 6.48% in Taxo_F1 for hierarchy expansion, demonstrating the effectiveness of the proposed ConceptE method.

Abstract

Ontology Learning

Explore similar work

Event Ontology Expansion via LLM-Based Conceptualization

EVENT5Ws: A Large Dataset for Open-Domain Event Extraction from Documents

A Scalable Cross-Domain Event Extraction System via a Unified Generative Training Framework