WISTERIA: Weak Implicit Signal-based Temporal Relation Extraction with Attention

Duy Dao Do; Anaïs Halftermeyer; Thi-Bich-Hanh Dao

cs.CLMar 24, 2026

WISTERIA: Weak Implicit Signal-based Temporal Relation Extraction with Attention

Authors: Duy Dao Do, Anaïs Halftermeyer, Thi-Bich-Hanh Dao

Organizations: University of Orléans, INSA Centre Val de Loire, LIFO EA 4022, France

Abstract

Temporal Relation Extraction (TRE) requires identifying how two events or temporal expressions are related in time. Existing attention-based models often highlight globally salient tokens but overlook the pair-specific cues that actually determine the temporal relation. We propose WISTERIA (Weak Implicit Signal-based Temporal Relation Extraction with Attention), a framework that examines whether the top-K attention components conditioned on each event pair truly encode interpretable evidence for temporal classification. Unlike prior works assuming explicit markers such as before, after, or when, WISTERIA considers signals as any lexical, syntactic, or morphological element implicitly expressing temporal order. By combining multi-head attention with pair-conditioned top-K pooling, the model isolates the most informative contextual tokens for each pair. We conduct extensive experiments on TimeBank-Dense, MATRES, TDDMan, and TDDAuto, including linguistic analyses of top-K tokens. Results show that WISTERIA achieves competitive accuracy and reveals pair-level rationales aligned with temporal linguistic cues, offering a localized and interpretable view of temporal reasoning.

Figures & tables

Figure 1: An example from TimeBank-Dense ( Cassidy et al., 2014 ) , based on TimeBank 1.2 ( Pustejovsky et al., 2006 ) . Explicit signals link BEFORE(takeover, news) and SIMULTANEOUS(spent, thought) , while spent-sold depends on implicit contextual cues.

Figure 2: Architecture of WISTERIA. BERT and a transformer encoder generate contextualized representations, while pair-conditioned top- $K$ cross-attention extracts key context for each entity pair. The biaffine, context, and pair-fusion outputs are integrated via late fusion for temporal relation classification.

Dataset	Train	Validation	Test	Labels
TDDMan	4000	650	1500	a, b, s, i, ii
TDDAuto	32609	1435	4258	a, b, s, i, ii
MATRES*	10404	1836	817	e, a, b, v
TBD	4032	629	1427	a, b, s, i, ii, v

Table 1: Train/Validation/Test data distribution for TDDMan, TDDAuto, MATRES, and TimeBank-Dense. Label abbreviations: a = After , b = Before , s = Simultaneous , i = Includes , ii = Is_included , v = Vague , e = Equal . (* ( Ning et al., 2019 ) use TimeBank and Aquaint for training, Platinum for testing, and 20% of the training data as validation.)

Setting	Value
Pretrained model	BERT-base-uncased
Encoder layer	1 Transformer layer
Optimizer	AdamW (fused)
Dropout rate	0.5
Learning rate	$1\times 10^{-5}$
Batch size	128

Table 2: Hyperparameter configuration for training WISTERIA.

Model	PLM	TBD	MATRES	TDDAuto	TDDMan
BiLSTM ( Cheng and Miyao, 2017 )	word2vec	0.484	0.595	0.518	0.243
BERT-based ( Ballesteros et al., 2020 )	BERT-base	0.622	0.772	0.623	0.375
Unified-Framework ( Huang et al., 2023 )	RoBERTa-Large	0.681	0.826	–	–
TIMERS ( Mathur et al., 2021 )	BERT-Large	0.678	0.823	0.711	0.455
DTRE ( Wang et al., 2022 )	BERT-base	0.692	–	0.702	0.500
MuLCo ( Yao et al., 2024 )	BERT-base	0.814	0.857	0.662	0.473

Table 3: Comparison of F1-scores across four benchmark datasets: TimeBank-Dense (TBD), MATRES, TDDAuto, and TDDMan. The best performance for each dataset is highlighted in bold.

Figure 3: Effect of Top- $K$ values on F1 performance across four datasets.

Dataset	Avg. Character Distance	Avg. Token Distance	Min Distance	Max Distance
TBD	116.99	19.51	1	366
MATRES	128.94	24.01	1	416
TDDAuto	800.99	127.61	2	3349
TDDMan	913.61	146.65	61	3074

Table 4: Average, Minimum, and Maximum Character and Token Distance between Entity Pairs across Temporal Datasets

Figure 4: Example from the TimeBank-Dense test set illustrating pair-conditioned top- $K$ attention. The model predicts BEFORE between said ( $E_{1}$ ) and force ( $E_{2}$ ). Pair-level attention highlights connective cues (e.g., as , itself ), while entity-level attention anchors event-specific tokens (e.g., visits , Saddam ). The complementary patterns indicate structured evidence selection for temporal inference.

Figure 5: POS feature distribution of TimeBank-Dense

Figure 6: POS feature distribution of MATRES

Figure 7: POS feature distribution of TDDAuto

Figure 8: POS feature distribution of TDDman

Figure 9: Dependency feature distribution of TimeBank-Dense

Figure 10: Dependency feature distribution of MATRES

Figure 11: Dependency feature distribution of TDDAuto

Figure 12: Dependency feature distribution of TDDMan

Figure 13: Morphological feature distribution of TimeBank-Dense

Figure 14: Morphological feature distribution of MATRES

Figure 15: Morphological feature distribution of TDDAuto

Figure 16: Morphological feature distribution of TDDMan

Relation Extraction

Relation extraction identifies and classifies relationships between entities or events in text and multimodal data.

See all 82 papers

Event Interaction in Low-Rank Bottlenecks for Temporal Relation Extraction
Sep 6, 2026Wei Sun, Tingyu Qu, Jesse Davis +1Relation Extraction Feature Interaction Modeling
Department of Computer Science, KU Leuven Celestijnenlaan 200A 3001 Heverlee, Belgium
Looking for the Bottleneck in Fine-grained Temporal Relation Classification
Apr 27, 2026Hugo Sousa, Ricardo Campos, Alípio JorgeRelation Extraction
University of Porto · INESC TEC · Porto, Portugal +1
Q-TIE: A Lightweight and Generalizable Re-ranking Framework for Temporal Information Retrieval
Sep 20, 2026Soyeon Kim, Hyunjin Kim, JinYeong Bak +1Temporal IR
Korea Advanced Institute of Science and Technology · Sungkyunkwan University

Sep 6, 2026J/K move · Enter open · S save

Event Interaction in Low-Rank Bottlenecks for Temporal Relation Extraction

Wei Sun, Tingyu Qu, Jesse Davis, Marie-Francine Moens

Department of Computer Science, KU Leuven Celestijnenlaan 200A 3001 Heverlee, Belgium

Relation Extraction Feature Interaction Modeling Low-Rank Adaptation Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Temporal relation extraction determines whether an event occurs before, after, or simultaneously with another event, and therefore relies on accurately modeling how the two events interact. Mainstream systems achieve this by concatenating event spans or using shallow fusion, which works well when all model parameters are trainable. However, in parameter-efficient fine-tuning, low-rank bottlenecks restrict information flow and prevent these interaction signals from passing through, leading to clear performance drops. To address this limitation, we propose a theoretically grounded architecture, Convolutional Bottleneck Interaction (CBI), which first applies lightweight depthwise convolution to enhance event representations and then uses element-wise multiplication to capture effective event-event interactions inside the bottleneck. Across five datasets and seven backbone models in the Adapter and LoRA settings, CBI provides consistent and substantial gains, up to +31.7 micro F1, while adding minimal computational cost, showing that explicit interaction inside low-rank spaces is crucial for temporal relation extraction. The code is available at https://github.com/VRCMF/CIF.git.

Abstract

Relation Extraction

Explore similar work

Event Interaction in Low-Rank Bottlenecks for Temporal Relation Extraction

Looking for the Bottleneck in Fine-grained Temporal Relation Classification

Q-TIE: A Lightweight and Generalizable Re-ranking Framework for Temporal Information Retrieval