Temporal Relation Extraction (TRE) requires identifying how two events or temporal expressions are related in time. Existing attention-based models often highlight globally salient tokens but overlook the pair-specific cues that actually determine the temporal relation. We propose WISTERIA (Weak Implicit Signal-based Temporal Relation Extraction with Attention), a framework that examines whether the top-K attention components conditioned on each event pair truly encode interpretable evidence for temporal classification. Unlike prior works assuming explicit markers such as before, after, or when, WISTERIA considers signals as any lexical, syntactic, or morphological element implicitly expressing temporal order. By combining multi-head attention with pair-conditioned top-K pooling, the model isolates the most informative contextual tokens for each pair. We conduct extensive experiments on TimeBank-Dense, MATRES, TDDMan, and TDDAuto, including linguistic analyses of top-K tokens. Results show that WISTERIA achieves competitive accuracy and reveals pair-level rationales aligned with temporal linguistic cues, offering a localized and interpretable view of temporal reasoning.
Figures & tables
Figure 1: An example from TimeBank-Dense ( Cassidy et al., 2014 ) , based on TimeBank 1.2 ( Pustejovsky et al., 2006 ) . Explicit signals link BEFORE(takeover, news) and SIMULTANEOUS(spent, thought) , while spent-sold depends on implicit contextual cues.
Figure 2: Architecture of WISTERIA. BERT and a transformer encoder generate contextualized representations, while pair-conditioned top- K cross-attention extracts key context for each entity pair. The biaffine, context, and pair-fusion outputs are integrated via late fusion for temporal relation classification.
Dataset
Train
Validation
Test
Labels
TDDMan
4000
650
1500
a, b, s, i, ii
TDDAuto
32609
1435
4258
a, b, s, i, ii
MATRES*
10404
1836
817
e, a, b, v
TBD
4032
629
1427
a, b, s, i, ii, v
Table 1: Train/Validation/Test data distribution for TDDMan, TDDAuto, MATRES, and TimeBank-Dense. Label abbreviations: a = After , b = Before , s = Simultaneous , i = Includes , ii = Is_included , v = Vague , e = Equal . (* ( Ning et al., 2019 ) use TimeBank and Aquaint for training, Platinum for testing, and 20% of the training data as validation.)
Setting
Value
Pretrained model
BERT-base-uncased
Encoder layer
1 Transformer layer
Optimizer
AdamW (fused)
Dropout rate
0.5
Learning rate
1×10−5
Batch size
128
Table 2: Hyperparameter configuration for training WISTERIA.
Model
PLM
TBD
MATRES
TDDAuto
TDDMan
BiLSTM ( Cheng and Miyao, 2017 )
word2vec
0.484
0.595
0.518
0.243
BERT-based ( Ballesteros et al., 2020 )
BERT-base
0.622
0.772
0.623
0.375
Unified-Framework ( Huang et al., 2023 )
RoBERTa-Large
0.681
0.826
–
–
TIMERS ( Mathur et al., 2021 )
BERT-Large
0.678
0.823
0.711
0.455
DTRE ( Wang et al., 2022 )
BERT-base
0.692
–
0.702
0.500
MuLCo ( Yao et al., 2024 )
BERT-base
0.814
0.857
0.662
0.473
Table 3: Comparison of F1-scores across four benchmark datasets: TimeBank-Dense (TBD), MATRES, TDDAuto, and TDDMan. The best performance for each dataset is highlighted in bold.
Figure 3: Effect of Top- K values on F1 performance across four datasets.
Dataset
Avg. Character Distance
Avg. Token Distance
Min Distance
Max Distance
TBD
116.99
19.51
1
366
MATRES
128.94
24.01
1
416
TDDAuto
800.99
127.61
2
3349
TDDMan
913.61
146.65
61
3074
Table 4: Average, Minimum, and Maximum Character and Token Distance between Entity Pairs across Temporal Datasets
Figure 4: Example from the TimeBank-Dense test set illustrating pair-conditioned top- K attention. The model predicts BEFORE between said ( E1 ) and force ( E2 ). Pair-level attention highlights connective cues (e.g., as , itself ), while entity-level attention anchors event-specific tokens (e.g., visits , Saddam ). The complementary patterns indicate structured evidence selection for temporal inference.
Figure 5: POS feature distribution of TimeBank-Dense
Figure 6: POS feature distribution of MATRES
Figure 7: POS feature distribution of TDDAuto
Figure 8: POS feature distribution of TDDman
Figure 9: Dependency feature distribution of TimeBank-Dense
Figure 10: Dependency feature distribution of MATRES
Figure 11: Dependency feature distribution of TDDAuto
Figure 12: Dependency feature distribution of TDDMan
Figure 13: Morphological feature distribution of TimeBank-Dense
Figure 14: Morphological feature distribution of MATRES
Figure 15: Morphological feature distribution of TDDAuto
Figure 16: Morphological feature distribution of TDDMan
Temporal relation extraction determines whether an event occurs before, after, or simultaneously with another event, and therefore relies on accurately modeling how the two events interact. Mainstream systems achieve this by concatenating event spans or using shallow fusion, which works well when all model parameters are trainable. However, in parameter-efficient fine-tuning, low-rank bottlenecks restrict information flow and prevent these interaction signals from passing through, leading to clear performance drops. To address this limitation, we propose a theoretically grounded architecture, Convolutional Bottleneck Interaction (CBI), which first applies lightweight depthwise convolution to enhance event representations and then uses element-wise multiplication to capture effective event-event interactions inside the bottleneck. Across five datasets and seven backbone models in the Adapter and LoRA settings, CBI provides consistent and substantial gains, up to +31.7 micro F1, while adding minimal computational cost, showing that explicit interaction inside low-rank spaces is crucial for temporal relation extraction. The code is available at https://github.com/VRCMF/CIF.git.
Wei Sun, Tingyu Qu, Jesse Davis +1
Department of Computer Science, KU Leuven Celestijnenlaan 200A 3001 Heverlee, Belgium
Temporal relation classification is the task of determining the temporal relation between pairs of temporal entities in a text. Despite recent advancements in natural language processing, temporal relation classification remains a considerable challenge. Early attempts framed this task using a comprehensive set of temporal relations between events and temporal expressions. However, due to the task complexity, datasets have been progressively simplified, leading recent approaches to focus on the relations between event pairs and to use only a subset of relations. In this work, we revisit the broader goal of classifying interval relations between temporal entities by considering the full set of relations that can hold between two time intervals. The proposed approach, Interval from Point, involves first classifying the point relations between the endpoints of the temporal entities and then decoding these point relations into an interval relation. Evaluation on the TempEval-3 dataset shows that this approach can yield effective results, achieving a temporal awareness score of 70.1 percent, a new state-of-the-art on this benchmark.
Hugo Sousa, Ricardo Campos, Alípio Jorge
University of Porto · INESC TEC · Porto, Portugal +1
Temporal Information Retrieval (TIR) has been increasingly critical given the rise of Retrieval-Augmented Generation (RAG). Since temporally mismatched evidence can be highly misleading, TIR aims to retrieve documents that are both semantically and temporally relevant to a query. Two TIR paradigms have emerged - temporal retrievers and temporal re-rankers - differing in how temporal relevance is modeled. While these paradigms provide complementary strengths, our analysis reveals that each alone falls short of robust TIR: temporal retrievers provide flexible query understanding via learned representations, but often fail to explicitly account for temporal constraints; temporal re-rankers can enforce such constraints more explicitly, but often rely on predefined re-ranking rules. To address this, we propose Q-TIE, a re-ranking framework based on learned Temporal Intent Extraction (TIE). By introducing a TIE model that maps each query's temporal constraint into a unified interval representation (i.e., ⟨tstart,tend⟩), Q-TIE generalizes beyond predefined rules via model-based learning while explicitly modeling temporal constraints as a separate signal - jointly achieving what each paradigm typically trades off. Experiments demonstrate that Q-TIE consistently outperforms existing TIR methods with stronger generalizability across temporal query types, and provides a lightweight yet effective add-on for temporally-aware RAG pipelines. Code: https://github.com/ssoy0701/Q-TIE.
Soyeon Kim, Hyunjin Kim, JinYeong Bak +1
Korea Advanced Institute of Science and Technology · Sungkyunkwan University