Temporal relation extraction determines whether an event occurs before, after, or simultaneously with another event, and therefore relies on accurately modeling how the two events interact. Mainstream systems achieve this by concatenating event spans or using shallow fusion, which works well when all model parameters are trainable. However, in parameter-efficient fine-tuning, low-rank bottlenecks restrict information flow and prevent these interaction signals from passing through, leading to clear performance drops. To address this limitation, we propose a theoretically grounded architecture, Convolutional Bottleneck Interaction (CBI), which first applies lightweight depthwise convolution to enhance event representations and then uses element-wise multiplication to capture effective event-event interactions inside the bottleneck. Across five datasets and seven backbone models in the Adapter and LoRA settings, CBI provides consistent and substantial gains, up to +31.7 micro F1, while adding minimal computational cost, showing that explicit interaction inside low-rank spaces is crucial for temporal relation extraction. The code is available at https://github.com/VRCMF/CIF.git.
Figures & tables
Figure 1: Visualization of event interaction maps of the temporal relation extraction task for (a) Full Fine-tuning , Text Concatenation with (b) Adapter fine-tuning, (c) LoRA fine-tuning, Embedding Fusion with (d) Adapter fine-tuning, and (e) LoRA fine-tuning . marks tokens emphasized by different methods. Attention in Text Concatenation and Embedding Fusion is diffused over irrelevant tokens, limiting temporal cue propagation through the bottleneck, while full fine-tuning captures these cues effectively but at a high computational cost.
Figure 2: Illustration of (a) LoRA, (b) Adapter, and (c) The Convolutional Bottleneck Interaction adapter, which respectively replaces the LoRA or adapter block when integrated in the transformer block of (a) and (b). The parameters in the blue boxes are learned during the fine-tuning.
Backbone
PEFT
Methods
MATRES
TB-Dense
TCR
TDD-Auto
TDD-Man
AVG
/
FFT (upper bound)
82.15 ± 0.34
63.27 ± 0.41
81.12 ± 0.38
86.71 ± 0.33
80.82 ± 0.45
78.81 ± 0.36
RoBERTa-large
Adapter
Text Concatenation
58.31 ± 0.91
41.69 ± 1.07
59.72 ± 0.66
32.85 ± 1.12
31.59 ± 0.53
44.83 ± 0.78
Embedding Fusion
58.31 ± 0.74
41.69 ± 0.88
59.72 ± 1.13
32.85 ± 0.97
57.14 ± 0.49
49.94 ± 1.05
CBI
74.35 ± 0.85
68.18 ± 1.07
73.92 ± 1.61
86.85 ± 1.32
79.67 ± 1.18
76.59 ± 1.22
LoRA
Text Concatenation
67.58 ± 0.62
55.17 ± 0.97
67.44 ± 0.74
71.85 ± 0.91
64.57 ± 0.83
65.32 ± 0.58
Embedding Fusion
59.52 ± 1.01
46.08 ± 0.79
62.35 ± 1.18
64.95 ± 0.55
68.24 ± 0.47
60.23 ± 0.88
Table 1: Performance comparison of Convolutional Bottleneck Interaction (CBI) with two baselines (concatenation of event embeddings and element-wise product of event embeddings) under Adapter and LoRA PEFT settings evaluated on five temporal relation extraction datasets (micro-F1%). Each score shows the mean ± std of three runs. Full fine-tuning (FFT) and zero-shot tuning (ZST) are reported as reference upper and lower bounds for PEFT methods, respectively. The best score in each setting (with the same backbone and PEFT method) is shown in bold blue , the second-best is shown in gray underlined .
Figure 3: Effect of training data proportion on model performance. Both CBI and Multiplication are methods proposed in this paper, while Text Concatenation serves as the baseline.
Backbone
PEFT
Routing
Matres
TB-dense
TCR
TDD-auto
TDD-man
AVG
Roberta-large
Adapter
Multiplication
71.61 ± 0.78
60.82 ± 1.34
68.83 ± 1.22
89.15 ± 1.49
76.33 ± 0.91
73.35 ± 1.18
Adapter
Addition
58.79 ± 1.55
41.69 ± 0.92
59.72 ± 1.36
56.35 ± 1.21
34.04 ± 1.44
50.12 ± 1.47
Adapter
Subtraction
58.31 ± 1.09
41.69 ± 1.23
59.72 ± 1.28
85.40 ± 1.63
69.22 ± 0.98
62.87 ± 1.25
Adapter
CBI
74.35 ± 0.85
68.18 ± 1.07
73.92 ± 1.61
86.85 ± 1.32
79.67 ± 1.18
76.59 ± 1.22
LoRA
Multiplication
68.79 ± 1.31
60.34 ± 0.52
70.06 ± 0.95
87.40 ± 0.83
71.02 ± 0.59
71.52 ± 0.90
LoRA
Addition
66.45 ± 0.38
61.60 ± 0.83
65.43 ± 1.33
78.65 ± 0.68
73.71 ± 1.08
69.17 ± 0.99
Table 2: Temporal relation extraction under two PEFT settings (Adapter / LoRA), comparing different interaction methods that help routing features through the low-rank bottlenecks and their backbone LLMs. The best score in each setting (with the same backbone and PEFT method) is shown in bold blue , the second-best is shown in gray underlined .
Figure 4: Event interaction map for each event interaction method.
Computation Metrics
PEFT
Multiplication
Addition
CBI
Subtraction
Text Concatenation
Embedding Fusion
Full Fine-tuning
Memory Cost
LoRA
29269.37
29269.37
29271.05
29269.37
29271.40
29269.37
148792.17
Adapter
29030.91
29030.91
29031.75
29030.91
29030.91
29030.91
Time
LoRA
223.84
216.71
221.12
217.16
375.81
214.07
3939.75
Adapter
216.42
216.31
221.8
216.55
376.06
213.58
Table 3: Computational analysis of LoRA and Adapter on Llama-3.1-8B, reporting GPU memory usage (MB) and runtime in terms of seconds (s) across six temporal relation extraction models.
Kernel Size
MATRES
TB-Dense
TCR
TDD-Auto
TDD-Man
AVG
Adapter
7
67.90
56.11
63.89
88.80
71.43
69.63
5
64.84
55.80
63.43
89.20
72.98
69.25
3
66.13
58.31
65.58
86.35
70.45
69.36
1
62.34
54.86
62.96
78.75
61.31
64.04
LoRA
Table 4: Influence of the convolutional kernel size for CBI under Adapter and LoRA settings. Kernel size 5 provides the most stable and competitive performance across all datasets. Best results are in bold blue , second-best in gray underlined .
Methods
MATRES
TB-Dense
TCR
TDD-Auto
TDD-Man
AVG
Adapter
Text Concatenation
60.25
51.25
61.49
65.77
64.41
60.63
Embedding Fusion
57.70
43.13
59.76
48.31
37.20
49.22
CBI
66.61
56.23
65.15
90.02
75.99
70.80
LoRA
Text Concatenation
66.24
59.41
63.04
72.02
64.50
65.04
Table 5: Performance comparison of Convolutional Bottleneck Interaction (CBI) with two baselines under Adapter and LoRA settings augmented with inference over a global document-level event graph applied on five temporal relation extraction datasets (micro-F1%) using the Llama 3.1 8B backbone. Best results are in bold blue , second-best in gray underlined .
Dataset
Train
Dev
Test
MATRES
3,715
1,240
1,238
TB-Dense
1,910
638
636
TCR
1,939
648
646
TDD-Auto
22,981
7,660
7,661
TDD-Man
3,673
1,225
1,224
Table 6: Statistics of the five temporal relation extraction datasets in terms of number of temporal relation examples.
Methods
Adapter (rank r )
LoRA (rank r )
r =128
r =256
r =512
r =128
r =256
r =512
Text Concatenation
60.06
61.87
63.36
64.94
65.92
66.32
Embedding Fusion
42.69
43.37
44.14
41.61
42.78
43.94
CBI
69.25
70.31
71.34
73.30
73.94
74.71
Table 7: Performance comparison of CBI results with results baseline methods obtained with different bottleneck ranks r under Adapter and LoRA settings (the averaged micro-F1 score across all five benchmarks). Best results are in bold blue , second-best in gray underlined .
Figure 5: Visualization of event interaction maps for evaluating the temporal relation extraction task: (a) Full Fine-tuning , (b) Text Concatenation with Adapter fine-tuning, (c) LoRA fine-tuning, Embedding Fusion with (d) Adapter fine-tuning, and (e) LoRA fine-tuning, CBI with (f) Adapter fine-tuning, and (j) LoRA fine-tuning. marks tokens emphasized by different methods. CBI offers a computationally efficient alternative to full fine-tuning while effectively capturing distributed temporal cues, outperforming other parameter-efficient methods in preserving critical signals.
Temporal Relation Extraction (TRE) requires identifying how two events or temporal expressions are related in time. Existing attention-based models often highlight globally salient tokens but overlook the pair-specific cues that actually determine the temporal relation. We propose WISTERIA (Weak Implicit Signal-based Temporal Relation Extraction with Attention), a framework that examines whether the top-K attention components conditioned on each event pair truly encode interpretable evidence for temporal classification. Unlike prior works assuming explicit markers such as before, after, or when, WISTERIA considers signals as any lexical, syntactic, or morphological element implicitly expressing temporal order. By combining multi-head attention with pair-conditioned top-K pooling, the model isolates the most informative contextual tokens for each pair. We conduct extensive experiments on TimeBank-Dense, MATRES, TDDMan, and TDDAuto, including linguistic analyses of top-K tokens. Results show that WISTERIA achieves competitive accuracy and reveals pair-level rationales aligned with temporal linguistic cues, offering a localized and interpretable view of temporal reasoning.
Duy Dao Do, Anaïs Halftermeyer, Thi-Bich-Hanh Dao
University of Orléans, INSA Centre Val de Loire, LIFO EA 4022, France
Temporal relation classification is the task of determining the temporal relation between pairs of temporal entities in a text. Despite recent advancements in natural language processing, temporal relation classification remains a considerable challenge. Early attempts framed this task using a comprehensive set of temporal relations between events and temporal expressions. However, due to the task complexity, datasets have been progressively simplified, leading recent approaches to focus on the relations between event pairs and to use only a subset of relations. In this work, we revisit the broader goal of classifying interval relations between temporal entities by considering the full set of relations that can hold between two time intervals. The proposed approach, Interval from Point, involves first classifying the point relations between the endpoints of the temporal entities and then decoding these point relations into an interval relation. Evaluation on the TempEval-3 dataset shows that this approach can yield effective results, achieving a temporal awareness score of 70.1 percent, a new state-of-the-art on this benchmark.
Hugo Sousa, Ricardo Campos, Alípio Jorge
University of Porto · INESC TEC · Porto, Portugal +1
Reasoning temporal localization (RTL) requires a model to generate an answer that itself contains the time interval supporting it, so high-level reasoning and precise temporal grounding must be produced jointly in a single response. To tackle this challenging task, we propose the first event-centric video chain-of-thought framework, dubbed EventCoT. EventCoT first performs event-centric tokenization of the input video to convert it into compact event tokens, enabling efficient identification of question-relevant events. It then reasons within the identified events to generate the answer, grounding the time interval via embedding matching that aligns placeholder tokens with visual embeddings. EventCoT achieves state-of-the-art results on ActivityNet-RTL for reasoning temporal localization while using substantially fewer visual tokens than previous work. To verify its general performance, we further evaluate EventCoT on the grounded video question answering benchmark ReXTime, where it attains strong zero-shot results.
Youngkil Song, Yoonjae Baek, Dongwon Kim +3
1Pohang University of Science and Technology, Pohang, South Korea · 2Korea Advanced Institute of Science and Technology, Daejeon, South Korea · 3Handong Global University, Pohang, South Korea