Event Interaction in Low-Rank Bottlenecks for Temporal Relation Extraction

Wei Sun; Tingyu Qu; Jesse Davis; Marie-Francine Moens

cs.CLSep 6, 2026

Event Interaction in Low-Rank Bottlenecks for Temporal Relation Extraction

Authors: Wei Sun, Tingyu Qu, Jesse Davis, Marie-Francine Moens

Organizations: Department of Computer Science, KU Leuven Celestijnenlaan 200A 3001 Heverlee, Belgium

Abstract

Temporal relation extraction determines whether an event occurs before, after, or simultaneously with another event, and therefore relies on accurately modeling how the two events interact. Mainstream systems achieve this by concatenating event spans or using shallow fusion, which works well when all model parameters are trainable. However, in parameter-efficient fine-tuning, low-rank bottlenecks restrict information flow and prevent these interaction signals from passing through, leading to clear performance drops. To address this limitation, we propose a theoretically grounded architecture, Convolutional Bottleneck Interaction (CBI), which first applies lightweight depthwise convolution to enhance event representations and then uses element-wise multiplication to capture effective event-event interactions inside the bottleneck. Across five datasets and seven backbone models in the Adapter and LoRA settings, CBI provides consistent and substantial gains, up to +31.7 micro F1, while adding minimal computational cost, showing that explicit interaction inside low-rank spaces is crucial for temporal relation extraction. The code is available at https://github.com/VRCMF/CIF.git.

Figures & tables

Panel of Figure 1 — Figure 1: Visualization of event interaction maps of the temporal relation extraction task for (a) Full Fine-tuning , Text Concatenation with (b) Adapter fine-tuning, (c) LoRA fine-tuning, Embedding Fusion with (d) Adapter fine-tuning, and (e) LoRA fine-tuning . marks tokens emphasized by different methods. Attention in Text Concatenation and Embedding Fusion is diffused over irrelevant tokens, limiting temporal cue propagation through the bottleneck, while full fine-tuning captures these cues effectively but at a high computational cost.

Panel of Figure 2 — Figure 2: Illustration of (a) LoRA, (b) Adapter, and (c) The Convolutional Bottleneck Interaction adapter, which respectively replaces the LoRA or adapter block when integrated in the transformer block of (a) and (b). The parameters in the blue boxes are learned during the fine-tuning.

Backbone	PEFT	Methods	MATRES	TB-Dense	TCR	TDD-Auto	TDD-Man	AVG
	/	FFT (upper bound)	82.15 $\pm$ 0.34	63.27 $\pm$ 0.41	81.12 $\pm$ 0.38	86.71 $\pm$ 0.33	80.82 $\pm$ 0.45	78.81 $\pm$ 0.36
RoBERTa-large	Adapter	Text Concatenation	58.31 $\pm$ 0.91	41.69 $\pm$ 1.07	59.72 $\pm$ 0.66	32.85 $\pm$ 1.12	31.59 $\pm$ 0.53	44.83 $\pm$ 0.78
		Embedding Fusion	58.31 $\pm$ 0.74	41.69 $\pm$ 0.88	59.72 $\pm$ 1.13	32.85 $\pm$ 0.97	57.14 $\pm$ 0.49	49.94 $\pm$ 1.05
		CBI	74.35 $\pm$ 0.85	68.18 $\pm$ 1.07	73.92 $\pm$ 1.61	86.85 $\pm$ 1.32	79.67 $\pm$ 1.18	76.59 $\pm$ 1.22
	LoRA	Text Concatenation	67.58 $\pm$ 0.62	55.17 $\pm$ 0.97	67.44 $\pm$ 0.74	71.85 $\pm$ 0.91	64.57 $\pm$ 0.83	65.32 $\pm$ 0.58
		Embedding Fusion	59.52 $\pm$ 1.01	46.08 $\pm$ 0.79	62.35 $\pm$ 1.18	64.95 $\pm$ 0.55	68.24 $\pm$ 0.47	60.23 $\pm$ 0.88

Table 1: Performance comparison of Convolutional Bottleneck Interaction (CBI) with two baselines (concatenation of event embeddings and element-wise product of event embeddings) under Adapter and LoRA PEFT settings evaluated on five temporal relation extraction datasets (micro-F1%). Each score shows the mean

\pm

std of three runs. Full fine-tuning (FFT) and zero-shot tuning (ZST) are reported as reference upper and lower bounds for PEFT methods, respectively. The best score in each setting (with the same backbone and PEFT method) is shown in bold blue , the second-best is shown in gray underlined .

Panel of Figure 3 — Figure 3: Effect of training data proportion on model performance. Both CBI and Multiplication are methods proposed in this paper, while Text Concatenation serves as the baseline.

Backbone	PEFT	Routing	Matres	TB-dense	TCR	TDD-auto	TDD-man	AVG
Roberta-large	Adapter	Multiplication	71.61 $\pm$ 0.78	60.82 $\pm$ 1.34	68.83 $\pm$ 1.22	89.15 $\pm$ 1.49	76.33 $\pm$ 0.91	73.35 $\pm$ 1.18
	Adapter	Addition	58.79 $\pm$ 1.55	41.69 $\pm$ 0.92	59.72 $\pm$ 1.36	56.35 $\pm$ 1.21	34.04 $\pm$ 1.44	50.12 $\pm$ 1.47
	Adapter	Subtraction	58.31 $\pm$ 1.09	41.69 $\pm$ 1.23	59.72 $\pm$ 1.28	85.40 $\pm$ 1.63	69.22 $\pm$ 0.98	62.87 $\pm$ 1.25
	Adapter	CBI	74.35 $\pm$ 0.85	68.18 $\pm$ 1.07	73.92 $\pm$ 1.61	86.85 $\pm$ 1.32	79.67 $\pm$ 1.18	76.59 $\pm$ 1.22
	LoRA	Multiplication	68.79 $\pm$ 1.31	60.34 $\pm$ 0.52	70.06 $\pm$ 0.95	87.40 $\pm$ 0.83	71.02 $\pm$ 0.59	71.52 $\pm$ 0.90
	LoRA	Addition	66.45 $\pm$ 0.38	61.60 $\pm$ 0.83	65.43 $\pm$ 1.33	78.65 $\pm$ 0.68	73.71 $\pm$ 1.08	69.17 $\pm$ 0.99

Table 2: Temporal relation extraction under two PEFT settings (Adapter / LoRA), comparing different interaction methods that help routing features through the low-rank bottlenecks and their backbone LLMs. The best score in each setting (with the same backbone and PEFT method) is shown in bold blue , the second-best is shown in gray underlined .

Panel of Figure 4 — Figure 4: Event interaction map for each event interaction method.

Computation Metrics	PEFT	Multiplication	Addition	CBI	Subtraction	Text Concatenation	Embedding Fusion	Full Fine-tuning
Memory Cost	LoRA	29269.37	29269.37	29271.05	29269.37	29271.40	29269.37	148792.17
Memory Cost	Adapter	29030.91	29030.91	29031.75	29030.91	29030.91	29030.91	148792.17
Time	LoRA	223.84	216.71	221.12	217.16	375.81	214.07	3939.75
Time	Adapter	216.42	216.31	221.8	216.55	376.06	213.58	3939.75

Table 3: Computational analysis of LoRA and Adapter on Llama-3.1-8B, reporting GPU memory usage (MB) and runtime in terms of seconds (s) across six temporal relation extraction models.

Kernel Size	MATRES	TB-Dense	TCR	TDD-Auto	TDD-Man	AVG
Adapter
7	67.90	56.11	63.89	88.80	71.43	69.63
5	64.84	55.80	63.43	89.20	72.98	69.25
3	66.13	58.31	65.58	86.35	70.45	69.36
1	62.34	54.86	62.96	78.75	61.31	64.04
LoRA

Table 4: Influence of the convolutional kernel size for CBI under Adapter and LoRA settings. Kernel size 5 provides the most stable and competitive performance across all datasets. Best results are in bold blue , second-best in gray underlined .

Methods	MATRES	TB-Dense	TCR	TDD-Auto	TDD-Man	AVG
Adapter
Text Concatenation	60.25	51.25	61.49	65.77	64.41	60.63
Embedding Fusion	57.70	43.13	59.76	48.31	37.20	49.22
CBI	66.61	56.23	65.15	90.02	75.99	70.80
LoRA
Text Concatenation	66.24	59.41	63.04	72.02	64.50	65.04

Table 5: Performance comparison of Convolutional Bottleneck Interaction (CBI) with two baselines under Adapter and LoRA settings augmented with inference over a global document-level event graph applied on five temporal relation extraction datasets (micro-F1%) using the Llama 3.1 8B backbone. Best results are in bold blue , second-best in gray underlined .

Dataset	Train	Dev	Test
MATRES	3,715	1,240	1,238
TB-Dense	1,910	638	636
TCR	1,939	648	646
TDD-Auto	22,981	7,660	7,661
TDD-Man	3,673	1,225	1,224

Table 6: Statistics of the five temporal relation extraction datasets in terms of number of temporal relation examples.

Methods	Adapter (rank $r$ )			LoRA (rank $r$ )
	$r$ =128	$r$ =256	$r$ =512	$r$ =128	$r$ =256	$r$ =512
Text Concatenation	60.06	61.87	63.36	64.94	65.92	66.32
Embedding Fusion	42.69	43.37	44.14	41.61	42.78	43.94
CBI	69.25	70.31	71.34	73.30	73.94	74.71

Table 7: Performance comparison of CBI results with results baseline methods obtained with different bottleneck ranks

r

under Adapter and LoRA settings (the averaged micro-F1 score across all five benchmarks). Best results are in bold blue , second-best in gray underlined .

Panel of Figure 5 — Figure 5: Visualization of event interaction maps for evaluating the temporal relation extraction task: (a) Full Fine-tuning , (b) Text Concatenation with Adapter fine-tuning, (c) LoRA fine-tuning, Embedding Fusion with (d) Adapter fine-tuning, and (e) LoRA fine-tuning, CBI with (f) Adapter fine-tuning, and (j) LoRA fine-tuning. marks tokens emphasized by different methods. CBI offers a computationally efficient alternative to full fine-tuning while effectively capturing distributed temporal cues, outperforming other parameter-efficient methods in preserving critical signals.

Relation Extraction

Relation extraction identifies and classifies relationships between entities or events in text and multimodal data.

See all 82 papers

WISTERIA: Weak Implicit Signal-based Temporal Relation Extraction with Attention
Mar 24, 2026Duy Dao Do, Anaïs Halftermeyer, Thi-Bich-Hanh DaoRelation Extraction Temporal Reasoning
University of Orléans, INSA Centre Val de Loire, LIFO EA 4022, France
Looking for the Bottleneck in Fine-grained Temporal Relation Classification
Apr 27, 2026Hugo Sousa, Ricardo Campos, Alípio JorgeRelation Extraction
University of Porto · INESC TEC · Porto, Portugal +1
EventCoT: Event-centric Video Chain-of-thought for Reasoning Temporal Localization
Jul 6, 2026Youngkil Song, Yoonjae Baek, Dongwon Kim +3Temporal Video Grounding Multimodal CoT Reasoning
1Pohang University of Science and Technology, Pohang, South Korea · 2Korea Advanced Institute of Science and Technology, Daejeon, South Korea · 3Handong Global University, Pohang, South Korea

Mar 24, 2026J/K move · Enter open · S save

WISTERIA: Weak Implicit Signal-based Temporal Relation Extraction with Attention

Duy Dao Do, Anaïs Halftermeyer, Thi-Bich-Hanh Dao

University of Orléans, INSA Centre Val de Loire, LIFO EA 4022, France

Relation Extraction Temporal Reasoning Temporal Attention

Open paper AskPDF arXiv

Temporal Relation Extraction (TRE) requires identifying how two events or temporal expressions are related in time. Existing attention-based models often highlight globally salient tokens but overlook the pair-specific cues that actually determine the temporal relation. We propose WISTERIA (Weak Implicit Signal-based Temporal Relation Extraction with Attention), a framework that examines whether the top-K attention components conditioned on each event pair truly encode interpretable evidence for temporal classification. Unlike prior works assuming explicit markers such as before, after, or when, WISTERIA considers signals as any lexical, syntactic, or morphological element implicitly expressing temporal order. By combining multi-head attention with pair-conditioned top-K pooling, the model isolates the most informative contextual tokens for each pair. We conduct extensive experiments on TimeBank-Dense, MATRES, TDDMan, and TDDAuto, including linguistic analyses of top-K tokens. Results show that WISTERIA achieves competitive accuracy and reveals pair-level rationales aligned with temporal linguistic cues, offering a localized and interpretable view of temporal reasoning.

Abstract

Relation Extraction

Explore similar work

WISTERIA: Weak Implicit Signal-based Temporal Relation Extraction with Attention

Looking for the Bottleneck in Fine-grained Temporal Relation Classification

EventCoT: Event-centric Video Chain-of-thought for Reasoning Temporal Localization