Shrome at Touché: Soft-Vote Ensembling and Counter-Causal Augmentation for Causality Extraction

Roham Zendehdel Nobari; Shayan Sooratgar

cs.CLOct 2, 2026

Shrome at Touché: Soft-Vote Ensembling and Counter-Causal Augmentation for Causality Extraction

Authors: Roham Zendehdel Nobari, Shayan Sooratgar

Organizations: Department of Computational Linguistics, University of Zurich, Switzerland

Abstract

Touché 2026 extends causality extraction to counter-causal claims: news sentences whose surface form appears causal but whose meaning denies the causation, as in "It is falsely believed that X caused Y." A system that relies on surface cues such as "caused" or "led to" will accept such a sentence as causal and give it the wrong polarity. On the Countercausal News Corpus (CCNC), the task has three subtasks: deciding whether a sentence is causal (detection), locating its cause and effect spans (extraction), and labeling its polarity as procausal, counter-causal, or uncausal. We build one model per subtask. Detection is a fine-tuned classifier with a single cross-task rule that uses the extracted spans to remove false positives. For extraction, we ensemble three RoBERTa-large BILOU+CRF taggers by averaging their token-level scores before decoding, rather than voting on the spans each tagger produces. For polarity, where labeled counter-causal examples are scarcest, we add training sentences generated by a large language model prompted with nine patterns of counter-causal expression adapted from Hagen et al., keeping only those that pass automatic structural checks. On the held-out CCNC test set, the system reaches F1 0.869 on detection and macro-F1 0.817 on polarity, and in the organizers' final causal-only evaluation of extraction it scores granularity-adjusted F1 0.728, the highest extraction score among all submissions including the organizers' baseline. The development split is used only for component selection and the ablations reported in the paper.

Figures & tables

Subtask	Train	Dev	Gold-span or class units
Detection (binary)	3,075	340	1 label per sentence
Extraction	3,075	340	473 gold spans on dev
Polarity (3-class)	4,603	502	2,412/1,305/886 train; 264/120/118 dev

Table 1: CCNC counts on the splits used. Detection and extraction share sentence-level data; polarity has more rows because each candidate cause/effect pair within a sentence is annotated separately.

Class	Train rows	Neg. cues (dev)	After aug.
uncausal	2,412	29.5%	2,412
procausal	1,305	15.8%	1,305
counter-causal	886	67.8%	3,319
Total	4,603	—	7,036

Table 2: Polarity training composition. The counter-causal class is the smallest yet shows explicit negation cues most often (cue prevalence measured on dev). The nine-pattern LLM augmentation of Section 7 adds 2,433 rows, raising it from 886 to 3,319.

Figure 1: System overview on an illustrative input sentence. The detection, extraction, and polarity models are trained independently; the only cross-task interaction is the dashed arrow, through which the predicted spans $\hat{\mathcal{S}}$ gate the detection output via the one-rule stacker (causal iff $\hat{y}_{\text{det}}=1$ and $|\hat{\mathcal{S}}|\geq 1$ ).

Figure 2: Detection model. The input sentence is encoded by BERT-base; the [CLS] representation passes through the stock sequence-classification head, and an argmax over the two logits yields the binary causal/uncausal label.

Configuration	Binary F 1 (causal class)
Random	0.490
TF-IDF + logistic regression	0.793
BiLSTM	0.731
UniCausal sequence baseline	0.553
Zero-shot BART-MNLI	0.642
BERT-base (seed 42, standalone)	0.8690

Table 3: Detection dev results (binary F 1 ). The production system is the stacked variant in the last row. UniCausal baseline is the sequence-classification head of Tan et al. Tan et al. (2023) ; BART-MNLI is the zero-shot pipeline over facebook/bart-large-mnli .

Figure 3: One extraction seed. A shared RoBERTa-large encoder feeds three parallel BILOU layers, each decoded by its own linear-chain CRF; the decoded spans are unioned across layers, de-duplicated, and trimmed to produce the output spans.

Source	Base recipe	TAPT recipe
CCNC extraction train	3,075	3,075
CNCv2/RECESS (signals stripped)	3,075	3,075
UniCausal (signals stripped)	—	1,049
Source rows	6,150	7,199
After EDA + oversampling	$\approx$ 11,500	$\approx$ 13,400

Table 4: Extraction training-pool composition by recipe. Source rows are expanded by EDA augmentation and multi-span oversampling; the two base seeds differ slightly (seed 123:

11{,}489

; seed 2026:

11{,}492

rows) and the TAPT seed reaches

13{,}432

Figure 4: Emission-level soft-vote. Each of the three seeds produces per-layer emissions and transition scores $(\mathbf{E}_{k},\mathbf{T}_{k})$ ; these are averaged into $(\bar{\mathbf{E}},\bar{\mathbf{T}})$ before a single per-layer Viterbi decode, so near-miss agreement between seeds survives to the decoding step instead of being discarded by span-level voting. The two green encoders are the base-recipe seeds; the yellow encoder is the seed warm-started from the task-adaptively pre-trained (TAPT) checkpoint (Section 6.1 ).

Seed	Encoder warm-start	Recipe	Exact F 1	Overlap F 1
123	RoBERTa-large	base recipe	0.5742	0.7692
2026	RoBERTa-large	base recipe	0.5505	0.7569
42	RoBERTa-large + TAPT	+UniCausal, boundary-smoothing aux.	0.5260	0.7584
Soft-vote of the three seeds above			0.5933	0.7800

Table 5: Per-seed and ensemble dev F 1 for the production extraction panel.

Stage	Exact F 1 (overlap F 1 )	$\Delta$ exact
Hagen RoBERTa baseline Hagen et al. (2025) (Val, BIO-tag)	0.440 (n/a)	ref †
Single seed, base recipe (seed 123)	0.5742 (0.7692)	$+13.4\,\textnormal{pp}$
Hard span-level vote, matched three-seed panel	0.5781 (0.7634)	$+13.8\,\textnormal{pp}$
Emission soft-vote, same matched panel	0.5888 (0.7730)	$+14.9\,\textnormal{pp}$
Emission soft-vote, production panel with TAPT seed	0.5933 (0.7800)	$\mathbf{+15.3\,\textnormal{pp}}$
+ NLI span verifier ( $\tau=0.1$ , within-noise)	0.5953 (0.7781)	$+15.5\,\textnormal{pp}$

Table 6: Extraction dev progression. All numbers in this table except the first row are span-level exact-match F 1 on the CCNC dev split (overlap F 1 in parentheses). † The Hagen RoBERTa baseline is BIO-tag macro F 1 on Hagen’s evaluation split (Hagen et al. Table 7; their Table 6a labels this as Val, with no separate test column), not span exact-match on dev; the two metrics and the two splits differ, so

\Delta

is an indicative reference, not a directly comparable gain.

Figure 5: Polarity stack. The tagged input sentence is rewritten with PURE typed entity markers and classified by four independently seeded BERT-base models; their softmax outputs are averaged (mean-softmax) to produce the final three-way label.

Configuration	Macro F 1	uncausal F 1	procausal F 1	counter-causal F 1
Seed 42	0.8897	0.9275	0.8678	0.8739
Seed 123	0.8773	0.9091	0.8319	0.8908
Seed 2026	0.8805	0.9246	0.8468	0.8703
Seed 7	0.8867	0.9216	0.8571	0.8814
4-seed mean-softmax (production)	0.9029	—	—	0.891
5-seed mean-softmax (adds a weaker seed)	0.8953	—	—	—

Table 7: Polarity dev results. Per-class F 1 on the right. For the production ensemble row, only the macro and counter-causal F 1 are reported: the stored uncausal and procausal per-class numbers are rounded to three decimals and do not reproduce the full-precision macro F 1 of 0.9029 when averaged at that precision.

Gold $\downarrow$ / Predicted $\rightarrow$	uncausal	procausal	counter-causal	recall
uncausal (264)	243	12	9	0.920
procausal (120)	8	106	6	0.883
counter-causal (118)	8	5	105	0.890

Table 8: Polarity seed-42 confusion matrix on the dev split. Rows are gold; columns are predicted. Recall on the right.

Subtask	Metric	Dev	Test	$\Delta$
Detection (binary)	F 1 (causal class)	0.8946	0.869	$-2.6\,\textnormal{pp}$
	precision	—	0.848	—
	recall	—	0.890	—
	accuracy	—	0.832	—
Extraction (TIRA)	exact-match F 1	0.5933	0.473	$-12.0\,\textnormal{pp}$
	precision	—	0.475	—

Table 9: Held-out CCNC test-set results, with matched dev numbers. Rows marked TIRA are from the official TIRA Fröbe et al. (2023) evaluator; the “final, causal-only” extraction rows are from the organizers’ post-deadline re-evaluation of the extraction subtask on the causal test samples only, which raises scores without reordering submissions and will be reported in the lab overview Kiesel et al. (2026b) . For extraction the TIRA evaluator reports exact-match F 1 together with a token-level IoU; the dev overlap F 1 is a different relaxation of exact match and is not directly comparable.

\Delta

is test

-

dev where metrics align. Row-internal arithmetic does not reconcile: the TIRA extraction F 1 (0.473) is a per-sentence F 1 averaged over the test set while the displayed precision (0.475) and recall (0.490) are corpus-level micro values, whose harmonic mean would be 0.482; the polarity macro F 1 (0.817) is the mean of per-class F 1 scores rather than the harmonic mean of macro precision (0.796) and macro recall (0.851), which would be 0.823. We report values as published.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Parameter	Detection	Extraction (per seed)	Polarity (per seed)
Encoder	BERT-base	RoBERTa-large	BERT-base
Optimizer	AdamW	AdamW ( $\varepsilon=10^{-6}$ )	AdamW
Learning rate	$2\!\times\!10^{-5}$	$5\!\times\!10^{-5}$	$2\!\times\!10^{-5}$
CRF-group learning rate	—	$3\!\times\!10^{-4}$	—
Weight decay	0.01	0.01	—
Schedule	10% linear warmup, cosine	6% warmup, cosine	—

Appendix

Table 10: Fine-tuning hyperparameters per subtask model. Cells marked — do not apply; unlisted settings follow library defaults.

Relation Extraction

Relation extraction identifies and classifies relationships between entities or events in text and multimodal data.

See all 82 papers

PubMedCausal: A Span-Level Annotated Corpus for Causal Relation Extraction in Biomedical Text
May 27, 2026Ifeoluwa Kunle-John, Josiah Paul, Oluwatosin Agbaakin +3Relation Extraction Clinical NLP
Edyah Limited · University of Ibadan, NG · Indiana University, Indianapolis, IN +1
PROMETHEUS: Automating Deep Causal Research Integrating Text, Data and Models
May 13, 2026Sridhar MahadevanCausal World Models Causal Attribution
Adobe Research · University of Massachusetts, Amherst
Causal Tongue-Tie: LLMs Can Encode Causal Direction, But Their Yes/No Outputs Fail to Express
May 25, 2026Ziyi Ding, Xiao-Ping ZhangLLM Evaluation Causal Reasoning in Language Models
Tsinghua Shenzhen International Graduate School, Tsinghua University Shenzhen, China

May 27, 2026J/K move · Enter open · S save

PubMedCausal: A Span-Level Annotated Corpus for Causal Relation Extraction in Biomedical Text

Ifeoluwa Kunle-John, Josiah Paul, Oluwatosin Agbaakin, Peter Aina+2

Edyah Limited · University of Ibadan, NG · Indiana University, Indianapolis, IN+1

Relation Extraction Clinical NLP

Open paper AskPDF arXiv

Causal relation extraction (CRE) is central to biomedical text mining, but current resources often conflate causal relations with broader associations, restrict annotation to sentence-level examples, or focus mainly on explicit causal cues. This limits their usefulness for evaluating whether models can recover causal claims as they are actually expressed in biomedical text. We introduce PubMedCausal, a span-level annotated corpus for biomedical CRE built from PubMed abstracts. The corpus contains 30,000 paragraph-level rows, including 3,945 causal rows and 6,491 adjudicated cause--effect pairs. Each causal relation is annotated with full-text cause and effect spans, causality type, and sententiality, enabling evaluation of both causal detection and full-span causal extraction. We benchmark discriminative encoders and open-source generative models across detection and extraction settings. For causal detection, biomedical encoders are strongest, with PubMedBERT reaching an F

_1

score of 0.7391. For span-level extraction, the best generative baseline is DeepSeek-R1-32B with few-shot prompting, reaching a Cosine Pair F

_1

of 0.6765. We further test transfer learning by evaluating PubMedCausal-trained encoders on external causal relation datasets, showing that the resource supports cross-dataset evaluation. Our results show that biomedical CRE remains difficult under class imbalance, long causal spans, implicit causality, inter-sentential relations, and prompt sensitivity. Code and Data can be found here: https://github.com/josiahpaul07/PubMedCausal_Exp

Abstract

Relation Extraction

Explore similar work

PubMedCausal: A Span-Level Annotated Corpus for Causal Relation Extraction in Biomedical Text

PROMETHEUS: Automating Deep Causal Research Integrating Text, Data and Models

Causal Tongue-Tie: LLMs Can Encode Causal Direction, But Their Yes/No Outputs Fail to Express