KCLarity at SemEval-2026 Task 6: Encoder and Zero-Shot Approaches to Political Evasion Detection

Archie Sage; Salvatore Greco

cs.CLMar 6, 2026

KCLarity at SemEval-2026 Task 6: Encoder and Zero-Shot Approaches to Political Evasion Detection

Authors: Archie Sage, Salvatore Greco

Organizations: King’s College London

Abstract

This paper describes the KCLarity team's participation in CLARITY, a shared task at SemEval 2026 on classifying ambiguity and evasion techniques in political discourse. We investigate two modelling formulations: (i) directly predicting the clarity label, and (ii) predicting the evasion label and deriving clarity through the task taxonomy hierarchy. We further explore several auxiliary training variants and evaluate decoder-only models in a zero-shot setting under the evasion-first formulation. Overall, the two formulations yield comparable performance. Among encoder-based models, RoBERTa-large achieves the strongest results on the public test set, while zero-shot GPT-5.2 generalises better on the hidden evaluation set.

Figures & tables

Clarity Level	Evasion Level
Clear Reply	Explicit
Ambivalent Reply	Implicit, Dodging, General, Deflection, Partial
Clear Non-Reply	Declining, Ignorance, Clarification

Table 1: Taxonomy of response clarity classification.

Model	Direct clarity			Evasion-based clarity
Model	F1	P	R	F1	P	R
base	0.598	0.586	0.636	0.595	0.589	0.619
	(0.010)	(0.013)	(0.021)	(0.028)	(0.035)	(0.030)
large	0.658	0.681	0.661	0.661	0.694	0.641
	(0.024)	(0.081)	(0.030)	(0.022)	(0.013)	(0.023)

Table 2: RoBERTa models: direct vs evasion-based clarity on the public test split. Macro-F1 (F1), precision (P), and recall (R), averaged over three seeds (standard deviation in brackets). Higher scores between the two formulations are shown in bold.

Model	Evasion-based clarity			Evasion
Model	F1	P	R	$\mathbf{ACC}_{\mathbf{match}}$	F1 A1	F1 A2	F1 A3	F1 avg
RoBERTa-base	0.595	0.589	0.619	0.516	0.349	0.332	0.333	0.338
	(0.028)	(0.035)	(0.030)	(0.014)	(0.024)	(0.007)	(0.008)	(0.011)
RoBERTa-large	0.661	0.694	0.641	0.539	0.363	0.378	0.374	0.371
	(0.022)	(0.013)	(0.023)	(0.011)	(0.030)	(0.035)	(0.047)	(0.037)
DeBERTa-v3-base	0.535	0.564	0.565	0.505	0.304	0.337	0.303	0.314

Table 3: Fine-tuned encoder results on the public test set. For evasion-based clarity we report macro-F1 (F1), precision (P), and recall (R). For evasion we report

\mathrm{ACC}_{\mathrm{match}}

, the fraction of predictions matching at least one annotator, per-annotator macro-F1, and the average macro-F1 across annotators. Results are averaged over three seeds (mean on the main row, standard deviation in brackets on the row beneath). Best-performing metrics are shown in bold.

Model	Evasion-based clarity			Evasion
Model	F1	P	R	$\mathbf{ACC}_{\mathbf{match}}$	F1 A1	F1 A2	F1 A3	F1 avg
gemma_3_27b_it	0.413	0.501	0.446	0.344	0.137	0.149	0.137	0.141
gpt_oss_120b	0.354	0.374	0.354	0.357	0.104	0.121	0.115	0.114
Llama-3.1-8B-Instruct	0.346	0.385	0.370	0.351	0.054	0.079	0.068	0.067
Llama-3.3-70B-Instruct	0.532	0.544	0.547	0.416	0.284	0.292	0.294	0.290
Qwen3-8B	0.337	0.361	0.350	0.292	0.060	0.078	0.070	0.069

Table 4: Zero-shot results on the public test set. For evasion-based clarity we report macro-F1 (F1), precision (P), and recall (R). For evasion we report

\mathrm{ACC}_{\mathrm{match}}

, the fraction of predictions matching at least one annotator, per-annotator macro-F1, and the average macro-F1 across annotators. Best-performing metrics are shown in bold.

Panel of Figure 1 — Figure 1: Row-normalised confusion matrices on the test set for RoBERTa-large model. Rows correspond to gold labels and columns to predicted labels; values represent row-wise proportions such that each row sums to 1. Raw counts are averaged across random seeds and, for the evasion task, across annotators prior to row normalisation.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 2 — Figure 2: An example from the CLARITY dataset

Label	Definition
Clarity Level
Clear Reply	Answers that admit a single interpretation.
Clear Non-Reply	Answers that openly refuse to share information.
Ambivalent Reply	Valid answers that allow for multiple interpretations.
Evasion Level
Explicit	The requested information is clearly and directly stated in the expected form.

Appendix

Table 5: Clarity and evasion technique label definitions, adapted from Thomas et al. (2024) .

Figure 3: Class distribution in the training set.

Figure 4: President and clarity level distribution in the training set.

Loss	Evasion-based clarity			Evasion
Loss	F1	P	R	$\mathbf{ACC}_{\mathbf{match}}$	F1 A1	F1 A2	F1 A3	F1 avg
Unweighted	0.661	0.694	0.641	0.539	0.363	0.378	0.374	0.371
	(0.022)	(0.013)	(0.023)	(0.011)	(0.030)	(0.035)	(0.047)	(0.037)
Balanced	0.603	0.602	0.642	0.515	0.350	0.347	0.327	0.341
	(0.077)	(0.098)	(0.023)	(0.049)	(0.033)	(0.035)	(0.047)	(0.038)
Sqrt	0.602	0.617	0.625	0.505	0.318	0.304	0.302	0.308

Appendix

Table 7: RoBERTa-large loss-weighting ablation on the public test set. For evasion-based clarity we report macro-F1 (F1), precision (P), and recall (R). For evasion we report

\mathrm{ACC}_{\mathrm{match}}

, the fraction of predictions matching at least one annotator, per-annotator macro-F1, and the average macro-F1 across annotators. All models are fine-tuned on evasion labels, with clarity inferred via the taxonomy. Results are averaged over three seeds (mean on the main row, standard deviation in brackets on the row beneath). Best-performing metrics are shown in bold.

Input	Evasion-based clarity			Evasion
Input	F1	P	R	$\mathbf{ACC}_{\mathbf{match}}$	F1 A1	F1 A2	F1 A3	F1 avg
Segmented	0.518	0.520	0.534	0.473	0.277	0.272	0.261	0.270
	(0.019)	(0.018)	(0.021)	(0.004)	(0.015)	(0.016)	(0.003)	(0.004)
Marked	0.595	0.589	0.619	0.516	0.349	0.332	0.333	0.338
	(0.028)	(0.035)	(0.030)	(0.014)	(0.024)	(0.007)	(0.008)	(0.011)

Appendix

Table 8: RoBERTa-base input representation ablation on the public test set. For evasion-based clarity we report macro-F1 (F1), precision (P), and recall (R). For evasion we report

\mathrm{ACC}_{\mathrm{match}}

Input	Direct clarity
Input	F1	P	R
Segmented	0.573	0.552	0.621
	(0.010)	(0.018)	(0.020)
Marked	0.598	0.586	0.636
	(0.010)	(0.013)	(0.021)

Appendix

Table 9: RoBERTa-base input representation ablation for direct clarity on the public test set. We report macro-F1 (F1), precision (P), and recall (R). Results are averaged over three seeds (mean on the main row, standard deviation in brackets on the row beneath). Best-performing metrics are shown in bold.

Split Method	Evasion-based clarity			Evasion
Split Method	F1	P	R	$\mathbf{ACC}_{\mathbf{match}}$	F1 A1	F1 A2	F1 A3	F1 avg
Label-stratified	0.661	0.694	0.641	0.539	0.363	0.378	0.374	0.371
	(0.022)	(0.013)	(0.023)	(0.011)	(0.030)	(0.035)	(0.047)	(0.037)
President disjoint	0.624	0.642	0.624	0.529	0.330	0.306	0.318	0.318
	(0.029)	(0.027)	(0.035)	(0.012)	(0.036)	(0.033)	(0.058)	(0.039)

Appendix

Table 10: RoBERTa-large split-strategy ablation on the public test set. For evasion-based clarity we report macro-F1 (F1), precision (P), and recall (R). For evasion we report

\mathrm{ACC}_{\mathrm{match}}

Clarity label	Support	P	R	F1
Clear Reply	79	0.550	0.608	0.577
		(0.005)	(0.046)	(0.019)
Ambivalent	206	0.801	0.794	0.798
		(0.003)	(0.022)	(0.011)
Clear Non-Reply	23	0.733	0.522	0.608
		(0.039)	(0.087)	(0.072)

Appendix

Table 11: Per-label clarity (evasion-based) on the test set (RoBERTa-large). Clarity labels are inferred from evasion via the taxonomy. Support is the number of test instances per label. Results are averaged over 3 seeds (13, 21, 42): mean on the main row, sample standard deviation in brackets on the row beneath.

Evasion label	Support			F1
Evasion label	A1	A2	A3	F1 A1	F1 A2	F1 A3	F1 avg
Explicit	106	53	80	0.589	0.536	0.573	0.566
				(0.033)	(0.027)	(0.028)	(0.030)
Implicit	54	54	67	0.242	0.257	0.333	0.277
				(0.083)	(0.048)	(0.077)	(0.069)
Dodging	58	72	43	0.469	0.371	0.381	0.407

Appendix

Table 12: Per-label evasion F1 on the test set (RoBERTa-large). For each evasion label, we report F1 against each annotator (A1-A3) and the average across annotators. Support indicates the number of test instances for that label for each annotator. Results are averaged over 3 seeds (13, 21, 42): mean on the main row, sample standard deviation in brackets on the row beneath.

Setting	RoBERTa-base	RoBERTa-large	DeBERTa-v3-base	DeBERTa-v3-large
Max input length	512
Learning rate	$2\times 10^{-5}$
Warmup ratio	0.1
Weight decay	0.01
Dropout	0.1
Max epochs	20

Appendix

Table 13: Training and optimisation settings for all encoder experiments. All hyperparameters are shared across models except batch size and gradient accumulation steps, which are reduced for DeBERTa-v3-large to accommodate its larger memory footprint (effective batch size 32 throughout).

Political Discourse Analysis

Political discourse analysis studies how language expresses ideology, frames issues, persuades audiences, and shapes political debate.

See all 66 papers

SG-UniBuc-NLP at SemEval-2026 Task 6: Multi-Head RoBERTa with Chunking for Long-Context Evasion Detection
Apr 29, 2026Gabriel Stefan, Sergiu NisioiPolitical Discourse Analysis Text Classification
Faculty of Mathematics and Computer Science University of Bucharest
SemEval-2026 Task 6: CLARITY -- Unmasking Political Question Evasions
Mar 14, 2026Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou +2Political Discourse Analysis Political Communication
National Technical University of Athens · Instituto de Telecomunicações · Instituto Superior Técnico, Universidade de Lisboa +1
ttda704 at SemEval-2026 Task 6: Structured Chain-of-Thought Prompting for Political Evasion Detection
Jun 14, 2026Tai Tran Tan, An Dinh ThienCoT Reasoning Pragmatic Reasoning in Language Models
University of Information Technology, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam

Apr 29, 2026J/K move · Enter open · S save

SG-UniBuc-NLP at SemEval-2026 Task 6: Multi-Head RoBERTa with Chunking for Long-Context Evasion Detection

Gabriel Stefan, Sergiu Nisioi

Faculty of Mathematics and Computer Science University of Bucharest

Political Discourse Analysis Text Classification

Open paper AskPDF arXiv

We describe our system for SemEval-2026 Task 6 (CLARITY: Unmasking Political Question Evasions), which classifies English political interview responses by coarse-grained clarity (3-way) and fine-grained evasion strategy (9-way). Since responses frequently exceed the 512-token limit of standard Transformer encoders, we apply an overlapping sliding-window chunking strategy with element-wise Max-Pooling aggregation over chunk representations. A shared RoBERTa-large encoder supplies two task-specific heads trained jointly via a multi-task objective, with inference-time ensembling over 7-fold stratified cross-validation. Our system achieves a Macro-F1 of 0.80 on Subtask 1 and 0.51 on Subtask 2, ranking 11th in both subtasks.

Abstract

Political Discourse Analysis

Explore similar work

SG-UniBuc-NLP at SemEval-2026 Task 6: Multi-Head RoBERTa with Chunking for Long-Context Evasion Detection

SemEval-2026 Task 6: CLARITY -- Unmasking Political Question Evasions

ttda704 at SemEval-2026 Task 6: Structured Chain-of-Thought Prompting for Political Evasion Detection