Organizations: National Technical University of Athens · Instituto de Telecomunicações · Instituto Superior Técnico, Universidade de Lisboa · ELLIS Unit Lisbon
Political speakers often avoid answering questions directly while maintaining the appearance of responsiveness. Despite its importance for public discourse, such strategic evasion remains underexplored in Natural Language Processing. We introduce SemEval-2026 Task 6, CLARITY, a shared task on political question evasion consisting of two subtasks: (i) clarity-level classification into Clear Reply, Ambivalent, and Clear Non-Reply, and (ii) evasion-level classification into nine fine-grained evasion strategies. The benchmark is constructed from U.S. presidential interviews and follows an expert-grounded taxonomy of response clarity and evasion. The task attracted 124 registered teams, who submitted 946 valid runs for clarity-level classification and 539 for evasion-level classification. Results show a substantial gap in difficulty between the two subtasks: the best system achieved 0.89 macro-F1 on clarity classification, surpassing the strongest baseline by a large margin, while the top evasion-level system reached 0.68 macro-F1, matching the best baseline. Overall, large language model prompting and hierarchical exploitation of the taxonomy emerged as the most effective strategies, with top systems consistently outperforming those that treated the two subtasks independently. CLARITY establishes political response evasion as a challenging benchmark for computational discourse analysis and highlights the difficulty of modeling strategic ambiguity in political language.
Figures & tables
Figure 1: Annotators’ agreement using Fleiss κ for labels assigned to the clarity classification level.
Figure 2: Annotators’ agreement using Fleiss κ for labels assigned to the evasion classification level.
Split
# QA Pairs
Training
3448
Validation
308
Test (Evaluation Phase)
237
Table 1: Number of question–answer (QA) pairs in each split of the CLARITY dataset.
Taxonomy
Model
Macro-F1
Clarity-level (Subtask 1)
Llama-70b
0.82
Evasion-level (Subtask 2)
Llama-70b
0.57
Table 2: Baselines for clarity and evasion levels, reached by a fine-tuned Llama 70b model.
Subtask 1: Clarity Level
Subtask 2: Evasion Level
Rank
Team
Macro-F1
Rank
Team
Macro-F1
1
TeleAI
0.89
1
TeleAI
0.68
2
AsymVerify
0.85
2
moswisarut
0.61
3
CSE-UOI
0.85
3
CLaC @ CLARITY
0.59
4
Rasende Rakete
0.83
4
Rasende Rakete
0.59
5
Evaluators
0.83
5
YNU-HPCC
0.59
Table 3: Leaderboard results for both subtasks ranked by Macro-F1 score. Full leaderboard available at task website .
We describe our system for SemEval-2026 Task 6 (CLARITY: Unmasking Political Question Evasions), which classifies English political interview responses by coarse-grained clarity (3-way) and fine-grained evasion strategy (9-way). Since responses frequently exceed the 512-token limit of standard Transformer encoders, we apply an overlapping sliding-window chunking strategy with element-wise Max-Pooling aggregation over chunk representations. A shared RoBERTa-large encoder supplies two task-specific heads trained jointly via a multi-task objective, with inference-time ensembling over 7-fold stratified cross-validation. Our system achieves a Macro-F1 of 0.80 on Subtask 1 and 0.51 on Subtask 2, ranking 11th in both subtasks.
Gabriel Stefan, Sergiu Nisioi
Faculty of Mathematics and Computer Science University of Bucharest
This paper describes the KCLarity team's participation in CLARITY, a shared task at SemEval 2026 on classifying ambiguity and evasion techniques in political discourse. We investigate two modelling formulations: (i) directly predicting the clarity label, and (ii) predicting the evasion label and deriving clarity through the task taxonomy hierarchy. We further explore several auxiliary training variants and evaluate decoder-only models in a zero-shot setting under the evasion-first formulation. Overall, the two formulations yield comparable performance. Among encoder-based models, RoBERTa-large achieves the strongest results on the public test set, while zero-shot GPT-5.2 generalises better on the hidden evaluation set.
This paper describes our system for SemEval-2026 Task 6, which addresses the classification of political evasion strategies in English question-answer pairs extracted from U.S. presidential interviews. We systematically compare two distinct paradigms: (1) Parameter-Efficient Fine-Tuning of Qwen3 models (4B-32B) using QLoRA, enhanced with tiered upsampling and weighted cross-entropy loss to address severe class imbalance, and (2) structured Chain-of-Thought (CoT) prompting of reasoning-capable API models, namely DeepSeek-V3.2 and Grok-4-Fast. Our evaluation demonstrates that structured CoT prompting of reasoning-enabled models substantially outperforms our baseline parameter-efficient fine-tuning implementation in absolute Macro F1. Our best system, Grok-4-Fast with extended reasoning and few-shot hierarchical CoT prompting, achieves a Macro F1 of 0.5147 on Subtask 2 (9-class evasion) and 0.7979 on Subtask 1 (3-class clarity), ranking 8th out of 33 teams on Subtask 2 and 13th out of 41 teams on Subtask 1 on the official leaderboard. Furthermore, our ablation studies reveal key insights into effective prompt design for evasion detection: presenting labels within a hierarchical taxonomy helps structure model reasoning, while few-shot exemplars provide task calibration. However, the strongest prompt variants are not statistically distinguishable in Macro F1, and explicitly enabling extended reasoning modes yields substantial performance gains by facilitating the multi-step pragmatic analysis required to detect evasive intent.
Tai Tran Tan, An Dinh Thien
University of Information Technology, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam