CSE-UOI at SemEval-2026 Task 6: A Two-Stage Heterogeneous Ensemble with Deliberative Complexity Gating for Political Evasion Detection

Christos Tzouvaras; Konstantinos Skianis; Athanasios Voulodimos

cs.CLMar 12, 2026

CSE-UOI at SemEval-2026 Task 6: A Two-Stage Heterogeneous Ensemble with Deliberative Complexity Gating for Political Evasion Detection

Authors: Christos Tzouvaras, Konstantinos Skianis, Athanasios Voulodimos

Organizations: University of Ioannina · National Technical University of Athens

Abstract

This paper describes our system for SemEval-2026 Task 6, which classifies clarity of responses in political interviews into three categories: Clear Reply, Ambivalent, and Clear Non-Reply. We propose a heterogeneous dual large language model (LLM) ensemble via self-consistency (SC) and weighted voting, and a novel post-hoc correction mechanism, Deliberative Complexity Gating (DCG). This mechanism uses cross-model behavioral signals and exploits the finding that an LLM response-length proxy correlates strongly with sample ambiguity. To further examine mechanisms for improving ambiguity detection, we evaluated multi-agent debate as an alternative strategy for increasing deliberative capacity. Unlike DCG, which adaptively gates reasoning using cross-model behavioral signals, debate increases agent count without increasing model diversity. Our solution achieved a Macro-F1 score of 0.85 on the evaluation set, securing 3rd place and tied with the second-best reported score.

Figures & tables

Figure 1: Task label taxonomy by Thomas et al. (2024) (evasion labels mapped to 3 clarity classes).

Figure 2: Architecture of our two-stage pipeline. Stage 1 runs Grok and Gemini with

k=5

self-consistency (SC), maps evasion to clarity, and applies asymmetric weighted voting. Stage 2 (DCG) uses Gemini response length and Grok consistency to gate uncertain cases.

System Configuration	Dev F1	Eval F1
Grok-only ( $k=5$ )	0.7836	0.8264
Gemini-only ( $k=5$ )	0.7761	0.8083
Ensemble (Stage 1)	0.7941	0.8122
Ensemble + DCG (Stage 2)	0.8100	0.8505

Table 1: Main Results: Macro-F1 on Development (308 samples) and blind Evaluation (237 samples).

Self-Consistency Sweep (Stage 1)
Ablation Configuration	Dev F1
$k=1$ (single pass)	0.7845
$k=3$	0.7979
$k=5$ (production)	0.7941
Gemini Weight Sweep ( $k=5$ )
$w=0$	0.7859

Table 2: Ablations on Development set (

n=308

DCG Configuration	Dev F1	Eval F1
No DCG (Stage 1 only)	0.7941	0.8122
Dynamic $\theta_{1}$ (per-set $Q_{25}$ )	0.8100	0.8505
Fixed $\theta_{1}$ = Dev $Q_{25}$ applied to Eval	—	0.8416
Fixed $\theta_{1}$ = Eval $Q_{25}$ applied to Dev	0.8089	—

Table 3: Transductive

\theta_{1}

adaptation. Using a threshold from the wrong split degrades performance relative to per-set dynamic adaptation (drop of 0.11–0.89 Macro-F1 points).

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Candidate Signal	$d$ (Amb vs. CR)	$d$ (Amb vs. CNR)
Gemini response length	+1.03 / +1.20	+0.93 / +0.64
Grok response length	+0.55 / +0.56	+0.65 / +0.50
Grok self-consistency	-0.41 / -0.66	-0.54 / -0.73
Gemini self-consistency	-0.59 / -0.68	-0.66 / -0.37

Appendix

Table A1: Effect-size comparison for DCG candidate signals. Values are reported as Development / Evaluation.

Figure A1: Gemini response length by gold clarity class. Ambivalent samples are generally longer than clear classes on both splits.

Model	Length $d$ (A vs. CR)	Length $d$ (A vs. CNR)	Cons. $d$ (A vs. CR)	Cons. $d$ (A vs. CNR)	Unanim.
Grok	+0.55 / +0.56	+0.65 / +0.50	$-0.41$ / $-0.66$	$-0.54$ / $-0.73$	0.65 / 0.69
Gemini	+1.03 / +1.20	+0.93 / +0.64	$-0.59$ / $-0.68$	$-0.66$ / $-0.37$	0.60 / 0.75
Kimi K2.6	+0.60 / +0.82	+0.22 / +0.23	$-0.55$ / $-0.65$	$-0.55$ / $-0.67$	0.58 / 0.69
DS-MAX	+0.45 / +0.87	+0.62 / +0.39	$-0.06$ / $-0.52$	$-0.45$ / $-0.23$	0.54 / 0.64
Gemma 4	+0.27 / +0.50	+0.66 / +0.33	$-0.32$ / $-0.60$	$-0.14$ / $-0.19$	$0.67$ / $0.78$
Qwen 3.6	+0.40 / +0.57	+0.77 / +0.51	$-0.49$ / $-0.71$	$-0.26$ / $-0.29$	0.48 / 0.66

Appendix

Table A2: Diagnostic effect-size profiles for open-source DCG candidates. Cohen’s

d

between gold Ambivalent samples and gold Clear Reply (CR) / Clear Non-Reply (CNR) samples for length and self-consistency, plus the rate of unanimous

k=5

self-consistency outputs. Values are reported as Development / Evaluation. The two indicators used for OSS selection are Length

d

(A vs. CR) for the Gemini-role and Unanim. for the Grok-role; the remaining columns are reported for parallelism with Table A1 . Top block: proprietary baseline. Bottom block: open-source candidates.

Grok-role	Gemini-role	DEV $\Delta$ F1	EVAL $\Delta$ F1	Verdict
Gemma 4	Kimi	$\mathbf{+0.0026}$	$\mathbf{+0.0047}$	$\mathbf{++}$
Kimi	DS-MAX	$+0.0202$	$-0.0125$	$+-$
Kimi	DS-HIGH	$+0.0072$	$-0.0271$	$+-$
Kimi	Qwen 3.6	$-0.0200$	$-0.0092$	$--$
Kimi	Gemma 4	$-0.0147$	$-0.0202$	$--$
Qwen 3.6	Kimi	$-0.0447$	$-0.0322$	$--$

Appendix

Table A3: OSS-only pairings under Cohen’s

d

for self-consistency driven Grok-role selection. None of the eight Cohen’s-

d

-driven alternatives reproduces DCG’s positive transfer on both splits. The first row (Gemma 4 + Kimi, unanim-driven selection) is repeated for reference.

Figure A2: DCG sensitivity to percentile choice. Macro-F1 as

\theta_{1}

moves from the 5th to the 95th percentile on Development and Evaluation.

Figure A3: Agreement $\times$ consistency heatmap (post-DCG). Accuracy by Grok–Gemini agreement and Grok self-consistency bins on Development and Evaluation sets.

Rank	Team	Macro-F1
1	TeleAI	0.89
2	AsymVerify	0.85
3	CSE-UOI	0.85
4	Rasende Rakete	0.83
5	Evaluators	0.83
6	YNU-HPCC	0.83

Appendix

Table B1: Top-10 systems on Subtask 1. Total submissions: 41; median macro-F1: 0.72. Full leaderboard at https://konstantinosftw.github.io/CLARITY-SemEval-2026/ .

Stage / Class	P	R	F1
S1 Ambivalent	0.920	0.777	0.842
S1 Clear Non-Reply	0.710	0.957	0.815
S1 Clear Reply	0.641	0.835	0.725
S2 Ambivalent	0.870	0.874	0.872
S2 Clear Non-Reply	0.808	0.913	0.857
S2 Clear Reply	0.720	0.684	0.701

Appendix

Table C1: Per-class clarity metrics, Development (

n{=}308

Stage / Class	P	R	F1
S1 Ambivalent	0.854	0.752	0.800
S1 Clear Non-Reply	0.806	0.829	0.817
S1 Clear Reply	0.765	0.882	0.820
S2 Ambivalent	0.823	0.872	0.847
S2 Clear Non-Reply	0.936	0.829	0.879
S2 Clear Reply	0.842	0.812	0.826

Appendix

Table C2: Per-class clarity metrics, Evaluation (

n{=}237

Label	n	Grok	Gemini
Explicit	115	0.774	0.861
Implicit	99	0.748	0.667
General	113	0.637	0.611
Dodging	96	0.531	0.531
Deflection	51	0.647	0.490
Partial	14	0.714	0.643

Appendix

Table D1: Evasion-level accuracy per model, Development.

Label	n	Grok	Gemini
Explicit	84	0.881	0.976
Implicit	23	0.522	0.565
General	19	0.632	0.579
Dodging	94	0.362	0.575
Deflection	0	–	–
Partial	23	0.087	0.261

Appendix

Table D2: Evasion-level accuracy per model, Evaluation.

Gold $\backslash$ Pred	CR	Amb	CNR	Total
Clear Reply	54	25	0	79
Ambivalent	21	180	5	206
Clear Non-Reply	0	2	21	23
Total	75	207	26	308

Appendix

Table E1: Stage-2 confusion matrix, Development (

n{=}308

, accuracy

=0.828

). Bold diagonal entries are correct predictions.

Gold $\backslash$ Pred	CR	Amb	CNR	Total
Clear Reply	69	16	0	85
Ambivalent	13	102	2	117
Clear Non-Reply	0	6	29	35
Total	82	124	31	237

Appendix

Table E2: Stage-2 confusion matrix, Evaluation (

n{=}237

, accuracy

=0.844

). Bold diagonal entries are correct predictions.

Language Model Ensembles

Language model ensembles combine predictions or reasoning from multiple models to improve accuracy, reliability, coverage, or efficiency.

See all 134 papers

Duluth at SemEval-2026 Task 6: DeBERTa with LLM-Augmented Data for Unmasking Political Question Evasions
Apr 22, 2026Shujauddin Syed, Ted PedersenPolitical Discourse Analysis Text Classification
Department of Computer Science University of Minnesota Duluth, MN 55812 USA
CLaC at SemEval-2026 Task 6: Response Clarity Detection in Political Discourse
May 4, 2026Nawar Turk, Lucas Miquet-Westphal, Leila KosseimLLM Evaluation Language Model Ensembles
Computational Linguistics at Concordia (CLaC) Lab · Dept. of Computer Science and Software Engineering · Concordia University, Montréal, Québec, Canada
SemEval-2026 Task 6: CLARITY -- Unmasking Political Question Evasions
Mar 14, 2026Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou +2Political Discourse Analysis Political Communication
National Technical University of Athens · Instituto de Telecomunicações · Instituto Superior Técnico, Universidade de Lisboa +1

Apr 22, 2026J/K move · Enter open · S save

Duluth at SemEval-2026 Task 6: DeBERTa with LLM-Augmented Data for Unmasking Political Question Evasions

Shujauddin Syed, Ted Pedersen

Department of Computer Science University of Minnesota Duluth, MN 55812 USA

Political Discourse Analysis Text Classification

Open paper AskPDF arXiv

This paper presents the Duluth approach to SemEval-2026 Task 6 on CLARITY: Unmasking Political Question Evasions. We address Task 1 (clarity-level classification) and Task 2 (evasion-level classification), both of which involve classifying question--answer pairs from U.S.\ presidential interviews using a two-level taxonomy of response clarity. Our system is based on DeBERTa-V3-base, extended with focal loss, layer-wise learning rate decay, and boolean discourse features. To address class imbalance in the training data, we augment minority classes using synthetic examples generated by Gemini 3 and Claude Sonnet 4.5. Our best configuration achieved a Macro F1 of 0.76 on the Task 1 evaluation set, placing 8th out of 40 teams. The top-ranked system (TeleAI) achieved 0.89, while the mean score across participants was 0.70. Error analysis reveals that the dominant source of misclassification is confusion between Ambivalent and Clear Reply responses, a pattern that mirrors disagreements among human annotators. Our findings demonstrate that LLM-based data augmentation can meaningfully improve minority-class recall on nuanced political discourse tasks.

Abstract

Language Model Ensembles

Explore similar work

Duluth at SemEval-2026 Task 6: DeBERTa with LLM-Augmented Data for Unmasking Political Question Evasions

CLaC at SemEval-2026 Task 6: Response Clarity Detection in Political Discourse

SemEval-2026 Task 6: CLARITY -- Unmasking Political Question Evasions