UZHCL_for_the_win at Touche: Encoder Fine-Tuning, Synthetic Augmentation, and an Underpowered Encoder-LLM Comparison

Stylianos Psychias

cs.CLOct 12, 2026

UZHCL_for_the_win at Touche: Encoder Fine-Tuning, Synthetic Augmentation, and an Underpowered Encoder-LLM Comparison

Authors: Stylianos Psychias

Organizations: University of Zurich, Zurich, Switzerland

Abstract

Online discussions on platforms like Reddit are a productive testbed for fallacy detection: short texts, broad topical range, and the full inventory of informal fallacies. The Touche 2026 lab at CLEF formalises this as a shared task on a Reddit-derived corpus that combines fallacy detection with argument-scheme classification. This paper describes our participation in all three subtasks (ST1 binary fallacy detection, ST2 eight-way fallacy classification, ST3 four-way argument-scheme classification) and both evaluation tracks. Our submitted systems are fine-tuned RoBERTa-large encoders with task-specific heads, augmented with a small batch of LLM-generated synthetic contrastive pairs; for ST2 and ST3 on the enhanced track we additionally added confidence-filtered pseudo-labelled examples from the unlabelled pool. A wider backbone comparison spanning additional encoders (DeBERTa-v3, ModernBERT) and an LLM track (four Qwen variants from 3B to 32B, zero-shot through CoT-LoRA) showed no validated benefit over RoBERTa-large at this corpus scale, with 5-fold variance wide enough that we call the comparison underpowered rather than settled. We also characterise how much of the enhanced track's lift reflects organiser-rewritten inputs carrying label information rather than genuine argument clarification. On the official Touche 2026 test set our systems ranked first on both base-track classification cells (ST2 and ST3) and second on both binary-detection (ST1) tracks, with the two enhanced-track classification cells topped by cascaded and multitask systems. We close with the rare-class limitation on ST3 and the most useful follow-ups. Code: https://github.com/psychias/fallacy_detection_clef_2026

Figures & tables

Panel of Figure 1 — Figure 1: Training and inference pipeline for the submitted RoBERTa-large systems. Real, synthetic, and pseudo-labelled examples are mixed at per-stream loss weights (real at 1.0; the synthetic and pseudo-labelled streams down-weighted per cell, see Appendix B ) into a single fine-tuning loop. The audit gate sits between pseudo-label generation and pseudo-label training (§ 3.4 ). At inference time the trained classifier head is decoded with per-class thresholds for ST3 and default argmax for ST1/ST2.

Subtask	Track	Precision	Recall	F1	exp_id
ST1	base	0.769	0.762	0.761	exp_0161
ST1	enhanced	0.917	0.914	0.914	exp_0035
ST2	base	0.746	0.728	0.730	exp_0032
ST2	enhanced	0.970	0.972	0.970	exp_0072
ST3	base	—	—	0.523	exp_0069
ST3	enhanced	0.732	0.740	0.735	exp_0073

Table 1: Headline dev results for the six submitted runs. P, R, F1 are macro-averaged for ST2 and ST3. Precision and recall are omitted for ST3 base ( exp_0069 ), whose saved evaluation row has a corrupted precision column; its F1-macro is computed independently and is reliable. The two enhanced-track values should be read with the caveat of § 5.2 on the enhanced-track signal.

Subtask	Track	Precision	Recall	F1	Rank
ST1	base	0.770	0.754	0.751	2
ST1	enhanced	0.946	0.945	0.944	2
ST2	base	0.876	0.860	0.859	1
ST2	enhanced	0.958	0.954	0.954	6
ST3	base	0.710	0.724	0.716	1
ST3	enhanced	0.716	0.933	0.748	5

Table 2: Official Touché 2026 test-set results for the six submitted runs (macro-averaged precision, recall, F1). Rank is our placement among participating teams on that (subtask, track) cell. Bold marks the two cells we ranked first on.

Subtask	Track	Recipe	exp_id	F1	$\Delta$
ST1	base	real only	exp_0013	0.716	—
ST1	base	+ synth (sw=0.5)	exp_0033	0.734	+0.018
ST1	enhanced	real only	exp_0022	0.899	—
ST1	enhanced	+ synth (sw=0.5)	exp_0035	0.914	+0.015
ST1	enhanced	+ synth + pseudo @256 (sw=0.5)	exp_0071	0.906	$-$ 0.008
ST2	base	real only	exp_0014	0.711	—

Table 3: Per-subtask augmentation ablation, F1-macro on dev.

\Delta

is the change relative to the immediately preceding row within the same (subtask, track) block. The shorthand @N denotes max_len=N (tokeniser truncation length). Negative deltas are not hidden. a Single-seed unless noted; see § 5.4 for the seed-variance caveat.

Backbone	exp_id	F1 (ST3 enh)
RoBERTa-base	exp_0018	0.675
RoBERTa-large †	exp_0073	0.735 (CV $0.875\pm 0.083$ )
ModernBERT-large	exp_0086	0.719
Qwen2.5-3B zero-shot	exp_0091	0.467
Qwen2.5-3B kNN few-shot	exp_0143	0.529
Qwen2.5-7B CoT-LoRA (5-fold CV)	exp_0180	$0.851\pm 0.114$

Table 4: Matched backbone comparison on ST3 enhanced. Single-run dev F1 unless marked as a 5-fold cross-validation mean

\pm

sd. The submitted row is marked

\dagger

(its 5-fold CV mean is given alongside the single-run number).

Backbone	Subtask	Track	F1
DeBERTa-v3-base	ST3	base	0.277
DeBERTa-v3-large (collapsed)	ST3	base	0.205
Qwen3-4B-Thinking (5-fold CV)	ST2	enh	0.963
Qwen3-4B-Thinking (5-fold CV)	ST3	base	0.448

Table 5: Off-track context rows, reported for completeness and not comparable to the ST3-enhanced column of Table 4 : DeBERTa on ST3 base (the large collapsed under NaN, the base merely underperformed; § 3.2 ) and the Qwen3-4B-Thinking comparator on ST2 enhanced and ST3 base.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Subtask	Track	exp_id	max_len	batch	lr	epochs	extras
ST1	base	exp_0161	512	8	1e-5	12	synth_v002 sw=0.5, raw_concat
ST1	enhanced	exp_0035	256	8	1e-5	12	synth_v002 sw=0.5
ST2	base	exp_0032	256	8	1e-5	15	synth_v002 sw=0.5
ST2	enhanced	exp_0072	384	8	1e-5	12	synth + pseudo sw=0.3
ST3	base	exp_0069	384	8	1e-5	15	synth_v002 sw=0.5
ST3	enhanced	exp_0073	384	8	1e-5	15	synth + pseudo sw=1.0

Appendix

Table 6: Submitted-run hyperparameter grid. sw is the loss weight applied to synthetic (and pseudo-labelled) examples relative to real examples. max_len is the tokeniser truncation length.

Label	Precision	Recall	F1	Support
ST2 (8-way; exp_0072; argmax; $n=67$ )
authority	1.000	1.000	1.000	8
black-white	1.000	1.000	1.000	8
hasty-generalization	1.000	1.000	1.000	8
natural	1.000	0.889	0.941	9
population	0.875	1.000	0.933	7

Appendix

Table 7: Per-class precision, recall, F1, and support on the dev partition for ST2-enhanced ( exp_0072 ,

n=67

) and ST3-enhanced ( exp_0073 ,

n=69

), both under argmax decoding. Values are read from the submitted checkpoints’ saved metrics. The ST3 pe cell rests on a single dev example and is not an informative per-class estimate (see § 5.3 ).

	Predicted
True	ee	ei	pi	pe
ee (epistemic-external)	6	0	0	1
ei (epistemic-internal)	0	48	0	0
pi (practical-internal)	0	0	13	0
pe (practical-external)	0	0	1	0

Appendix

Table 8: Representative dev confusion matrix for ST3-enhanced ( exp_0073 recipe, seed nearest the five-seed mean; macro-F1

0.722

n=69

). Rows are the gold label, columns the argmax prediction. Reconstructed from the released dev_logits.npz .

Panel of Figure 2 — Figure 2: Which prior work fed which component of our pipeline. Solid arrows are routes that shipped; dashed arrows are routes we tried and dropped (DeBERTa-v3-large under NaN, ModernBERT-large underperforming RoBERTa-large, and the LLM alternative that did not out-perform RoBERTa-large under cross-validation within our compute envelope).

Computational Argumentation

Computational argumentation studies how to represent, analyze, and apply arguments and their relationships using formal and computational methods.

See all 99 papers

Fallacy Benchmarks Measure Scheme Recognition, Not Fallacy Detection
Sep 16, 2026Navyansh Singh, Animesh Pathak, Aarav SinghArgument Mining Computational Argumentation
IIIT Naya Raipur
ForEx: A Formal Verification Framework for Explainable Reasoning in Logical Fallacy Detection and Annotation
Jun 20, 2026Pei-Cing Huang, Chienyu Liu, Chan Hsu +3Logical Reasoning Explainability Evaluation
Department of Information Management National Sun Yat-Sen University Kaohsiung, Taiwan · Graduate Institute of Data Science & Information Computing National Chung Hsing University Taichung, Taiwan
Retrieving Relations, Detecting Fallacies: A RAG Approach to Political Debate Analysis
Aug 18, 2026Deborah Dore, Greta Damo, Elena Cabrio +1Computational Argumentation Political Discourse Analysis
Université Côte d’Azur, CNRS, INRIA, I3S, France

Sep 16, 2026J/K move · Enter open · S save

Fallacy Benchmarks Measure Scheme Recognition, Not Fallacy Detection

Navyansh Singh, Animesh Pathak, Aarav Singh

IIIT Naya Raipur

Argument Mining Computational Argumentation Classification Benchmark Validity

Open paper AskPDF arXiv

Fallacy-detection benchmarks pair fallacy classes with a single "valid" or "none" class that takes everything data collection did not label as a fallacy. A detector has two jobs, deciding whether an argument is fallacious and naming which fallacy it commits, and the false-positive rate is meant to measure the first. We show that what these benchmarks actually score is scheme recognition, the ability behind the second job. Their own test sets already show it: when a classifier misses a fallacy, the error lands on "none" rather than on another fallacy type, so detection is failing while classification holds. The reason is what the valid class lacks. The negatives that separate the two jobs are correct arguments using the same argumentation scheme as a fallacy, and they are scarce: nearly absent from the four benchmarks we examined, and rare even under deliberate search. A detector is therefore never tested where recognizing a scheme and judging its use come apart, and can pass on recognition alone. We construct the missing arguments, together with a control condition from the same pipeline that differs only in scheme, so whatever generation contributes, it contributes to both. The classifier labels the scheme-matched negatives as the source fallacy, and labels the wrong-scheme negatives as the scheme they actually use 85.9% of the time and as the source type 0.4%. The classifier has learned which scheme an argument uses, not whether it uses it correctly. The over-flagging follows: a model that scores 16.6% on CoCoLoFa's own valid class flags 58.9% of the constructed arguments. The same dissociation appears in three zero-shot LLM detectors that never saw these benchmarks. We release the items as Scheme Foils. A reported false-positive rate should not be trusted as a measure of detection until the valid class has been audited for scheme-matched coverage.

Abstract

Computational Argumentation

Explore similar work

Fallacy Benchmarks Measure Scheme Recognition, Not Fallacy Detection

ForEx: A Formal Verification Framework for Explainable Reasoning in Logical Fallacy Detection and Annotation

Retrieving Relations, Detecting Fallacies: A RAG Approach to Political Debate Analysis