Towards Robust Numerical Claim Verification

Peter Røysland Aarnes; Vinay Setty

cs.CLSep 30, 2026

Towards Robust Numerical Claim Verification

Authors: Peter Røysland Aarnes, Vinay Setty

Organizations: University of Stavanger · Factiverse AI and University of Stavanger

Abstract

Large language models (LLMs) are widely used for claim verification, yet remain brittle for numerical reasoning: even small changes in value can sharply degrade accuracy. We show that this brittleness persists in frontier LLMs, but can be mitigated through adversarial fine-tuning on numerically perturbed examples. Using parameter-efficient fine-tuning, small Qwen3 models (0.6B $\unicode{x2013}$ 8B) reach 98.7% accuracy on label-flipping perturbations, outperforming larger zero-shot models and frontier systems (GPT-5.4 Pro (74.0%) and Gemini 2.5 Flash (73.9%)). The gains generalise to unseen perturbation types, indicating robust numerical decision boundaries rather than memorised edits. Robustness also transfers without target-domain data, significantly improving cross-lingual performance in Spanish. We further show that the same fine-tuning recipe confers robustness to evidence-side perturbations, using the VitaminC dataset.

Figures & tables

Panel of Figure 1 — Figure 1: Numeration perturbation: rewriting “3.5%” as “six percent” flips the label to FALSE . Zero-shot predicts TRUE (wrong); adversarial fine-tuning predicts FALSE (correct).

Category	T $\rightarrow$ T	T $\rightarrow$ F	F $\rightarrow$ F	Total
Numeration	216	216	1002	1434
Approximation	139	139	784	1062
Range	190	190	843	1223
Masked	✗	216	501	717
Random	✗	216	501	717
Negative	✗	52	91	143

Table 1: QuanTemp test split: sample counts by perturbation type and transition direction. Original claims: 264 True , 615 False .

Panel of Figure 2 — Figure 2: Two PEFT strategies for numerical claim verification. (a) Soft prompt tuning prepends $m$ learnable vectors $P$ to frozen embeddings; (b) QLoRA inserts low-rank residuals into a 4-bit quantized backbone $\Theta$ . Both share a sigmoid veracity head trained on mixture $\mathcal{T}$ of original and perturbed examples.

	Model	Orig.	Approx.	Num.	Range	Neg.	Mask.	Rand.	M. Avg.
\Block 6-1 \rotate Base	$\text{M}_{\text{GPT-5.4}}$	78.12	92.70 +14.58	86.06 +7.94	93.44 +15.32	39.22 -38.90	17.00 -61.12	90.14 +12.02	73.96 -4.16
	$\text{M}_{\text{Gemini-2.5}}$	74.71	91.97 +17.26	86.05 +11.34	92.43 +17.72	53.85 -20.86	19.62 -55.09	92.06 +17.35	73.91 -0.80
	$\text{M}_{\text{Base}}$	86.35	85.05 -1.30	69.59 -16.76	82.35 -4.00	27.27 -59.08	8.87 -77.48	86.98 +0.63	62.80 -23.55
	$\text{M}_{\text{Base-Two-Shot}}$	81.96	88.72 +6.76	73.21 -8.75	84.78 +2.82	28.85 -53.11	12.44 -69.52	90.61 +8.65	66.10 -15.86
	$\text{M}_{\text{32B Base}}$	85.55	88.64 +3.09	78.89 -6.66	86.67 +1.12	36.00 -49.55	16.19 -69.36	86.41 +0.86	67.89 -17.66
	$\text{M}_{\text{Only-Original}}$	95.83	8.63 -87.20	7.41 -88.42	9.47 -86.36	7.69 -88.14	5.56 -90.27	12.50 -83.33	8.65 -87.18

Table 2: T

\rightarrow

F accuracy (%) on unperturbed claims ( Orig. , left of dotted rule) and label-flipping perturbations. Deltas vs. Orig. within each row. M. Avg. is the micro-average of accuracy across perturbation types.

	Model	Orig.	Approx.	Num.	Range	Neg.	Mask.	Rand.	M. Avg.
\Block 6-1 \rotate Base	$\text{M}_{\text{GPT-5.4}}$	95.33	95.84 +0.51	96.51 +1.18	94.65 -0.68	92.22 -3.11	96.69 +1.36	97.75 +2.42	96.03 +0.70
	$\text{M}_{\text{Gemini-2.5}}$	91.48	90.77 -0.71	90.87 -0.61	89.61 -1.87	90.00 -1.48	88.55 -2.93	96.77 +5.29	91.35 -0.13
	$\text{M}_{\text{Base}}$	94.30	95.51 +1.21	94.65 +0.35	92.18 -2.12	93.75 -0.55	94.03 -0.27	97.70 +3.40	94.83 +0.53
	$\text{M}_{\text{Base-Two-Shot}}$	95.42	96.43 +1.01	95.99 +0.57	95.23 -0.19	94.51 -0.91	94.37 -1.05	98.60 +3.18	96.08 +0.66
	$\text{M}_{\text{32B Base}}$	95.24	96.64 +1.40	95.37 +0.13	94.67 -0.57	93.33 -1.91	95.77 +0.53	97.77 +2.53	95.96 +0.72
	$\text{M}_{\text{Only-Original}}$	96.75	97.70 +0.95	96.41 -0.34	96.92 +0.17	96.70 -0.05	96.61 -0.14	98.20 +1.45	97.13 +0.38

Table 3: F

\rightarrow

F accuracy (%) on unperturbed claims ( Orig. ) and False -labelled perturbations. Deltas vs. Orig. within each row; M. Avg. is the micro-average of accuracy across perturbation types.

	Model	Approx.	Num.	Range	M. Avg.
\Block 6-1 \rotate Base	$\text{M}_{\text{GPT-5.4}}$	77.04 -1.08	81.43 +3.31	82.39 +4.27	80.28 +2.16
	$\text{M}_{\text{Gemini-2.5}}$	70.80 -3.91	74.65 -0.06	74.33 -0.38	73.56 -1.15
	$\text{M}_{\text{Base}}$	80.51 -5.84	87.44 +1.09	85.29 -1.06	84.92 -1.43
	$\text{M}_{\text{Base-Two-Shot}}$	73.33 -8.63	82.94 +0.98	79.56 -2.40	79.31 -2.65
	$\text{M}_{\text{32B Base}}$	77.44 -8.11	82.30 -3.25	81.01 -4.54	80.61 -4.94
	$\text{M}_{\text{Only-Original}}$	96.40 +0.57	95.83	95.26 -0.57	95.78 -0.05

Table 4: T

\rightarrow

T accuracy (%) on label-preserving perturbations of originally True claims (Negation, Random, Masked are False -only and excluded). Deltas vs. the unperturbed original; M. Avg. is the micro-average of accuracy across perturbation types.

AVeriTeC 2.0 dev
Model	T1	T2	T3	M. Avg
$\text{M}_{\text{Base}}$	95.65	95.09	96.83	95.31
$\text{M}_{\text{Excl-Approx}}$	89.72 $-$ 5.93	93.73 $-$ 1.37	97.20 $+$ 0.37	92.13 $-$ 3.18
$\text{M}_{\text{All-Perturbs}}$	90.12 $-$ 5.53	95.20 $+$ 0.11	97.20 $+$ 0.37	93.20 $-$ 2.11
CLEF 2025 Spanish
$\text{M}_{\text{Base}}$	95.51	97.17	97.40	96.44

Table 5: Cross-dataset generalisation accuracy (%). T1 = value-preserving; T2 = value-changing; T3 = random/obfuscation. Deltas vs.

\text{M}_{\text{Base}}

. M. Avg. is the micro-averages of accuracy of T1 , T2 and T3 .

	VitaminC		QuanTemp		QuanTemp by perturbation type
Training data	Acc.	Pair	Acc.	M-F1	Orig.	T1	T2	Mask	Rand
None ( $\text{M}_{\text{Base}}$ )	90.96	84.25	—	—	—	—	—	—	—
QuanTemp ( $\text{M}_{\text{All-Perturbs}}$ )	78.35	54.45	97.52	94.62	96.13	94.95	98.80	100.00	99.86
VitaminC ( $\text{M}_{\text{VitaminC}}$ )	98.05	96.56	80.86	69.80	79.29	72.69	92.26	57.88	95.12
Both ( $\text{M}_{\text{Mixed}}$ )	98.25	97.15	98.20	96.10	96.36	96.45	99.45	100.00	99.72

Table 6: Each training regime on both test sets (VitaminC: 11,887 instances, 5,227 pairs; QuanTemp: 6,175). Specialising on either corpus alone costs 17–20 pp on the other, so neither competency is a by-product of the other; training on both recovers it and beats each specialist on its own test set (

+

0.68 pp QuanTemp,

p=1.0\mathrm{e}{-4}

;

+

0.59 pp pair accuracy VitaminC,

p=0.002

). T1

=

value-preserving; T2

=

value-changing; Mask / Rand are the obfuscation variants.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Category	T $\rightarrow$ T	T $\rightarrow$ F	F $\rightarrow$ F	Total
Numeration	775	775	3872	5422
Approximation	539	539	3010	4088
Range	931	931	4868	6730
Masked	✗	775	1939	2714
Random	✗	775	1939	2714
Negative	✗	218	294	512

Appendix

Table 7: QuanTemp train split: sample counts by perturbation type and transition direction. Original claims: 931 True , 2,434 False .

Category	T $\rightarrow$ T	T $\rightarrow$ F	F $\rightarrow$ F	Total
Numeration	273	273	1206	1752
Approximation	197	197	938	1332
Range	324	324	1494	2142
Masked	✗	273	603	876
Random	✗	273	603	876
Negative	✗	75	95	170

Appendix

Table 8: QuanTemp validation split: sample counts by perturbation type and transition direction. Original claims: 324 True , 747 False .

Type	Label	Original claim fragment	Perturbed fragment	Variant
Numeration	T1	“fell to 3.5% ”	“fell to three point five percent ”	T1 (preserving)
			“fell to thirty-five percent ”	T2 (changing)
Approximation	T1	“matched 50 years ”	“matched about 50 years ”	T1 (preserving)
			“matched about 5 years ”	T2 (changing)
Range	T1	“over 30 million filed”	“over 25–35 million filed”	T1 (preserving)
			“over 10–15 million filed”	T2 (changing)

Appendix

Table 9: Perturbation type definitions and examples ( Aarnes and Setty, 2025 ) . T1 = value-preserving; T2 = value-changing; T3 = random replacement and masked obfuscation.

Category	True	False	True	False
	AVeriTeC 2.0 dev		CLEF 2025 Spanish
Num.	74	124	328	3,614
Approx.	64	90	322	3,470
Range	64	94	317	3,397
Rand.	45	62	164	1,807
Neg.	5	9	70	338

Appendix

Table 10: Evaluated perturbed instances for the cross-dataset generalisation splits, by label of the source claim. Numeration, Approximation and Range each contribute a value-preserving (T1) and a value-changing (T2) variant per claim, both of which remain False for False source claims; only the Random obfuscation variant is evaluated, and perturbations whose text duplicates another instance are removed. AVeriTeC 2.0 dev : 119 claims (49T / 70F), 750 instances including originals; CLEF 2025 Spanish : 2,633 claims (200T / 2,433F), 16,460 instances including originals.

Hyperparameter	Value
Epochs	2
Batch size (per device)	5
Gradient accumulation	1
Learning rate	$1\times 10^{-4}$
LR scheduler	Cosine
Warmup steps	50

Appendix

Table 11: QLoRA training hyperparameters.

Panel of Figure 3 — Figure 3: Prompt template for binary fact-checking. The model receives a claim and an evidence passage, then returns a JSON label indicating whether the claim is true or false.

Panel of Figure 4 — Figure 4: Approximation NCP example. The full prompt provides one example per perturbation type and is added to the instruction prompt.

Quantised (Q4_K_M)
Model	T $\to$ F	T $\to$ T	F $\to$ F
Gemini	14.38	16.19	6.64
GPT-5.4	3.00	4.00	2.50
$\text{M}_{\text{Base}}$	17.90	10.30	11.90
$\text{M}_{\text{Base-Two-Shot}}$	2.40	2.80	0.50
$\text{M}_{\text{32B Base}}$	4.40	3.70	1.30

Appendix

Table 12: Parse failure rates (%) by split and quantisation.

Column	Strict	Relaxed	$\Delta$
T $\rightarrow$ F
Orig.	76.16	73.93	$-$ 2.23
Approx.	89.66	94.93	$+$ 5.27
Num.	81.00	85.92	$+$ 4.92
Range	87.23	91.94	$+$ 4.71
Neg.	47.83	72.55	$+$ 24.72

Appendix

Table 13:

\text{M}_{\text{Base-NCP}}

accuracy (%) under strict (used in main tables) vs. relaxed JSON parsing. Relaxed parsing extracts the first {"label":…} substring anywhere in the response, recovering 96.8% of strict-parser rejections (overall parse-failure rate drops from 58.3% to 1.9%). Recovery improves T

\rightarrow

F and F

\rightarrow

F cells but worsens T

\rightarrow

T cells.

Dataset	Protocol	$\text{M}_{\text{Base}}$	$\text{M}_{\text{All-Perturbs}}$	$\Delta$ (pp)	95% CI (pp)	McNemar $p$
CLEF Spanish	Parseable-only (reported)	96.44	97.36	$+$ 0.93	$[+0.53,+1.35]$	—
	Paired ( $n=9{,}090$ )	96.44	98.77	$+$ 2.33	$[+1.99,+2.69]$	$8.0\mathrm{e}{-}41$
	Failures as errors	53.26	97.36	$+$ 44.11	$[+43.34,+44.86]$	$<10^{-300}$
AVeriTeC	Parseable-only (reported)	95.31	93.20	$-$ 2.11	$[-4.51,+0.33]$	—
	Paired ( $n=469$ )	95.31	94.24	$-$ 1.07	$[-3.60,+1.46]$	$0.511$
	Failures as errors	59.60	93.20	$+$ 33.60	$[+29.87,+37.33]$	$4.4\mathrm{e}{-}57$

Appendix

Table 14: Significance of the cross-dataset gains (overall accuracy, %). Deltas are

\text{M}_{\text{All-Perturbs}}

minus

\text{M}_{\text{Base}}

, with a 10,000-resample paired bootstrap and an exact McNemar test on stored per-instance predictions. The three protocols differ only in how

\text{M}_{\text{Base}}

’s unparseable responses are handled: excluded (as in Table 5 ), restricted to the instances both models parse, or scored as errors. On CLEF Spanish the gain is significant under every protocol; on AVeriTeC the apparent parseable-only deficit is not significant once the comparison is paired (

p=0.511

), and the fine-tuned model’s real advantage there is coverage:

\text{M}_{\text{Base}}

returns no parseable verdict on 281 of 750 instances (37.5%) against zero for

\text{M}_{\text{All-Perturbs}}

	True			False
Model	P	R	F1	P	R	F1	M-F1
$\text{M}_{\text{Base}}$	42.4	80.9	55.6	99.4	96.9	98.1	76.88
$\text{M}_{\text{Excl-Approx}}$	65.7	82.5	73.1	99.2	98.1	98.7	85.89
$\text{M}_{\text{All-Perturbs}}$	64.4	81.6	72.0	99.2	98.0	98.6	85.31

Appendix

Table 15: Per-class results on CLEF 2025 Spanish (%). The split is 95.8% False , so overall accuracy is close to the majority-class rate and macro-F1 ( M-F1 ) is the informative figure: fine-tuning improves it by 8.4–9.0 pp, and the improvement is concentrated almost entirely in the minority True class (

+

16.4 pp F1 for

\text{M}_{\text{All-Perturbs}}

), where

\text{M}_{\text{Base}}

’s precision is only 42.4%. Support differs across rows because

\text{M}_{\text{Base}}

is scored on the 9,090 instances it parses (251 True ) whereas the fine-tuned models are scored on all 16,460 (684 True );

\text{M}_{\text{Base}}

fails to parse 7,370 instances (44.8%) against zero for either fine-tuned model.

T $\rightarrow$ F (label-flipping)
Model	Approx.	Num.	Range	Neg.	Rand.	M. Avg.
$\text{M}_{\text{Base}}$	72.97	80.00	82.50	35.29	76.09	73.71
$\text{M}_{\text{Excl-Approx}}$	98.76 $+$ 25.78	97.56 $+$ 17.56	95.57 $+$ 13.07	100.00 $+$ 64.71	96.95 $+$ 20.86	97.49 $+$ 23.78
$\text{M}_{\text{All-Perturbs}}$	98.14 $+$ 25.16	96.34 $+$ 16.34	96.20 $+$ 13.70	100.00 $+$ 64.71	96.34 $+$ 20.25	97.07 $+$ 23.36
(n)	161	164	158	70	164	717
T $\rightarrow$ T (label-preserving)

Appendix

Table 16: CLEF 2025 Spanish accuracy (%) by perturbation type within each label transition. Deltas are against

\text{M}_{\text{Base}}

in the same column. Accuracy is computed on each model’s parseable responses, as in Table 5 . Neg. and Rand. produce False labels by construction and so cannot appear under T

\rightarrow

T . Reporting within transitions separates the 717 label-flipping instances from the 15,059 F

\rightarrow

F ones that dominate this split; the aggregated columns of Table 5 pool them and understate the label-flipping gain by an order of magnitude.

Relative distance	$n$	$\text{M}_{\text{Base}}$	$\text{M}_{\text{All-Perturbs}}$	$\Delta$
0 (value preserved)	1,373	90.17	96.07	$+$ 5.90
$(0,10\%]$	399	87.99	95.49	$+$ 7.50
$(10\%,25\%]$	345	90.11	98.55	$+$ 8.44
$(25\%,50\%]$	661	94.14	99.39	$+$ 5.26
$(50\%,100\%]$	513	94.06	98.83	$+$ 4.77
$>100\%$	503	96.69	99.60	$+$ 2.91

Appendix

Table 17: Accuracy (%) by the relative distance between the original and the perturbed quantity, on the QuanTemp test split (3,794 of 4,579 perturbed instances align to a distance, 83% coverage).

\text{M}_{\text{Base}}

is scored on parseable responses only. Every bin is individually significant (exact McNemar,

p<10^{-2}

throughout and

p<10^{-7}

in three bins). Near-miss values are the harder case for both models, and the gain from adversarial fine-tuning is largest exactly there (

+

7.5 to

+

8.4 pp below 25% distance) and smallest for implausible replacements beyond 100%, which is the opposite of what a model that had merely learned to recognise perturbation artefacts would show.

		Per instance		Per pair	Pair error mode
Model	Prompt	Acc.	Macro-F1	Pair Acc.	Refutes	Confirms	PF
No target-domain training
$\text{M}_{\text{Base}}$	Zero-shot	90.96	90.86	84.25	8.8	8.4	704
Trained on QuanTemp claim-side perturbations
$\text{M}_{\text{All-Perturbs}}$	Zero-shot	78.35 $-$ 12.60	77.63	54.45 $-$ 29.81	15.0	31.5	0
$\text{M}_{\text{All-Perturbs}}$	SCoT	74.98 $-$ 15.98	73.20	46.15 $-$ 38.10	10.7	44.2	0

Appendix

Table 18: Results on the VitaminC numerical test subset (11,887 instances; 5,227 contrastive pairs). Pair Acc. is the share of contrastive pairs for which both snapshots of a revision are labelled correctly. Refutes is the share of pairs whose supporting snapshot is wrongly refuted, and Confirms the share whose refuting snapshot is wrongly confirmed; the asymmetry between them identifies the failure mode. PF counts parse failures, which are scored as errors throughout so that all rows share the same denominator. Deltas are against

\text{M}_{\text{Base}}

. SCoT

=

Structured Chain-of-Thought, NCP

=

Numerical Counterexample Prompt, TCP

=

Threshold Comparison Prompt.

	Model	Orig.	Approx.	Num.	Range	Neg.	Mask.	Rand.	Micro Avg.
\Block 5-1 \rotate Base	$\text{M}_{\text{Base}}$	94.35	81.82 -12.53	35.94 -58.41	78.18 -16.17	3.70 -90.65	2.50 -91.85	71.43 -22.92	48.74 -45.61
	$\text{M}_{\text{Base-Two-Shot}}$	86.92	86.61 -0.31	62.50 -24.42	80.00 -6.92	21.74 -65.18	8.63 -78.29	84.70 -2.22	60.28 -26.64
	$\text{M}_{\text{32B Base}}$	88.24	85.86 -2.38	66.42 -21.82	85.34 -2.90	29.27 -58.97	7.29 -80.95	83.80 -4.44	61.90 -26.34
	$\text{M}_{\text{Only-Original}}$	94.70	8.63 -86.07	6.94 -87.76	9.47 -85.23	7.69 -87.01	6.94 -87.76	15.28 -79.42	9.42 -85.28
\Block 4-1 \rotate All Perturb	$\text{M}_{\text{0.6B All-Perturbs}}$	81.06	93.53 +12.47	75.00 -6.06	83.68 +2.62	100.00 +18.94	99.54 +18.48	96.76 +15.70	90.09 +9.03
	$\text{M}_{\text{1.7B All-Perturbs}}$	85.23	89.93 +4.70	88.43 +3.20	88.42 +3.19	98.08 +12.85	99.54 +14.31	94.91 +9.68	92.81 +7.58

Appendix

Table 19: T

\rightarrow

F accuracy (%) for Qwen3-8B (unquantized fp16), selected models. Columns as in Table 2 .

	Model	Orig.	Approx.	Num.	Range	Micro Avg.
\Block 5-1 \rotate Base	$\text{M}_{\text{Base}}$	94.35	93.67 -0.68	97.33 +2.98	98.17 +3.82	96.69 +2.34
	$\text{M}_{\text{Base-Two-Shot}}$	86.92	80.83 -6.09	86.14 -0.78	89.41 +2.49	85.93 -0.99
	$\text{M}_{\text{32B Base}}$	88.24	89.29 +1.05	90.37 +2.13	87.84 -0.40	89.21 +0.97
	$\text{M}_{\text{Only-Original}}$	94.70	95.68 +0.98	95.37 +0.67	94.21 -0.49	95.04 +0.34
\Block 4-1 \rotate All Perturb	$\text{M}_{\text{0.6B All-Perturbs}}$	81.06	84.17 +3.11	78.70 -2.36	87.37 +6.31	83.12 +2.06
	$\text{M}_{\text{1.7B All-Perturbs}}$	85.23	89.93 +4.70	85.65 +0.42	91.58 +6.35	88.81 +3.58

Appendix

Table 20: T

\rightarrow

T accuracy (%) for Qwen3-8B (unquantised, BF16/FP16), selected models. Columns as in Table 4 .

	Model	Orig.	Approx.	Num.	Range	Neg.	Mask.	Rand.	Micro Avg.
\Block 5-1 \rotate Base	$\text{M}_{\text{Base}}$	91.69	95.73 +4.04	94.37 +2.68	86.88 -4.81	88.10 -3.59	92.52 +0.83	95.91 +4.22	92.98 +1.29
	$\text{M}_{\text{Base-Two-Shot}}$	94.95	96.77 +1.82	96.30 +1.35	92.31 -2.64	93.10 -1.85	93.62 -1.33	98.33 +3.38	95.42 +0.47
	$\text{M}_{\text{32B Base}}$	95.99	96.69 +0.70	95.87 -0.12	94.48 -1.51	93.75 -2.24	95.50 -0.49	98.21 +2.22	96.09 +0.10
	$\text{M}_{\text{Only-Original}}$	97.56	97.45 -0.11	97.21 -0.35	96.92 -0.64	95.60 -1.96	96.81 -0.75	98.40 +0.84	97.30 -0.26
\Block 4-1 \rotate All Perturb	$\text{M}_{\text{0.6B All-Perturbs}}$	95.93	95.15 -0.78	95.41 -0.52	88.86 -7.07	100.00 +4.07	100.00 +4.07	100.00 +4.07	96.30 +0.37
	$\text{M}_{\text{1.7B All-Perturbs}}$	96.59	94.64 -1.95	95.21 -1.38	92.18 -4.41	100.00 +3.41	100.00 +3.41	99.40 +2.81	96.64 +0.05

Appendix

Table 21: F

\rightarrow

F accuracy (%) for Qwen3-8B (unquantized fp16), selected models. Columns as in Table 3

Explore similar work

DS@GT ARC at CheckThat! 2026: LLM-Based Trace Ranking and Grouped Reward Modeling for Multilingual Numerical Claim Verification
Jul 27, 2026Sagnik Sinha, Shreyas ShresthaCategory-Aware Atomic Claims LLM Reasoning Strategies
Georgia Institute of Technology, North Ave, Atlanta, GA 30332, United States
Testing LLM Arithmetic Reasoning Generalization with Automatic Numeric-Remapping Attacks
Jun 2, 2026Malia Barker, Bishal Lakha, Edoardo Serra +1Mathematical Reasoning Benchmarks LLM Reasoning Strategies
Department of Computer Science, Boise State University, Boise, ID, USA · University of L'Aquila, L'Aquila, Italy
Small LLMs for Biomedical Claim Verification: Cost-Effective Fine-Tuning, Structural Dataset Shortcuts, and Cross-Domain Generalization
Jun 11, 2026Gaurav KumarSmall Large Language Models Model Fine-Tuning
Moveworks AI · University of California San Diego

Jul 27, 2026, cs.CLJ/K move · Enter open · S save

DS@GT ARC at CheckThat! 2026: LLM-Based Trace Ranking and Grouped Reward Modeling for Multilingual Numerical Claim Verification

Sagnik Sinha, Shreyas Shrestha

Georgia Institute of Technology, North Ave, Atlanta, GA 30332, United States

Category-Aware Atomic Claims LLM Reasoning Strategies Reasoning Traces Text Corpora Ranking

Open paper AskPDF arXiv

Automated verification of numerical claims is a challenging problem, as it requires both language understanding and quantitative reasoning. This paper describes our system for CLEF 2026 CheckThat! Task 2, which focuses on ranking reasoning traces generated by large language models (LLMs) and predicting a final verdict for numerical claims in English and Arabic. We explore two approaches. The first approach fine-tunes an LLM-based verifier using LoRA to score each reasoning trace independently as a binary classification problem, and selects the final verdict using Best-of-N selection. We further experiment with adaptive sub-claim decomposition to break complex claims into simpler parts before verification. The second approach uses a lightweight TF-IDF reward model with handcrafted numeric and temporal overlap features to score traces, and aggregates scores by verdict group to determine the final prediction. For Arabic, we compare a general multilingual model against AraBERT, a language-specific model pretrained on Arabic text. Our results show that the LLM-based approach outperforms the lightweight reward model on most metrics, particularly Recall@5, while the reward-based approach shows stronger performance on the Conflicting class. Sub-claim decomposition did not improve performance, suggesting that claim splitting introduces noise rather than aiding reasoning. For Arabic, AraBERT outperforms the multilingual baseline across most metrics.

Abstract

Explore similar work

DS@GT ARC at CheckThat! 2026: LLM-Based Trace Ranking and Grouped Reward Modeling for Multilingual Numerical Claim Verification

Testing LLM Arithmetic Reasoning Generalization with Automatic Numeric-Remapping Attacks

Small LLMs for Biomedical Claim Verification: Cost-Effective Fine-Tuning, Structural Dataset Shortcuts, and Cross-Domain Generalization