Understanding Errors in LLM-Based Question Answering over Imperfect Tables

Baowen Zhang; Wei Fan; Ruman Wang; Hangting Ye

cs.CLOct 3, 2026

Understanding Errors in LLM-Based Question Answering over Imperfect Tables

Authors: Baowen Zhang, Wei Fan, Ruman Wang, Hangting Ye

Organizations: University of Wisconsin–Madison · University of Auckland, Auckland, New Zealand · Liaoning Provincial People’s Hospital · School of Artificial Intelligence, Jilin University

Abstract

Answering questions over imperfect tables requires handling errors that can affect the answer. We investigate two challenges for large language models (LLMs): whether error discovery depends on where errors appear in a table, and whether providing their locations is sufficient for accurate question answering (QA). Using human-reviewed instances from RADAR-T, we conduct controlled studies across three LLMs by varying row order and comparing original, error-marked, and repaired tables. First, reordering rows changes error discovery even when the table contents and gold answer remain unchanged. During direct inspection, LLMs are more likely to discover all rows containing relevant errors when these rows appear later in the table or are grouped more closely together. Second, providing verified error locations alone is insufficient for accurate QA: with code execution, accuracy on repaired tables exceeds that on error-marked tables by 39.0-59.1 percentage points across the three LLMs. As a practical application of these findings, we combine error discovery across shuffled table views with explicit guidance for verifying and handling the reported errors in a simple workflow, Geometry-Balanced Discovery and Intervention (GBDI). On RADAR-T, GBDI improves QA accuracy by 3.8-18.5 percentage points over a code-agent baseline across five LLMs (paired 95% confidence intervals exclude zero for four), at the cost of additional inference. These results highlight the importance of both reliable error discovery and effective error handling in QA over imperfect tables. Code is available at https://github.com/645-t/GBDI-ICLR-2027.

Figures & tables

Figure 1: Position and dispersion.

Panel of Figure 2 — Figure 2: Complete discovery rate under (a) Direct and (b) Code Agent. In each subfigure, rows correspond to systems; the left column varies position (q1–q5, front to back), and the right column varies dispersion (Wide to Compact). Tick labels report mean realized $\mu$ or $\sigma$ .

Figure 3: The shared Direct task template (top) and three input conditions (below). Each notice, when present, precedes the task in the actual prompt. {table} receives the corresponding table; {question} is unchanged. Code Agent uses the same notices with its own task template. Full prompts appear in Appendix F .

Figure 4: Final-answer accuracy (%) on 154 reviewed RADAR-T instances. Raw uses the original table, Discovery (error-marked) marks critical cells, and Action (repaired) provides the table with reviewed cell corrections and Drop row removals already applied. Paired intervals are reported in Appendix E .

Figure 5: GBDI workflow.

M

independently shuffled views of the input table are inspected in separate Direct contexts. Candidate errors and supporting evidence are mapped to the original rows and merged. The Code Agent verifies each nomination, applies supported actions to a working table, and computes the final answer. The highlighted row illustrates a change in position across views.

Figure 6: QA accuracy (%) on 313 RADAR-T instances. Numerical results and paired CIs are in Appendix E.4 , Table 20 .

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

RADAR task ID and question summary	Source	Reviewed	Position	Dispersion
actor-age-gaps Mean age gap for couples with an older male actor.	D12	M B O F L	M O F L	M O F L
board-games-min-players Count games meeting both player-count limits.	D11	M B O F L	M B L	M B L
car-co2 Mean estimated car emissions for 2018–2023.	D06	M B O F L	F	F
daily-activity Mean distance share from moderate-or-higher activity.	D20	M B O F L	O F	O F
employee-years Count older employees with at least five years of tenure.	D25	M B O F L	—	—
england-wales-ethnicity Find the area ranked seventh by Black Caribbean population.	D14	M B O F L	—	—

Appendix

Table 1: Task coverage and exact error-type membership of the three cohorts. Each listed error type identifies one RADAR-T instance.

Source	Dataset
D01	2021 Green Taxi Trip Data
D02	2014–15 to 2017–19 NYC Regents Exam Results – Public
D03	Emissions from Industrial Facilities in Queensland – 2004
D04	Traffic Violations
D05	Tracking data, Subject (a) MC Motion
D06	Fuel Economy Data

Appendix

Table 2: Source datasets represented in the human-reviewed reference. Source identifiers and dataset names follow RADAR.

System	API identifier	Provider
Qwen Plus	qwen-plus	Bailian
MiniMax-M2.5	MiniMax-M2.5	Bailian
GPT-5 Mini	openai/gpt-5-mini	OpenRouter/OpenAI
DeepSeek-V4.1-Flash	deepseek-flash	DeepSeek native API
GLM-4.5-Air	glm-4.5-air	Bailian

Appendix

Table 3: Systems evaluated on RADAR-T and their API providers. API identifiers are request identifiers rather than checkpoint hashes.

(a) Position $\mu$
		Direct				Code Agent
System	Layout	CDR	Prec.	Rec.	F1	CDR	Prec.	Rec.	F1
Qwen Plus	q1	61.1	70.6	76.8	70.2	66.1	68.8	72.4	67.5
	q2	68.9	72.4	77.8	71.9	75.6	75.7	80.1	74.8
	q3	66.7	71.7	76.0	70.9	76.1	76.1	82.2	76.9
	q4	70.6	75.5	79.5	74.7	73.9	75.4	79.2	75.1

Appendix

Table 4: Discovery metrics by critical-row position and dispersion. Precision (Prec.), recall (Rec.), and F1 are macro averages; all entries are percentages. Each position band uses 180 views of 60 instances. Each dispersion level uses 531 views of 59 instances, averaged over three values of

\mu

Target $\mu$	Compact	Medium	Wide
25%	0.2495 / 0.0081	0.2495 / 0.0677	0.2495 / 0.1268
50%	0.5008 / 0.0081	0.5008 / 0.0677	0.5008 / 0.1268
75%	0.7499 / 0.0081	0.7499 / 0.0677	0.7499 / 0.1268

Appendix

Table 5: Realized geometry of the

3\times 3

layout design. Each cell gives mean position

\bar{\mu}

/ mean dispersion

\bar{\sigma}

System	$\mu$	Dispersion	CDR	Prec.	Rec.	F1
Qwen Plus	25%	Compact	71.2	72.7	80.2	73.3
		Medium	53.1	71.9	75.8	70.5
		Wide	52.0	73.0	74.0	70.0
	50%	Compact	70.1	75.9	81.0	75.7
		Medium	59.3	77.2	79.1	74.9
		Wide	54.2	76.9	77.6	73.8

Appendix

Table 6: Joint position–dispersion discovery metrics under Direct. Each condition uses the same 59 instances and three backgrounds (177 views). Precision, recall, and F1 are macro averages; all entries are percentages.

System	$\mu$	Dispersion	CDR	Prec.	Rec.	F1
Qwen Plus	25%	Compact	68.9	72.4	76.0	71.3
		Medium	70.6	72.4	76.7	72.0
		Wide	68.9	73.2	76.4	71.8
	50%	Compact	71.8	75.6	78.3	74.8
		Medium	68.4	78.1	80.5	77.2
		Wide	72.9	76.3	80.4	75.9

Appendix

Table 7: Joint position–dispersion discovery metrics under Code Agent. Each condition uses the same 59 instances and three backgrounds (177 views). Precision, recall, and F1 are macro averages; all entries are percentages.

Figure 7: Direct discovery on WikiSQL missingness instances. Position runs from q1 (front) to q5 (back); dispersion runs from Wide to Compact. Each point averages 123 views of the same 41 instances.

(a) Position
	Qwen-Turbo				Qwen3-8B
Layout	CDR	Prec.	Rec.	F1	CDR	Prec.	Rec.	F1
q1	70.7	47.2	74.0	49.5	60.2	20.9	66.1	26.0
q2	81.3	74.6	83.2	75.7	54.5	38.9	58.8	42.1
q3	82.9	77.7	84.0	78.4	52.8	46.1	58.3	48.4
q4	81.3	75.8	83.5	76.8	62.6	57.1	66.1	58.9

Appendix

Table 8: Discovery metrics on WikiSQL missingness instances. All entries are percentages; precision, recall, and F1 are macro averages. Each system uses 615 position views and 369 dispersion views.

System	Views	CDR	Prec.	Rec.	F1	Mean rows	Empty FP
Qwen Plus	Repeated-5	58.9	71.0	77.1	68.9	9.11	61.5
Qwen Plus	Random-5	78.7	72.5	85.6	74.6	10.03	76.9
GLM-4.5-Air	Repeated-5	61.0	60.5	74.6	62.5	9.11	84.6
GLM-4.5-Air	Random-5	66.7	59.7	76.7	62.3	10.24	76.9
MiniMax-M2.5	Repeated-5	38.3	47.1	48.9	45.4	4.98	38.5
MiniMax-M2.5	Random-5	58.2	65.1	69.4	64.3	7.14	23.1

Appendix

Table 9: Discovery under an equal five-call allowance. Repeated-5 uses the released row order; Random-5 permutes all rows independently in each call. CDR and macro precision, recall, and F1 use 141 nonempty references. Mean rows uses all 154 reviewed instances; Empty FP uses 13 empty references. All rate columns are percentages.

(a) View family $\times$ guidance
Views	Generic	Skill	$\Delta_{\mathrm{Skill}}$	95% CI
Repeated-5	172 (55.0%)	190 (60.7%)	$+5.8$	$[-0.7,12.0]$
Random-5	183 (58.5%)	203 (64.9%)	$+6.4$	$[1.6,11.2]$
(b) Random-5 vs. Repeated-5
Metric	Repeated-5	Random-5	$\Delta_{\mathrm{view}}$	95% CI
Complete discovery rate	58.9%	78.7%	$+19.9$	$[11.1,29.2]$

Appendix

Table 10: Controlled tests of the two findings with Qwen Plus. (a) QA accuracy on all 313 instances when intervention guidance is varied while the discovery ledger is held fixed within each view family. (b) Random-5 varies row order across five inspections, whereas Repeated-5 repeats the released order at the same call allowance; the QA comparison holds Skill guidance fixed. Differences and intervals are reported in percentage points.

		Generic		Skill
Input type	$n$	Repeated-5	Random-5	Repeated-5	Random-5
Bad values	53	26	28	26	25
Clean	53	46	49	47	49
Logic	53	18	23	17	28
Formatting	53	29	28	40	38
Missingness	53	25	25	31	30

Appendix

Table 11: Counts of correct QA answers for the five error types and clean inputs in the Qwen Repeated-5/Random-5 guidance factorial.

Guidance	Correct	QA (%)	$\Delta$ (pp)	95% CI (pp)
Full Skill	203/313	64.9	–	–
Without recovery rules	186/313	59.4	$-5.4$	$[-11.3,0.0]$
Without formatting rules	190/313	60.7	$-4.2$	$[-7.6,-0.4]$
Without execution checks	197/313	62.9	$-1.9$	$[-6.7,3.0]$

Appendix

Table 12: Qwen Plus final-answer accuracy on all 313 RADAR-T instances when one part of the Intervention Skill is removed. Differences are variant minus Full Skill, in percentage points; intervals use paired source-table cluster resampling.

System	Views	Guidance	$D^{+}A^{+}$	$D^{+}A^{-}$	$D^{-}A^{+}$	$D^{-}A^{-}$
Qwen Plus	Repeated	Generic	37	46	27	31
	Repeated	Skill	51	32	28	30
	Random	Generic	57	54	16	14
	Random	Skill	72	39	16	14
GLM-4.5-Air	Repeated	Generic	32	54	7	48
	Repeated	Skill	38	48	16	39

Appendix

Table 13: Discovery completeness and final-answer outcomes on the 141 reviewed instances with at least one critical row. Complete means that the union of five inspections contains every critical row. Counts are descriptive paired outcomes; the 13 empty-critical-row instances are excluded because completeness is vacuous for them.

(a) View budget with union ledger ( $k=1$ )
$M$	CDR	Prec.	Rec.	F1	Mean rows	Empty FP
1	45.5	71.8	71.8	68.6	6.92	64.6
2	65.9	72.6	80.5	72.9	8.41	70.0
3	73.5	72.7	83.2	73.9	9.11	73.1
4	76.7	72.7	84.6	74.4	9.61	75.4
5	78.7	72.5	85.6	74.6	10.03	76.9

Appendix

Table 14: Qwen Plus sensitivity to the Random-view budget

M

and ledger support

k

. Panel (a) averages over all subsets of size

M

from five views. CDR and macro precision, recall, and F1 use 141 nonempty references; Mean rows uses all 154 reviewed instances; Empty FP uses 13 empty references. Rate columns are percentages. The deployed setting is bold.

(a) Discovery with fixed prefixes
$M$	CDR	Prec.	Rec.	F1	Mean rows	Empty FP
0	0.0	0.0	0.0	0.0	0.00	0.0
1	44.7	72.9	71.1	68.6	6.51	61.5
3	73.0	72.1	82.6	73.1	9.05	76.9
5	78.7	72.5	85.6	74.6	10.03	76.9

Appendix

Table 15: Qwen Plus discovery and final QA using the first

M

of five permuted views;

M=0

supplies no ledger. In panel (a), CDR and macro precision, recall, and F1 use 141 nonempty references, Mean rows uses all 154 reviewed instances, and Empty FP uses 13 empty references. Rate columns are percentages. Panel (b) evaluates all 313 instances; QA gains are relative to the preceding row, with paired source-table cluster 95% intervals.

		Position (60 instances)		Dispersion (59 instances)
System	Mode	Endpoint	Trend	Wide $-$ Compact
Qwen Plus	Direct	$+11.7\;[+4.0,+20.2]$	$+2.5\;[+0.7,+4.5]$	$-16.2\;[-22.4,-10.5]$
	Code Agent	$+7.2\;[-0.6,+15.8]$	$+1.3\;[-0.4,+3.1]$	$-2.3\;[-6.6,+1.8]$
MiniMax-M2.5	Direct	$+11.7\;[+4.6,+19.1]$	$+1.9\;[+0.3,+3.7]$	$-9.6\;[-13.0,-6.2]$
	Code Agent	$+5.0\;[-2.6,+13.6]$	$+1.5\;[-0.1,+3.4]$	$-3.8\;[-9.3,+1.3]$
GPT-5 Mini	Direct	$+16.7\;[+8.1,+26.9]$	$+4.1\;[+2.0,+6.3]$	$-7.9\;[-13.0,-2.5]$

Appendix

Table 16: Position and dispersion contrasts in complete discovery rate. Entries give estimates with pointwise 95% confidence intervals in brackets. Endpoint is q5

-

q1; trend is the slope per ordinal position band. Dispersion is Wide

-

Compact. Differences are in percentage points; trends are in percentage points per band.

System	Mean	95% CI	$\mu=25\%$	$\mu=50\%$	$\mu=75\%$
Qwen Plus	+13.9	[+8.4, +20.0]	+19.2	+16.9	+5.6
MiniMax-M2.5	+5.8	[-0.2, +11.7]	+4.5	+13.0	+0.0
GPT-5 Mini	+5.8	[+0.4, +11.5]	+3.4	+11.9	+2.3

Appendix

Table 17: Code Agent vs. Direct differences in the Wide vs. Compact CDR effect, in percentage points.

System	Mode	Raw	Discovery	Action	Discovery vs. Raw	Action vs. Discovery
Qwen Plus	Direct	16.2	18.8	29.2	$+2.6\;[-5.5,10.1]$	$+10.4\;[-1.2,23.3]$
Qwen Plus	Code Agent	46.8	56.5	98.1	$+9.7\;[0.0,19.5]$	$+41.6\;[34.1,49.2]$
MiniMax-M2.5	Direct	35.7	31.8	67.5	$-3.9\;[-16.4,9.0]$	$+35.7\;[22.4,47.4]$
MiniMax-M2.5	Code Agent	25.3	33.8	92.9	$+8.4\;[-0.6,16.8]$	$+59.1\;[49.5,68.9]$
GPT-5 Mini	Direct	18.8	22.7	37.0	$+3.9\;[-3.3,11.5]$	$+14.3\;[5.7,24.1]$
GPT-5 Mini	Code Agent	47.4	58.4	97.4	$+11.0\;[-2.4,24.1]$	$+39.0\;[29.0,49.3]$

Appendix

Table 18: Final-answer accuracy (%) and paired table-state contrasts on the 154 reviewed instances. Differences and source-table cluster 95% intervals are reported in percentage points. Raw uses the original table, Discovery uses the error-marked table, and Action uses the repaired table.

System	$\Delta$ CDR	95% CI
Qwen Plus	$+19.86$	$[+11.11,\,+29.23]$
GLM-4.5-Air	$+5.67$	$[-1.44,\,+13.48]$
MiniMax-M2.5	$+19.86$	$[+9.59,\,+29.73]$

Appendix

Table 19: Random-5 vs. Repeated-5 CDR on 141 reviewed instances. Differences are in percentage points.

System	Direct	Code Agent	GBDI	$\Delta_{\mathrm{Direct}}$	$\Delta_{\mathrm{Code}}$	95% CI
	Baseline		Test condition	Gain (pp)
Qwen Plus	11.8	54.0	64.9	+53.0	+10.9	$[5.3,\,16.5]$
GLM-4.5-Air	4.5	41.2	49.5	+45.0	+8.3	$[2.9,\,14.0]$
MiniMax-M2.5	44.4	32.6	51.1	+6.7	+18.5	$[11.9,\,25.6]$
GPT-5 Mini	27.2	56.9	60.7	+33.5	+3.8	$[-2.5,\,9.8]$
DeepSeek-V4.1-Flash	49.8	59.7	67.7	+17.9	+8.0	$[2.9,\,12.5]$

Appendix

Table 20: End-to-end QA accuracy (%) on all 313 RADAR-T instances. GBDI combines five randomized inspections with intervention guidance before final Code Agent QA.

\Delta_{\mathrm{Direct}}

and

\Delta_{\mathrm{Code}}

denote the gains over each baseline; the paired 95% interval is for

\Delta_{\mathrm{Code}}

. All differences are reported in percentage points.

System	Reference	$\Delta$	95% CI
Qwen Plus	Code Agent	$+10.86$	$[+5.26,\,+16.51]$
	Repeated + Skill	$+4.15$	$[-0.32,\,+9.03]$
MiniMax-M2.5	Code Agent	$+18.53$	$[+11.87,\,+25.62]$
	Repeated + Skill	$+4.79$	$[+0.00,\,+9.59]$
GPT-5 Mini	Code Agent	$+3.83$	$[-2.50,\,+9.80]$
DeepSeek-V4.1-Flash	Code Agent	$+7.99$	$[+2.94,\,+12.54]$

Appendix

Table 21: Paired GBDI QA gains on 313 instances (percentage points).

Explore similar work

When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errors
Jun 30, 2026Yuqing Yang, Qi Zhu, Zhen Han +5LLM Evaluation LLM Grounding
University of Southern California · AWS AI Labs
TabScope: Question-Adaptive Scope Selection for Table Question Answering
Sep 3, 2026Yuxiang Wang, Junhao Gan, Jianzhong QiTable QA
The University of Melbourne
Efficient Table QA via TableGrid Navigation and Progressive Inference Prompting
May 18, 2026Amritansh Maurya, Navjot Singh, Mohammed Javed +1Table QA Structured Prompting
Vision Intelligence Lab, IIIT Allahabad, Prayagraj, India · iMeDIA Lab, IIIT Allahabad, Prayagraj, India · CV:HCI Lab, Karlsruhe Institute of Technology, Karlsruhe, Germany

Jun 30, 2026, cs.CLJ/K move · Enter open · S save

When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errors

Yuqing Yang, Qi Zhu, Zhen Han, Boran Han+4

University of Southern California · AWS AI Labs

LLM Evaluation LLM Grounding LLM Reliability

Open paper AskPDF arXiv

While large language models (LLMs) perform well on table tasks, they still make data referencing errors (DREs), i.e., incorrectly citing or omitting table values, despite understanding the table structure. Beyond final-answer accuracy, DREs directly compromise the correctness and reliability of intermediate reasoning steps. Yet prior studies have only offered limited, small-scale analyses. In this work, we present the first systematic evaluation of tabular data referencing errors across different models and tasks. Our results show that DREs occur across all tested models (1.7B to 20B parameters). Furthermore, we demonstrate that incorporating data referencing as a critic significantly improves answer accuracy up to 12.0%, through critic-based filtering and rejection sampling. Finally, we trained a lightweight 4B-parameter critic model that achieves an average F1 score of 78.2% in detecting both in-distribution and out-of-distribution DREs, and effectively assists inference for larger models.

Abstract

Explore similar work

When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errors

TabScope: Question-Adaptive Scope Selection for Table Question Answering

Efficient Table QA via TableGrid Navigation and Progressive Inference Prompting