TABQAWORLD: Optimizing Multimodal Reasoning for Multi-Turn Table Question Answering
Authors: Tung Sum Thomas Kwok, Xinyu Wang, Xiaofeng Lin, Peng Lu, Chunhe Wang, Changlun Li, Hanwei Wu, Nan Tang, Elisa Kreiss, Guang Cheng
Organizations: University of California, Los Angeles · SimpleWay · McGill University · Université de Montréal · The Hong Kong University of Science and Technology (Guangzhou)
Multimodal reasoning has emerged as a powerful framework for enhancing reasoning capabilities of reasoning models. While multi-turn table reasoning methods have improved reasoning accuracy through tool use and reward modeling, they rely on fixed text serialization for table state readouts. This introduces representation errors in table encoding that significantly accumulate over multiple turns. Such accumulation is alleviated by tabular grounding methods in the expense of inference compute and cost, rendering real world deployment impractical. To address this, we introduce TABQAWORLD, a table reasoning framework that jointly optimizes tabular action through representation and estimation. For representation, TABQAWORLD employs an action-conditioned multimodal selection policy, which dynamically switches between visual and textual representations to maximize table state readout reliability. For estimation, TABQAWORLD optimizes stepwise reasoning trajectory through table metadata including dimension, data types and key values, safely planning trajectory and compressing low-complexity actions to reduce conversation turns and latency. Designed as a training-free framework, empirical evaluations show that TABQAWORLD achieves state-of-the-art performance with 4.87% accuracy improvements over baselines, with 5.42% accuracy gain and 33.35% inference latency reduction over static settings, establishing a new standard for reliable and efficient table reasoning.
Figures & tables
Figure 1: Motivation and overview of TabQAWorld . Fixed text serialization introduces state tracking noise (representation bottleneck), which propagates across multi-step reasoning and causes trajectory drift (estimation bottleneck). TabQAWorld addresses such failure process by jointly optimizing what to see and what to expect .
WTQ
MMQA
MMTU
Accuracy ± SD under different perceived modality
json
0.722 (0.032)
0.594 (0.035)
0.303 (0.033)
latex
0.707 (0.032)
0.574 (0.035)
0.333 (0.034)
Markdown
0.712 (0.032)
0.584 (0.035)
0.364 (0.034)
Image
0.818 (0.027)
0.614 (0.035)
0.374 (0.034)
Multimodal
0.788 (0.029)
0.635 (0.034)
0.364 (0.034)
Table 1: Reasoning performance and costs.
Figure 2: An illustrative example of how image-based parsing facilitates more human-preferred column attention than text-serialized tables. The value below each table indicates the mean-squared error (MSE) against human-preferred binary attention.
Figure 3: Hallucinations in full table estimation from frontier GPT-5.4 motivate lower-dimensional state estimation.
Figure 4: TabQAWorld dynamically selects the optimal data modality based on task purposes, and optimizes reasoning trajectory based on low dimensional metadata to minimize token usage and latency while maintaining a rigorous feedback loop to ensure convergence on an accurate final answer.
Figure 5: Illustration of metadata-guided execution. A mismatch in key_output (1964 vs. 1968) triggers replanning.
TQA
TFV
Summary
Method
WTQ
TABMWP
TAT-QA
HiTab
FeTaQA
TabFact
InfoTabs
WTQ + TabFact
Average
(Acc.)
(Acc.)
(Acc.)
(Acc.)
(BLEU)
(Acc.)
(Acc.)
(Acc.)
(Acc.)
Table-as-Text Baselines
Llama-2-7B
16.39
22.82
13.73
10.72
10.93
9.20
38.92
12.78
18.63
Llama3-Instruct-8B
21.24
42.01
13.08
6.97
12.66
73.89
54.00
47.57
35.20
TableLlama-7B
24.97
10.10
19.04
46.57
38.38
79.37
46.57
52.17
37.77
Table 2: Benchmark results for TabQAWorld and baseline models.
Figure 6: Ablation studies of TabQAWorld . (a): TabQAWorld brings model-agnostic joint improvement in accuracy and latency. (b): Visual-based modalities consistently outperform across question types. (c, d): TabQAWorld agent has internalized respective policy in modality selection and action compression preference. (e): Inference latency after trajectory optimization drops while retaining comparable compute.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Task Type
Dataset
Train
Validation
Test
Table Question Answering (TQA)
TABMWP
500
200
1,000
WTQ
500
200
1,000
HiTab
500
200
1,000
TAT-QA
—
—
772
FeTaQA
500
200
1,000
Table Fact Verification (TFV)
TabFact
500
200
1,000
Appendix
Table 3: Dataset statistics by task type.
Parameter
Value / Default
Description
Image Rendering Hyperparameters
Max rows displayed
30
Truncate overly wide tables
Max cols displayed
10
Truncate overly long tables
Font size
8pt
Standardize patch size for each word
Row scale
1.2 ×
Standardize patch size for each row
DPI
200
Standardize resolution
Appendix
Table 4: Summary of image rendering, model generation, and experiment hyperparameters.
Figure 7: Overall performance of all models in the three datasets.
Accuracy
Modality
json
latex
Markdown
Image
Multimodal
Adaptive
json
latex
Markdown
Image
Image
Multimodal
Adaptive
Optimization
✗
✗
✗
✗
✗
✗
✓
✓
✓
✓(Metadata)
✓(Table)
✓
✓
WTQ
Qwen2.5-VL-7B
0.4343
0.4293
0.5101
0.5450
0.5859
0.6364
0.4293
0.4343
0.4646
0.5606
0.4814
0.6061
0.7020
Qwen3-VL-8B
0.7222
0.7071
0.7121
0.8182
0.7929
0.8333
0.7071
0.6869
0.7323
0.7980
0.7525
0.8131
0.8889
GPT-5-nano
0.8434
0.8535
0.8586
0.8636
0.8535
0.9040
0.8535
0.8131
0.8636
0.8586
0.8131
0.8687
0.9293
Appendix
Table 5: Detailed accuracy of Figure 6 (a) and Figure 7
Standard Deviation
Modality
json
latex
Markdown
Image
Multimodal
Adaptive
json
latex
Markdown
Image
Image
Multimodal
Adaptive
Optimization
✗
✗
✗
✗
✗
✗
✓
✓
✓
✓(Metadata)
✓(Table)
✓
✓
WTQ
Qwen2.5-VL-7B
0.0352
0.0352
0.0355
0.0354
0.0350
0.0342
0.0352
0.0352
0.0354
0.0353
0.0355
0.0347
0.0325
Qwen3-VL-8B
0.0318
0.0323
0.0322
0.0274
0.0288
0.0265
0.0323
0.0330
0.0315
0.0285
0.0307
0.0277
0.0223
GPT-5-nano
0.0258
0.0251
0.0248
0.0244
0.0251
0.0209
0.0251
0.0277
0.0244
0.0248
0.0277
0.0240
0.0182
Appendix
Table 6: Detailed standard deviation for Figure 6 (a) and Figure 7
Inference Latency (in seconds)
Modality
json
latex
Markdown
Image
Multimodal
Adaptive
json
latex
Markdown
Image
Image
Multimodal
Adaptive
Optimization
✗
✗
✗
✗
✗
✗
✓
✓
✓
✓(Metadata)
✓(Table)
✓
✓
WTQ
Qwen2.5-VL-7B
46.47
20.05
27.62
33.35
12.73
10.28
10.50
8.58
4.59
7.95
12.68
3.50
6.75
Qwen3-VL-8B
75.96
56.56
78.85
36.22
52.48
34.44
69.67
35.59
59.70
54.28
40.89
35.00
30.83
GPT-5-nano
31.86
37.71
28.48
26.07
22.73
29.80
18.99
18.17
14.31
13.27
27.34
15.61
26.53
Appendix
Table 7: Detailed time latency of Figure 6 (a) and Figure 7
ROUGE / BLEU
Modality
json
latex
Markdown
Image
Multimodal
Adaptive
json
latex
Markdown
Image
Image
Multimodal
Adaptive
Optimization
✗
✗
✗
✗
✗
✗
✓
✓
✓
✓(Metadata)
✓(Table)
✓
✓
WTQ
Qwen2.5-VL-7B
0.608 / 0.029
0.618 / 0.039
0.580 / 0.035
0.599 / 0.037
0.640 / 0.024
0.582 / 0.026
0.546 / 0.009
0.496 / 0.024
0.512 / 0.016
0.573 / 0.024
0.467 / 0.000
0.565 / 0.029
0.642 / 0.038
Qwen3-VL-8B
0.709 / 0.038
0.713 / 0.030
0.715 / 0.028
0.769 / 0.034
0.768 / 0.034
0.810 / 0.040
0.717 / 0.036
0.722 / 0.031
0.753 / 0.034
0.774 / 0.026
0.780 / 0.031
0.775 / 0.035
0.852 / 0.035
GPT-5-nano
0.745 / 0.045
0.742 / 0.033
0.758 / 0.037
0.764 / 0.031
0.746 / 0.025
0.785 / 0.040
0.772 / 0.035
0.732 / 0.040
0.738 / 0.037
0.734 / 0.038
0.701 / 0.031
0.744 / 0.040
0.793 / 0.039
Appendix
Table 8: ROUGE ( Lin, 2004 ) and BLEU ( Papineni et al., 2002 ) score of Main Result Figure 6 (a)
Figure 8: We annotate human-preferred attention standard, and visual-included representations not only lead to higher accuracy in different question types, but also lower MSE to human-preferred attention onto the table. This suggests that the relationship between more precise grounding over relevant table regions and quality of table reasoning.
Question
Ground Truth
Required Actions
How many plants are in Algeria?
6
sum_over_column
What is the average capacity of Algeria plants?
4.933
average_over_column
What is the name and capacity of the plant in Brunei?
Lumut 1 with 7.2 mmtpa
lookup_row
What is the average capacity of Malaysia plants?
6.267
lookup_row , average_over_column
What is the latest plant built in Qatar?
2010
lookup_row , max_over_column
What is the average capacity of all plants?
8.508
average_over_column
Appendix
Table 9: Designed questions and required reasoning actions.
Table 10: Canonical serialization templates for the same 3-column row.
Overall Metrics (Correct Q / Wrong Q)
Bin (Tokens)
JSON
LaTeX
Markdown
Accuracy (%)
72.2
70.7
71.2
Average Structural Token
231
147
121
Total Token
469
416
417
Format overhead ratio (%)
51.8 (51/55)
37.0 (38/34)
46.9 (48/45)
Header-data attention ratio
1.29 (1.3/1.2)
1.95 (2.0/1.7)
2.10 (2.3/1.6)
Appendix
Table 11: LLM Accuracy (%) by Token-Size Bin: Aggregated and Per-Dataset Breakdown
Figure 9: Heatmap of question-type performance and attention statistics across JSON , LaTeX and Markdown table serializations. Results show that while JSON devotes more attention to tables to improve lookup tasks, its heavier format overhead leads to underperformance in ordered computation tasks across multiple rows.
Overall Accuracy (%) (WTQ/MMQA/MMTU)
Bin (Tokens)
JSON
LaTeX
Markdown
Image
Multimodal
Adaptive
41–263
59.5% (80/60/21)
60.2% (82/59/21)
58.3% (81/59/20)
62.8% (85/63/30)
63.1% (84/63/30)
70.1% (91/68/41)
264–375
74.5% (83/65/75)
75.0% (85/67/79)
76.9% (85/70/75)
77.7% (86/63/82)
80.0% (85/72/82)
83.5% (88/74/86)
376–510
73.5% (88/67/74)
74.8% (84/66/74)
76.1% (89/68/86)
74.3% (88/69/81)
74.8% (88/67/86)
78.5% (92/73/83)
513–816
64.9% (69/62/30)
66.8% (70/65/33)
69.7% (73/69/34)
69.0% (73/64/34)
70.7% (74/65/38)
75.5% (79/69/43)
817–1695
67.3% (74/67/27)
66.0% (73/66/31)
68.9% (75/66/32)
71.4% (80/68/34)
72.9% (78/69/38)
77.8% (82/77/42)
Appendix
Table 12: LLM Accuracy (%) by Token-Size Bin: Aggregated and Per-Dataset Breakdown
School of Astronautics, Beihang University, Beijing 102206, China · Longcat Interaction Team, Meituan, Beijing 100102, China · Tianmushan Laboratory, Beihang University, Hangzhou 311115, China