cs.CLMar 16, 2026
SaveA Closer Look into LLMs for Table Understanding
Organizations: Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China
Abstract
Despite the success of Large Language Models (LLMs) in table understanding, their internal mechanisms remain unclear. In this paper, we conduct an empirical study on 16 LLMs, covering general LLMs, specialist tabular LLMs, and Mixture-of-Experts (MoE) models, to explore how LLMs understand tabular data and perform downstream tasks. Our analysis focus on 4 dimensions including the attention dynamics, the effective layer depth, the expert activation, and the impacts of input designs. Key findings include: (1) LLMs follow a three-phase attention pattern -- early layers scan the table broadly, middle layers localize relevant cells, and late layers amplify their contributions; (2) tabular tasks require deeper layers than math reasoning to reach stable predictions; (3) MoE models activate table-specific experts in middle layers, with early and late layers sharing general-purpose experts; (4) Chain-of-Thought prompting increases table attention, further enhanced by table-tuning. We hope these findings and insights can facilitate interpretability and future research on table-related tasks.
Figures & tables
| Dataset | Task | Structure | Avg Tokens | #Samples |
|---|---|---|---|---|
| WTQ | TQA | Flat | 1012 | 500 |
| HiTab | TQA | Hierarchical | 964 | 500 |
| AITQA | TQA | Hierarchical | 651 | 500 |
| TabFact | TFV | Flat | 681 | 500 |
Table 1: Summary of 2,000 evaluation samples used in our empirical study.
Figure 1: The attention dynamics of Llama3.1-8B-Instruct and Qwen2.5-7B-Instruct on tabular tasks. Each input consists of three segments: system prompt , table content , and user question . Upper : A case study at selected layers—left sub-panels show step-wise attention ratio trend throughout generation steps; right sub-panels show cell-level attention heatmaps. Lower : Aggregated results— Layer-wise Segment Attn Ratio shows the proportion of attention allocated to each segment per layer; Table Attn Entropy measures the degree of focus towards table cells, with lower entropy indicating a more concentrated attention distribution on specific cells); Attn Contribution measures the influence of each segment on the final output. Notably, the entropy minimum at Layer 10 and Layer 14 for Llama3.1-8B-Instruct align with the concentrated attention on the answer cell “China” in the upper heatmap.
Figure 2: Performance drop when masking table attention in different layer ranges across 5 models (averaged over 3 benchmarks 2 formats). Llama-family models show minimal impact from Deep masking, while Qwen-family models degrade uniformly across all ranges.
Figure 3: Prediction stability analysis using LogitLens. Bars show KL divergence between each layer’s decoded distribution and the final output (lower = closer to final prediction). Lines show top-5 token overlap with the final output. Vertical dashed lines mark where predictions stabilize.
| KL Depth | Overlap Depth | ||||
| Model | Layers | Table | Math | Table | Math |
| Instruct LLMs | |||||
| Qwen2.5-3B | 36 | 32 (0.89) † | 27 (0.75) | 33 (0.92) | 31 (0.86) |
| Qwen2.5-7B | 28 | 25 (0.89) † | 23 (0.82) | 25 (0.89) | 25 (0.89) |
| Qwen2.5-14B | 48 | 44 (0.92) † | 43 (0.90) | 44 (0.92) | 43 (0.90) |
| Llama-3.1-8B | 32 | 26 (0.81) | 29 (0.91) | 25 (0.78) | 29 (0.91) |
Table 2: Stabilization depth comparison between table and math (GSM8K) tasks. Values are reported as effective layer / total layers (ratio). KL-based depth is higher for table tasks in 5 of 7 models ( ), while Top- overlap depth is broadly comparable.
Figure 4: Expert activation analysis in MoE models across three architectures (DeepSeek-V2, Qwen3-30B-A3B, OLMoE). Left : Activation heatmaps showing table-specific experts per layer (darker = higher activation probability). Middle : Entropy of expert activation distribution for table tasks (lower entropy indicates concentrated activation on fewer experts). Right : Number of overlapping experts between table and math (GSM8K) tasks per layer.
Figure 5: Comparison between Markdown and HTML table formats.
Figure 6: Comparison between direct answering and Chain-of-Thought (CoT) reasoning.
| Markdown | HTML | |||
| Model | Vanilla | CoT | Vanilla | CoT |
| Instruct LLMs | ||||
| Qwen2.5-7B | 62.15 | 67.20 | 61.35 | 67.80 |
| Llama3.1-8B | 56.95 | 62.35 | 57.95 | 58.35 |
| Table-Specific Fine-tuned LLMs | ||||
| Table-R1-7B | 69.75 | 82.30 | 70.30 | 81.60 |
Table 3: Average performance over 2,000 test samples under different input formats and inference strategies.
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Examples of table serialization formats.
| Model | WTQ | TabFact | HiTab | AITQA |
|---|---|---|---|---|
| Instruct LLMs | ||||
| LLaMA3.2-1B-Instruct | 6.00 | 5.40 | 6.40 | 9.80 |
| LLaMA3.2-3B-Instruct | 33.00 | 58.00 | 30.80 | 52.00 |
| LLaMA3.1-8B-Instruct | 50.20 | 70.80 | 49.00 | 57.80 |
| Qwen2.5-3B-Instruct | 52.40 | 78.00 | 49.80 | 55.60 |
| Qwen3-4B-Instruct-2507 | 52.00 | 75.20 | 49.60 | 60.00 |
Table 4: LLMs’ performance (accuracy %) on evaluation data of four benchmarks. Best results are bolded.
| Markdown | HTML | |||
| Model | Direct | CoT | Direct | CoT |
| Instruct LLMs | ||||
| Qwen2.5-7B | 62.2 2.1 | 67.2 2.0 † | 61.4 2.1 | 67.8 2.0 † |
| Llama-3.1-8B | 57.0 2.2 | 62.4 2.1 † | 58.0 2.1 | 58.4 2.1 † |
| Table-Specific Fine-tuned LLMs | ||||
| Table-R1-7B | 69.8 2.0 | 82.3 1.7 † | 70.3 1.9 | 81.6 1.7 † |
Table 5: Performance (%) with 95% bootstrap confidence intervals under different input formats and inference strategies. marks statistically significant differences between CoT and Direct (non-overlapping CIs).
| Model | Total Layers | Eff. Layers | Eff. Ratio |
| Instruct LLMs | |||
| Llama-3.2-1B-Instruct | 16 | 14 | 0.88 |
| Llama-3.2-3B-Instruct | 28 | 23 | 0.82 |
| Meta-Llama-3.1-8B-Instruct | 32 | 25 | 0.78 |
| Qwen2.5-3B-Instruct | 36 | 33 | 0.92 |
| Qwen2.5-7B-Instruct | 28 | 25 | 0.89 |
Table 6: Layer effectiveness analysis on table-related tasks using LogitLens. Eff. Layers identifies the first layer where the predicted vocabulary distribution aligns with the final output (defined by the top-5 token overlap first exceeds 0.3). Eff. Ratio (Eff. Layers / Total Layers) reflects the stage at which the model stabilizes its prediction; a lower ratio indicates that the model reaches its final decision in its shallower layers.
Figure 8: The qualitative case example for Q1.
Figure 9: Top-32 retrieval heads’ overlap with attention norm heatmaps across LLMs. Background color indicates attention norm intensity (yellow = higher); red boxes mark the Top-32 retrieval heads identified by prior work. The substantial overlap, especially in middle-to-late layers, suggests that table understanding reuses pre-existing retrieval circuits.
Figure 10: Attention dynamics of Qwen3-30B-A3B-Instruct-2507.
Figure 11: Attention dynamics of TAMA.
Figure 12: Attention dynamics of Table-R1-Zero-8B.
Figure 13: Attention dynamics of Table-R1-Zero-7B.
Figure 14: Attention dynamics of TAMA-Qwen2.5.
Figure 15: Attention dynamics of Qwen3-4B-Instruct.
Figure 16: Attention dynamics of Qwen2.5-3B-Instruct.
Figure 17: Attention dynamics of Qwen2.5-14B-Instruct.
Figure 18: Attention dynamics of Qwen2.5-32B-Instruct.
Figure 19: Attention dynamics of TableGPT2-7B.
Figure 20: Attention dynamics of LLaMA3.2-3B-Instruct.
Figure 21: Attention dynamics of LLaMA3.2-1B-Instruct.
Figure 22: Attention dynamics of DeepSeek-V2-Lite-Chat.
Figure 23: Attention dynamics of OLMoE-1B-7B-Instruct.
Figure 24: Attention dynamics of Qwen’s 1B gated attention baseline.
Figure 25: Attention dynamics of Qwen’s 1B model with element-wise gated attention.
Figure 26: Attention dynamics of Qwen’s 1B model with Headwise Gated Attention.
Figure 27: Analysis results of LLMs’ effectiveness layer.
More in this topic
LLM Interpretability
Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.
See all 758 papers