Towards Platonic Representation for Table Reasoning: A Foundation for Permutation-Invariant Retrieval
Authors: Willy Carlos Tchuitcheu, Tan Lu, Ann Dooms
Organizations: Department of Mathematics and Data Science, Vrije Universiteit Brussel (VUB), Brussels, Belgium · Data Science Lab, Royal Library of Belgium (KBR), Brussels, Belgium
Historical approaches to Table Representation Learning (TRL) have largely adopted the sequential paradigms of Natural Language Processing (NLP). We argue that this linearization of tables discards their essential geometric and relational structure, creating representations that are brittle to layout permutations. This paper introduces the Platonic Representation Hypothesis (PRH) for tables, positing that a semantically robust latent space for table reasoning must be intrinsically Permutation Invariant (PI). To ground this hypothesis, we first conduct a retrospective analysis of table-reasoning tasks, highlighting the pervasive serialization bias that compromises structural integrity. We then propose a formal framework to diagnose this bias, introducing two principled metrics based on Centered Kernel Alignment (CKA): (i) PI, which measures embedding drift under complete structural derangement, and (ii) rho, a Spearman-based metric that tracks the convergence of latent structures toward a canonical form as structural information is incrementally restored. Our empirical analysis quantifies an expected flaw in modern Large Language Models (LLMs): even minor layout permutations induce significant, disproportionate semantic shifts in their table embeddings. This exposes a fundamental vulnerability in RAG systems, in which table retrieval becomes fragile to layout-dependent noise rather than to semantic content. In response, we present a novel, structure-aware TRL encoder architecture that explicitly enforces the cognitive principle of cell header alignment. This model demonstrates superior geometric stability and moves towards the PI ideal. Our work provides both a foundational critique of linearized table encoders and the theoretical scaffolding for semantically stable, permutation invariant retrieval, charting a new direction for table reasoning in information systems.
Figures & tables
Figure 1. Platonic view of permutation invariance in table embeddings and prospective long-term impact paradigm shift in IR. The left part illustrates the theoretical concept of "Platonic table embedding," investigating whether an original input table and its structurally transposed counterpart (with rows and columns swapped) can be projected by embedding models into a consistent, semantically invariant latent space. The t-SNE visualizations and the question mark between them highlight the core research question regarding the alignment of these embedding spaces across different table representations. The right panel depicts prospective downstream applications that could be built upon such robust embeddings. The top section shows an "Online Table Question Answer" system that uses cell embeddings and nearest-neighbor search to directly answer user queries, while the bottom section outlines a "table-centric RAG application" that leverages table vectors for RAG tasks.
Club
Played
Won
Drawn
Lost
Points for
Points against
Tries for
Tries against
Try bonus
Losing bonus
Points
Bridgend Athletic RFC
22
16
0
6
523
303
68
31
10
4
78
Builth Wells RFC
22
17
0
5
473
305
57
29
7
2
77
Kidwelly RFC
22
14
1
7
532
386
63
45
5
3
66
Loughor RFC
22
13
1
8
532
388
69
43
9
1
64
Ammanford RFC
22
13
0
9
447
394
58
51
6
4
62
Waunarlwydd RFC
22
12
2
8
504
439
57
55
6
3
61
Figure 2. Single-table test example of Rugby Club Performance Statistics ( from WikiSQL dataset, Table ID 560). Identical content values (e.g., “22”) can denote different semantics depending on headers.
Figure 3. LLM-only table embeddings across LLMs ( OpenAI , nomic-embed-text , Gemma2 , DeepSeek-R1 , and llama3 ). Rows correspond to models. Columns show: Left t - SNE projections of LLM-based cell embeddings for the baseline table (Figure 2 ); a point indexed by (row_id,col_id) denotes the projected embedding of a cell. Center pairwise cosine similarity within the resulting cell-embedding set. Right heatmaps H[a,b] of similarity between the original table and its 169 permutations Tab .
Figure 4. Distribution of heatmap entries H[a,b] and the mean xˉ comparing LLM and TRL-derived cell embeddings under two settings: (i) intra-model , which examines embeddings of permuted tables generated by the same model that was trained on the original table. (ii) Cross-model , where embeddings of permuted tables are extracted from different models trained on each other’s samples.
Figure 6. Analysis of the Platonic Representation Hypothesis (PRH) under progressive row and column shuffling. Each curve illustrates the similarity score H[a,b] as the number of fixed rows ( a ) or columns ( b ) increases, ranging from full derangement ( T00 ) to complete identity ( Tnm ). Solid lines represent column permutations ( H[12,b] ), while dashed lines represent row permutations ( H[a,12] ). The three panels compare: (left) baseline LLM embeddings, (center) TRL-refined embeddings (intra-model), and (right) TRL-refined embeddings (cross-model). The legend indicates the model size and embedding dimension. The TRL curves are flatter and more stable, demonstrating stronger PI, whereas the LLM-only curves exhibit a steeper decline, particularly under column permutations.
Transformer-based table retrieval systems flatten structured tables into token sequences, making retrieval sensitive to the choice of serialization even when table semantics remain unchanged. We show that semantically equivalent serializations, such as csv, tsv, html, markdown, and ddl, can produce substantially different embeddings and retrieval results across multiple benchmarks and retriever families. To address this instability, we treat serialization embedding as noisy views of a shared semantic signal and use its centroid as a canonical target representation. We show that centroid averaging suppresses format-specific variation and can recover the semantic content common to different serializations when format-induced shifts differ across tables. Empirically, centroid representations outrank individual formats in aggregate pairwise comparisons across MPNet, BGE-M3, ReasonIR, and SPLADE. We further introduce a lightweight residual bottleneck adapter on top of a frozen encoder that maps single-serialization embeddings towards centroid targets while preserving variance and enforcing covariance regularization. The adapter improves robustness for several dense retrievers, though gains are model-dependent and weaker for sparse lexical retrieval. These results identify serialization sensitivity as a major source of retrieval variance and show the promise of post hoc geometric correction for serialization-invariant table retrieval.
Kushal Raj Bhandari, Adarsh Singh, Jianxi Gao +2
Department of Computer Science Arizona State University Tempe, Arizona, USA · ScaleAI San Francisco, CA, USA
Multi-step LLM reasoning over structured tables fails because planning and execution share no explicit cell-grounding contract. Existing methods constrain the planner to a left-to-right factorization at odds with table permutation invariance, and score intermediate states by generated content alone, overlooking cell grounding. We conduct a pilot study showing that diffusion language models (DLMs) produce more human-aligned and permutation-stable cell attention on tables than autoregressive models, with a 40.2% median reduction in attention-AUROC variability under row reordering. Motivated by this, we propose TABALIGN, a planned table reasoning framework that operationalizes the contract. TABALIGN pairs a masked DLM planner, whose bidirectional denoising emits plan steps as binary cell masks, with TABATTN, a lightweight verifier trained on 1,600 human-verified attention standards to score each step by its attention overlap with the plan-designated mask. Across eight benchmarks covering table question answering and fact verification, TABALIGN improves average accuracy by 15.76 percentage points over the strongest open-source baseline at comparable 8B-class scale, with a matched-backbone ablation attributing 2.87 percentage points of this gain to the DLM planner over an AR planner on a fixed reasoner. Cleaner DLM plans also accelerate downstream reasoning execution by 44.64%.
Tung Sum Thomas Kwok, Zeyong Zhang, Xinyu Wang +6
University of California, Los Angeles · SimpleWay · New Jersey Institute of Technology +4
Table question answering requires models to recover semantic relations encoded implicitly by two-dimensional layout, merged cells, and hierarchical headers. Current pipelines typically use HTML or Markdown as intermediate table representations, but these layout-oriented serializations introduce markup overhead and require large language models to infer header-cell alignments from row and column spans. We propose Semantic Triplet Restoration (STR), a protocol that rewrites each cell as an atomic fact <item path, feature path, value>, where the item path specifies the row-wise entity, the feature path specifies the hierarchical attribute, and the value contains the cell content. We also present TripletQL, a lightweight query-aware router that uses STR to select an appropriate rendering or filtered subset of triplets for each question. Across four Chinese and English table-QA benchmarks, STR matches or improves upon HTML-based baselines while reducing input tokens. The relative benefit grows for smaller language models and longer table contexts, suggesting that explicit semantic representations are especially useful under constrained inference budgets. Code and data are available at https://github.com/Phoenix-ni/STR.git .
Yibin Zhao, Fangxin Shang, Dingrui Yang +1
Taiyuan University of Technology · AI Lab, Qifu Technology, Beijing, China · AI Lab, Greensea Technology, Shenzhen, China