Grounding Long Financial Text with Semantically-Enriched Tables via Multimodal Learning

Xiaoyu Hu

cs.CLOct 27, 2026

Grounding Long Financial Text with Semantically-Enriched Tables via Multimodal Learning

Authors: Xiaoyu Hu

Organizations: Rutgers University

Abstract

We propose a multimodal transformer framework for efficiently modeling long1 financial documents that combine structured tabular data with noisy, unstructured2 textual disclosures. The framework represents financial tables as semantically3 enriched token sequences and integrates them with long-form text through a cross-4 attention mechanism that explicitly captures cross-modal dependencies. In par-5 ticular, we augment each numerical variable with a semantic embedding derived6 from its textual definition, enabling the model to encode both numerical magnitude7 and economic meaning. This design yields a more expressive and parameter-8 efficient representation of tabular data compared to value-only embeddings. Using9 a large-scale dataset of financial statements and MD&A disclosures, we show that10 structured numerical features remain the dominant source of predictive signal;11 however, textual information provides consistent incremental gains when incor-12 porated via cross-attention rather than simple concatenation. Our results further13 demonstrate that noisy textual disclosures are most useful when conditioned on a14 stable, semantically structured tabular representation. In particular, using frozen15 variable-level tabular representations as cross-attention queries enables the model16 to weight MD&A chunks differently for each financial variable and observation.17 These findings suggest that cross-attention provides an efficient mechanism for18 table-conditioned aggregation of long-form narrative evidence.

Figures & tables

Model	AUC	Acc	Macro-F1
Tabular Deep Learning Model
Resnet ( Rubachev et al., 2022 )	$0.6395\pm 0.006$	$0.5784\pm 0.009$	$0.5382\pm 0.010$
FT-Transformer ( Rubachev et al., 2022 )	$0.7025\pm 0.004$	$0.6005\pm 0.006$	$0.5702\pm 0.009$
TabTransformer ( Huang et al., 2020 )	$0.6546\pm 0.006$	$0.6055\pm 0.004$	$0.5520\pm 0.009$
TabNet ( Arik and Pfister, 2021 )	$0.6388\pm 0.006$	$0.6033\pm 0.005$	$0.5473\pm 0.007$
TabSTAR ( Arazi et al., 2025 )	$0.6929\pm 0.005$	$0.6453\pm 0.004$	$0.6024\pm 0.005$

Table 1: Comparison of model performance between benchmark models including tabular deep learning models and tree-based models in multi-modal setting (tabular + text) under earnings prediction task. Results are reported as mean

\pm

the half-width of the 95% standard normal confidence interval over 50 runs.

Model Design	ROC-AUC	Acc	Macro-F1
Learned Tabular Embedding	$0.6984\pm 0.002$	$0.6549\pm 0.001$	$0.6032\pm 0.002$
Learned Text Embedding + attention pooling	$0.5903\pm 0.002$	$0.5748\pm 0.003$	$0.5365\pm 0.003$
Learned Text Embedding + mean pooling	$0.5807\pm 0.003$	$0.5700\pm 0.004$	$0.5263\pm 0.002$
Freeze Text Embedding + Learned Tabular Embedding + Concatenation	$0.6765\pm 0.002$	$0.6483\pm 0.001$	$0.5987\pm 0.002$

Learned Text Embedding + Freeze Learned Tabular Embedding + Concatenation	$0.5892\pm 0.002$	$0.6442\pm 0.004$	$0.5515\pm 0.002$

Table 2: Ablation study for different model components. This table compares different model architecture choices (freeze text encoder vs unfreeze text encoder; concatenation vs cross-attention; freeze tabular transformer vs unfreeze tabular transformer) in earnings prediction performance. Results are reported as mean

\pm

the half-width of the 95% standard normal confidence interval over 50 runs.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 1 — Figure 1: A sample MD&A from Apple Inc. 10K

Figure 2: Structured Table From Apple Inc. 10K

Table 5

Hyperparameter	Search Space
$d\_model$	$\mathcal{U}(192,384,768)$
$Num\_Layers$	$\mathcal{U}(3,6,12)$
$Num\_Attention\_Layers$	$\mathcal{U}(8,16,32)$
$Batch\_Size$	$\mathcal{U}(16,32,64)$
$Dropout\_out$	$\mathcal{U}(0,0.1,0.2)$
$Learning\_Rate$	$\mathcal{U}($ 1e-5 , 2e-5 , 1e-4 $)$

Appendix

Table 5: Tabular Transformer Model Search Space

Hyperparameter	Search Space
learning_rate	$\log\mathcal{U}(e^{-5},1)$
random_strength	$\mathcal{U}(\{1,2,\dots,20\})$
l2_leaf_reg	$\log\mathcal{U}(1,10)$
bagging_temperature	$\mathcal{U}(0.0,1.0)$
leaf_estimation_iterations	$\mathcal{U}(\{1,2,\dots,20\})$
iterations	$\mathcal{U}(\{100,101,\dots,4000\})$

Appendix

Table 6: CatBoost-Tuned Hyperparameters Search Space

Hyperparameter	Search Space
learning_rate	$\log\mathcal{U}(e^{-7},1)$
max_depth	$\mathcal{U}(\{1,2,\dots,10\})$
subsample	$\mathcal{U}(0.2,1)$
colsample_bytree	$\mathcal{U}(0.2,1)$
colsample_bylevel	$\mathcal{U}(0.2,1)$
min_child_weight	$\log\mathcal{U}(e^{-16},e^{5})$

Appendix

Table 7: XGBoost-Tuned Hyperparameters Search Space

Hyperparameter	Search Space
num_leaves	$\mathcal{U}(\{5,6,\dots,50\})$
max_depth	$\mathcal{U}(\{3,4,\dots,20\})$
learning_rate	$\log\mathcal{U}(e^{-3},1)$
n_estimators	$\mathcal{U}(\{50,51,\dots,2000\})$
min_child_weight	$\{10^{-5},10^{-3},10^{-2},10^{-1},1,10,10^{2},10^{3},10^{4}\}$
subsample	$\mathcal{U}(0.2,0.8)$

Appendix

Table 8: LightGBM-Tuned Hyperparameters Search Space

Multimodal Transformer

See all 23 papers

Apr 4, 2026

Document-Level Numerical Reasoning across Single and Multiple Tables in Financial Reports

Despite the strong language understanding abilities of large language models (LLMs), they still struggle with reliable question answering (QA) over long, structured documents, particularly for numerical reasoning. Financial annual reports exemplify this difficulty: financial statement analysis often hinges on accurate arithmetic, and analysts derive key indicators by integrating evidence scattered across multiple tables and narrative text. However, existing benchmarks focus largely on single-table settings, leaving cross-table document-level numerical reasoning underexplored. To address this gap, we introduce FinLongDocQA, a dataset for both single-table and cross-table financial numerical reasoning in long-context reports. Evaluating both closed-source and open-source LLMs on FinLongDocQA reveals two bottlenecks: (1) annual reports often exceed 129k tokens, exacerbating the context rot problem for locating relevant tables; and (2) even when relevant evidence is located, LLMs remain prone to errors in multi-step numerical reasoning. We propose FinLongDocAgent, a Multi-Agent Multi-Round Retrieval-Augmented Generation (RAG) approach that iteratively retrieves evidence, performs intermediate calculations, and verifies results across rounds. Experiments highlight the importance of iterative retrieval and verification for reliable numerical QA in long financial documents.

Yi-Cheng Wang, Wei-An Wang, Chu-Song Chen

Department of Computer Science and Information Engineering, National Taiwan University · FinTech Center, National Taiwan University

Numerical Reasoning in Language Models Financial QA Mathematical Reasoning Retrieval-Augmented Generation

Ask PDF

Aug 11, 2026

Unlocking the Power of Medical Tabular Data via Semantic-Aware Multimodal Pre-training

While vision-language models dominate medical representation learning, unstructured text lacks the dense, quantitative diagnostic phenotypes inherent in structured clinical tables. However, existing multimodal pre-training methods underutilize this potential due to semantic-agnostic designs that treat tabular inputs as flat vectors and employ unstable continuous regression objectives. To overcome this, we propose a novel semantic-aware framework explicitly modeling the intrinsic two-dimensional structure of tabular data. First, addressing the inter-feature hierarchy of varying diagnostic importance, we introduce Importance-Aware Adaptive Masking to construct a label-free curriculum prioritizing salient features. Second, addressing the intra-feature continuity-discreteness duality, we propose a Soft-Label Discretized Module that replaces unstable numerical regression with stable distribution matching, thereby mathematically preserving ordinal relationships. Extensive experiments across large-scale dermatology (SLICE-3D, HOP) and ophthalmology (EyePACS) datasets establish a new state-of-the-art (SOTA), demonstrating exceptional robustness and cross-domain generalizability.

Yingsheng Liu, Haiming Li, Jingmin Zhu +6

Faculty of Information Technology, Monash University, Melbourne, Australia · Monash University, Melbourne, Victoria, Australia · The University of Queensland, Brisbane, Queensland, Australia

Cross-Modal Representation Learning Multimodal Pretraining Tabular ML Tabular Representation Learning

Ask PDF

Apr 21, 2026

TabEmb: Joint Semantic-Structure Embedding for Table Annotation

Table annotation is crucial for making web and enterprise tables usable in downstream NLP applications. Unlike textual data where learning semantically rich token or sentence embeddings often suffice, tables are structured combinations of columns wherein useful representations must jointly capture column's semantics and the inter-column relationships. Existing models learn by linearizing the 2D table into a 1D token sequence and encoding it with pretrained language models (PLMs) such as BERT. However, this leads to limited semantic quality and weaker generalization to unseen or rare values compared to modern LLMs, and degraded structural modeling due to 2D-to-1D flattening and context-length constraints. We propose TabEmb, which directly targets these limitations by decoupling semantic encoding from structural modeling. An LLM first produces semantically rich embeddings for each column, and a graph-based module over columns then injects relationships into the embeddings, yielding joint semantic-tructural representations for table annotation. Experiments show that TabEmb consistently outperforms strong baselines on different table annotation tasks. Source code and datasets are available at https://github.com/hoseinzadeehsan/TabEmb

Ehsan Hoseinzade, Ke Wang, Anandharaju Durai Raju

School of Computing Science Simon Fraser University Burnaby, Canada

Representation Learning Tabular Representation Learning

Ask PDF

Abstract

Multimodal Transformer

Explore similar work

Document-Level Numerical Reasoning across Single and Multiple Tables in Financial Reports

Unlocking the Power of Medical Tabular Data via Semantic-Aware Multimodal Pre-training

TabEmb: Joint Semantic-Structure Embedding for Table Annotation