HARP-LoRA: Hierarchy-Conditioned Rank Projection for Low-Rank Adaptation in Scientific Table Understanding

Hossein Sholehrasa; Mohammadhossein Amini; Doina Caragea; Remya Ampadi Ramachandran; Dan Wu; Zihang Shi; Zelin Li; Lindsey Hulbert

cs.CLOct 25, 2026

HARP-LoRA: Hierarchy-Conditioned Rank Projection for Low-Rank Adaptation in Scientific Table Understanding

Authors: Hossein Sholehrasa, Mohammadhossein Amini, Doina Caragea, Remya Ampadi Ramachandran, Dan Wu, Zihang Shi, Zelin Li, Lindsey Hulbert,

Organizations: 1DATA Consortium and FARAD Program, Kansas State University, Olathe, KS, USA · Department of Computer Science, Kansas State University, Manhattan, KS, USA · Olin Business School, Washington University in St. Louis, St. Louis, Missouri, USA · Department of Mathematics, Kansas State University, Manhattan, KS, USA ·

Abstract

Tabular data is widely used in scientific applications, where complex layouts, including multi-level headers, merged cells, captions, and footnotes, make information extraction difficult. This is particularly important in pharmacokinetics (PK), where values must be correctly associated with the drug, species, dose, administration route, unit, and experimental condition. Parameter-efficient fine-tuning methods such as Low-Rank Adaptation (LoRA) reduce adaptation cost but do not explicitly incorporate relationships between table values and surrounding headers. We introduce HARP-LoRA, a hierarchy-conditioned rank projection method for low-rank adaptation in scientific table understanding. HARP-LoRA extends projected LoRA by constructing token-specific intrinsic representations from the row- and column-header hierarchies of each data token. These representations condition the low-rank down-projection, incorporating hierarchical context while keeping the pretrained model and random projection tensors frozen. We evaluate HARP-LoRA on 950 manually annotated PK tables from 587 scientific articles and on WikiTableQuestions (WikiTQ) using Llama~~3.2 3B, Llama~~3.1 8B, and Gemma~~3 27B. With Llama~~3.1 8B, HARP-LoRA achieves a PK F1-score of 57.64% and WikiTQ accuracy of 53.15%, improving over LoRA by 1.42 and 6.56 points, respectively. With Gemma~3 27B, it achieves the best PK F1-score of 70.63% and WikiTQ accuracy of 56.87%. These results use only 172K, 197K, and 547K trainable parameters, far fewer than LoRA and TableLoRA. Overall, conditioning lightweight adapters on table-header hierarchies improves extraction from scientific tables and reasoning over flatter general-domain tables while maintaining parameter efficiency. Code is available at: https://github.com/hosseinsholehrasa/HARP-LoRA

Figures & tables

Fig. 1: Overview of HARP-LoRA. For each table data token, the corresponding hierarchical row and column headers are identified and encoded separately. Their representations are combined using separate trainable vectors to produce a token-specific intrinsic vector, which conditions the projected LoRA down-projection. The resulting low-rank representation is mapped back to the model hidden space using a shared up-projection.

Model	Method	PK Dataset			WikiTQ	# Train. Params.
Model	Method	P (%)	R (%)	F1 (%)	Acc (%)	# Train. Params.
Llama3.2 3B	LoRA	53.57	58.76	53.57	44.17	1.84M (0.057%)
	TableLoRA 2D	53.13	57.90	52.87	43.80	2.59M (0.080%)
	projected LoRA	52.26	57.34	52.20	47.32	115K (0.004%)
	HARP-LoRA	53.26	58.09	53.08	45.23	172K (0.005%)
Llama3.1 8B	LoRA	57.12	59.98	56.22	46.59	2.62M (0.033%)

TABLE I: Performance comparison on the PK and WikiTQ datasets across three backbone models. HARP-LoRA achieves the strongest results for the 8B and 27B backbones while using substantially fewer trainable parameters than LoRA and TableLoRA 2D. Best results within each backbone are shown in bold , and second-best results are underlined . P, R, F1, and Acc denote precision, recall, F1-score, and answer accuracy, respectively.

Intrinsic Dimension $k$	F1 (%)
256	30.68
512	55.23
1024	57.64
2048	56.79

TABLE II: Effect of the intrinsic dimension

k

on the PK data F1-score for HARP-LoRA with Llama 3.1 8B. Performance improves substantially as

k

increases from 256 to 1024, with the best F1-score achieved at k=1024.

Configuration	F1 Score (%)
Full model	57.64
Without row-header hierarchy	56.91
Without column-header hierarchy	56.36
Without level-preserving pooling	56.58
Without RMS normalization	56.97
Shared intrinsic vector (projected LoRA)	56.78

TABLE III: Ablation results on the PK dataset using Llama 3.1 8B. Removing any component reduces the F1-score, with the largest degradation observed when the column-header hierarchy is excluded. The full HARP-LoRA model achieves the best performance.

Low-Rank Adaptation

Low-rank adaptation (LoRA) fine-tunes pretrained models with compact weight updates, reducing trainable parameters and deployment costs.

See all 542 papers

DeepTable: Structural Attention Biases and Tree Path Encoding for Hierarchical Table Understanding
Sep 7, 2026Jyun-Ying Yen, Cheng-Kuan Lin, Yu-Chee TsengTable QA Table Structure Recognition
Department of Computer Science National Yang Ming Chiao Tung University, Taiwan
H2Table: Hierarchical Hypergraph-Enhanced Large Language Models for Complex Table Reasoning
Sep 1, 2026Jia Ling, Yangfan Wang, Chen Tang +4Table QA Hierarchical Representation Learning
Harbin Institute of Technology · AI Research Center, Midea Group (Shanghai) Co., Ltd. · Changchun University of Science and Technology +1
MatryoshkaLoRA: Learning Accurate Hierarchical Low-Rank Representations for LLM Fine-Tuning
May 8, 2026Ionut-Vlad Modoranu, Mher Safaryan, Dan AlistarhAdapter Tuning LLM Fine-Tuning
Institute of Science and Technology Austria (ISTA). · Lancaster University, UK · ISTA

Sep 7, 2026J/K move · Enter open · S save

DeepTable: Structural Attention Biases and Tree Path Encoding for Hierarchical Table Understanding

Jyun-Ying Yen, Cheng-Kuan Lin, Yu-Chee Tseng

Department of Computer Science National Yang Ming Chiao Tung University, Taiwan

Table QA Table Structure Recognition Attention Mechanisms

Open paper AskPDF arXiv

Large language models (LLMs) have demonstrated strong performance in table understanding. However, they typically process table content and headers as linearized token sequences. This representation weakens the two-dimensional and hierarchical structural relationships encoded by multi-level row and column headers. Existing parameter-efficient fine-tuning methods incorporate basic row and column information but do not explicitly capture the rich structural dependencies induced by hierarchical table headers. We propose DeepTable, a structure-aware approach for table understanding with LLMs. DeepTable comprises two complementary components. Structural Attention Bias (SAB) introduces learnable biases into the attention logits to explicitly represent whether pairs of table tokens share the same row or column. Tree Path Encoding (TPE) represents each table token using the ancestor paths of its row and column headers, preserving its position within the multi-level table structure. We integrate DeepTable with TableLoRA (He et al., 2025) to inject structural information into parameter-efficient adaptation. Across three LLM backbones, DeepTable consistently improves the corresponding TableLoRA baselines on three table question answering benchmarks, achieving average gains of 7.42 points on HiTab, 3.23 points on WikiTQ, and 2.01 BLEU points on FeTaQA. These results demonstrate the effectiveness of the proposed structural biases across different LLM backbones.

Abstract

Low-Rank Adaptation

Explore similar work

DeepTable: Structural Attention Biases and Tree Path Encoding for Hierarchical Table Understanding

H2Table: Hierarchical Hypergraph-Enhanced Large Language Models for Complex Table Reasoning

MatryoshkaLoRA: Learning Accurate Hierarchical Low-Rank Representations for LLM Fine-Tuning