TICDA: Tabular In-Context Data Attribution

Yacine Benihaddadene; Milan Bhan; Eliot Dugelay; Mohammed Jawhar; Benjamin Wong; Nicolas Chesneau; Duong Nguyen

cs.LGOct 6, 2026

TICDA: Tabular In-Context Data Attribution

Authors: Yacine Benihaddadene, Milan Bhan, Eliot Dugelay, Mohammed Jawhar, Benjamin Wong, Nicolas Chesneau, Duong Nguyen

Organizations: Ekimetrics · ETH Zurich

Abstract

Tabular foundation models (TFMs) achieve strong predictive performance by conditioning on labeled demonstrations provided in context, without any parameter update. Yet how individual demonstrations shape a given prediction remains poorly understood. This gap matters in practice: the context is often assembled from whatever labeled data is available, potentially leading to the inclusion of mislabeled, redundant, or low-quality examples that degrade performance. Standard data attribution methods do not transfer to the TFM setting: resampling-based approaches such as DemoShapley require a combinatorial number of forward passes, and gradient-based estimators such as influence functions require computing training point's effect on the model parameters, which in-context learning never updates. We introduce TICDA, a method that measures the influence of every demonstration in the context directly from linear surrogates trained on TFM latent embeddings, in a single forward pass and at negligible cost. We show that TICDA offers the best compromise against competitors across four tasks: detecting labeling errors, curating context to preserve predictive accuracy while lowering inference cost, producing attribution scores that transfer across TFMs, and supporting an acquisition strategy for efficient active learning.

Figures & tables

Figure 1: TICDA overview. TICDA (1) extracts TFM embeddings, (2) trains a linear surrogate on the classification task of interest, (3) estimates influence score based on the surrogate and (4) can be applied for label-error detection, context curation, cross-TFM transfer and active learning.

		LOO	DemoShapley	IG	DETAIL	TICDA (ours)
TabDPT	Faith. $\uparrow$	—	$\mathbf{0.91}^{***}$ $\pm 0.07$	$0.20$ $\pm 0.35$	$-0.06$ $\pm 0.18$	$\underline{0.79}$ $\pm 0.09$
TabDPT	AUC $\uparrow$	$\mathbf{0.90}^{***}$ $\pm 0.08$	$\underline{0.88}$ $\pm 0.10$	$0.60$ $\pm 0.20$	$0.47$ $\pm 0.13$	$\underline{0.88}$ $\pm 0.08$
TabICL	Faith. $\uparrow$	—	$\underline{0.65}$ $\pm 0.15$	$0.30$ $\pm 0.22$	$-0.11$ $\pm 0.22$	$\mathbf{0.67}$ $\pm 0.21$
TabICL	AUC $\uparrow$	$\underline{0.80}$ $\pm 0.16$	$0.76$ $\pm 0.11$	$0.64$ $\pm 0.18$	$0.37$ $\pm 0.17$	$\mathbf{0.89}^{***}$ $\pm 0.08$

Table 1: Data attribution for in-context demonstrations on tabular foundation models. Results are aggregated over 38 datasets (mean

\pm

std). Best results are bold , second best underlined ; stars compare the winner with the runner-up (two-sided paired

t

-test,

{}^{*}p<.10

{}^{**}p<.05

{}^{***}p<.01

Figure 2: Average computation time per method.

Configuration		Metrics
Influence	Dimension	Faith. $\uparrow$	AUC $\uparrow$	Time (s) $\downarrow$
query	$20\%$	$0.32_{\scriptscriptstyle\pm 0.16}$	$0.85_{\scriptscriptstyle\pm 0.10}$	$4.90_{\scriptscriptstyle\pm 1.30}$
query	full	$0.33_{\scriptscriptstyle\pm 0.16}$	$0.86_{\scriptscriptstyle\pm 0.10}$	$5.09_{\scriptscriptstyle\pm 1.31}$
self	$20\%$	$0.67_{\scriptscriptstyle\pm 0.21}$	$0.89_{\scriptscriptstyle\pm 0.08}$	$3.51_{\scriptscriptstyle\pm 0.68}$
self	full	$0.67_{\scriptscriptstyle\pm 0.21}$	$0.89_{\scriptscriptstyle\pm 0.08}$	$3.71_{\scriptscriptstyle\pm 0.69}$

Table 2: Ablation study on TabICL (38 datasets,

20\%

label corruption, mean

\pm

std).

Figure 3: Curation: aggregated balanced accuracy evolution on TabDPT and TabICL under different levels of demonstration corruption. Each method ranks in-context demonstrations by its attribution scores, and the lowest-scored demonstrations are removed.

		Demonstrations removed
Target	Source	$0\%$	$10\%$	$20\%$	$50\%$
TabPFN-3	Random	$59.0$ $\pm 14.9$	$58.7$ $\pm 14.6$	$58.3$ $\pm 14.4$	$56.6$ $\pm 14.1$
	TabICLv2	$59.0$ $\pm 14.9$	$\mathbf{63.5}$ $\pm 15.8$	$\mathbf{65.0}$ $\pm 15.8$	$\mathbf{64.4}$ $\pm 15.4$
	TabDPT	$59.0$ $\pm 14.9$	$62.6$ $\pm 15.7$	$64.0$ $\pm 15.7$	$63.4$ $\pm 15.2$
TabPFN-3.5	Random	$60.4$ $\pm 15.5$	$60.1$ $\pm 15.2$	$59.8$ $\pm 15.1$	$57.9$ $\pm 14.3$
	TabICLv2	$60.4$ $\pm 15.5$	$\mathbf{64.1}$ $\pm 15.9$	$\mathbf{64.5}$ $\pm 15.9$	$\mathbf{64.6}$ $\pm 15.5$

Table 3: TICDA TDA Transfer for context curation between source and target models. Balanced accuracy (mean

\pm

std, aggregated over all datasets) is reported at several levels of removal.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Demonstrations in the context	Remaining demonstrations	Queries
512	2.90 $\pm$ 0.60	2.93 $\pm$ 0.62
1,024	1.73 $\pm$ 0.61	1.71 $\pm$ 0.61
2,048	0.89 $\pm$ 0.24	0.88 $\pm$ 0.25
4,096	0.48 $\pm$ 0.14	0.48 $\pm$ 0.15
8,192	0.24 $\pm$ 0.07	0.24 $\pm$ 0.08

Appendix

Table 5: Relative change

\rho_{ij}

of the TabICL representations when one demonstration is removed (Equation 17 ), in percent of the norm of the representation, for contexts of increasing size. For each dataset, we take the median of

\rho_{ij}

over all pairs of a removed demonstration

i

and a row

j

, and we report the mean

\pm

standard deviation of this median over the four datasets. A value of zero would mean that the representations do not change.

Figure 5: Relative change

\rho_{ij}

of the TabICL representations when one demonstration is removed, against the number

n

of demonstrations in the context, on logarithmic axes. The markers show the mean over the four datasets of the per-dataset median, as in Table 5 , and the band one standard deviation for the remaining demonstrations. The change of the queries coincides with that of the remaining demonstrations, and both follow the dotted line, which decreases as

1/n

Panel of Figure 6 — Figure 6: TICDA sensitivity to $\lambda$ regularization.

Explore similar work

Closing the Context Gap: Activation Alignment for Tabular In-Context Learning
Oct 5, 2026Yoel ZeldesIn-Context Learning
Independent Researcher
STRIDE: Training Data Attribution via Sparse Recovery from Subset Perturbations
Jun 3, 2026Rishit Dagli, Abir Harrasse, Luke Zhang +4Training Data
Jinesis AI Lab, University of Toronto & Vector Institute · Max Planck Institute for Intelligent Systems, Tübingen, Germany · Thoughtworks +3
TAFFY: A Task-Adaptive Tabular Foundation Model with In-Context Diversity
Oct 6, 2026Zijian Li, Xiangchen Song, Gongxu Luo +7Tabular Foundation Models In-Context Learning
Carnegie Mellon University · Mohamed bin Zayed University of Artificial Intelligence · Guangdong University of Technology +1

Oct 5, 2026, cs.LGJ/K move · Enter open · S save

Closing the Context Gap: Activation Alignment for Tabular In-Context Learning

Yoel Zeldes

Independent Researcher

In-Context Learning

Open paper AskPDF arXiv

Tabular foundation models perform in-context learning (ICL) by conditioning predictions on labeled training examples provided as context. Unlike traditional models that separate training from inference, these models must process all training examples in every forward pass, making each prediction expensive. Restricting the number of training examples reduces this cost but substantially degrades performance. Instead of discarding context, we propose activation alignment, a method that leverages the full context to teach a model how to behave when seeing only a subset. This is achieved by training a lightweight linear transformation on synthetic unlabeled data to map the intermediate activations of a data-constrained "student" (using partial context) toward those of a full-context "teacher" (using all data). Training the aligner requires no GPU and converges in seconds to minutes on commodity hardware. We evaluate on 38 classification datasets from the TabArena benchmark using the leading two tabular foundation models, TabPFN-3 and TabFM. Across all context budgets, the aligned student yields broad, statistically significant improvements over the unaligned baseline for both models. In low-data regimes, alignment recovers nearly half of the teacher's predictive advantage. The method provides a practical, low-overhead approach to achieving the inference speed of compact contexts while closing a significant fraction of the performance gap to the full-context teacher.

Abstract

Explore similar work

Closing the Context Gap: Activation Alignment for Tabular In-Context Learning

STRIDE: Training Data Attribution via Sparse Recovery from Subset Perturbations

TAFFY: A Task-Adaptive Tabular Foundation Model with In-Context Diversity