Tabular in-context learning (TabICL) enables large language models (LLMs) to predict target records using a handful of labeled demonstrations in their prompts. However, these demonstrations often contain sensitive personal information. Differential privacy (DP) offers principled protection, but noise-based methods introduce substantial noise, distorting the feature-label relationships essential for accurate prediction, especially as data dimensionality grows. In this work, we study DP data synthesis for generating privacy-preserving TabICL demonstrations. We identify two limitations of directly applying general-purpose DP synthesis to entire datasets: modeling a global distribution can obscure local structure and minority patterns in heterogeneous data, while measuring broad attribute dependencies under a limited privacy budget can reduce the accuracy available for relationships most relevant to prediction. More broadly, global statistical fidelity may fail to preserve heterogeneous structures needed by downstream predictive and generative models. To address these challenges, we propose PriTabICL, a general framework compatible with existing DP synthesis algorithms. It partitions records into DP clusters and synthesizes each independently, using a fixed feature-target workload to prioritize prediction-relevant statistical relationships. At inference, each query is routed to a DP cluster, and demonstrations are sampled from its synthetic dataset without additional privacy loss for the original records. Experiments on four real-world datasets demonstrate the effectiveness of our methods across heterogeneous datasets. In particular, one of our instantiations, PriTabICL-PrivSyn, improves over global PrivSyn in 8 out of 12 settings and outperforms existing private TabICL approaches in nearly all evaluated settings. Open-source code is available.
Figures & tables
Fig. 1: A TabICL example and its privacy concern.
Fig. 2: Overview of PriTabICL. Offline, DP clustering groups the private records and one DP synthetic dataset is built per cluster from statistics that relate each feature to the label. Online, each query is routed to its nearest DP center and receives label-balanced demonstrations drawn at random from that cluster’s synthetic dataset.
Notation
Description
D , n , d
Private dataset, its size n=∣D∣ (private), number of non-target features
N
Public size parameter, fixed in advance (not ∣D∣ )
K , μk
Number of clusters, DP center of cluster k
Dk , Dk
Private records assigned to cluster k , its synthetic dataset
WY
Feature–target workload {(Xj,Y)}j=1d
x∗ , k∗ , m
Query, the cluster it is routed to, number of demonstrations
TABLE I: Notation.
Dataset
#Rows
#Attr.
#Num.
#Cat.
Adult
19,313
14
6
8
Airline
77,675
22
4
18
Diabetes
152,208
21
21
0
Phishing
7,075
30
0
30
TABLE II: Datasets used in the evaluation.
Adult
Airline
Diabetes
Phishing
Method
ϵ=1
ϵ=2
ϵ=4
ϵ=1
ϵ=2
ϵ=4
ϵ=1
ϵ=2
ϵ=4
ϵ=1
ϵ=2
ϵ=4
Non-private TabICL (reference)
0.716 ± 0.030
0.716 ± 0.030
0.716 ± 0.030
0.724 ± 0.015
0.724 ± 0.015
0.724 ± 0.015
0.740 ± 0.019
0.740 ± 0.019
0.740 ± 0.019
0.727 ± 0.025
0.727 ± 0.025
0.727 ± 0.025
Always predict positive
0.667
0.667
0.667
0.667
0.667
0.667
0.667
0.667
0.667
0.667
0.667
0.667
LDP-TabICL
0.658 ± 0.016
0.664 ± 0.015
0.670 ± 0.023
0.680 ± 0.025
0.677 ± 0.013
0.690 ± 0.024
0.725 ± 0.033
0.717 ± 0.037
0.725 ± 0.026
0.686 ± 0.010
0.677 ± 0.007
0.673 ± 0.007
GDP-TabICL
0.704 ± 0.055
0.673 ± 0.079
0.654 ± 0.080
0.697 ± 0.020
0.704 ± 0.018
0.709 ± 0.009
0.672 ± 0.022
0.662 ± 0.016
0.669 ± 0.014
0.625 ± 0.027
0.612 ± 0.014
0.610 ± 0.013
Global PrivSyn
0.679 ± 0.036
0.676 ± 0.042
0.657 ± 0.037
0.732 ± 0.036
0.725 ± 0.024
0.722 ± 0.029
0.744 ± 0.013
0.734 ± 0.012
0.741 ± 0.015
0.695 ± 0.041
0.691 ± 0.014
0.721 ± 0.029
TABLE III: F1 (mean ± std over 10 seeds, 100 class-balanced queries each). All methods except LDP-TabICL and GDP-TabICL draw four demonstrations of each class; Global PrivSyn and Global GEM draw them at random, “+ kNN” takes the four nearest synthetic records of each class. Bold: best private method. ↑ / ↓ : PriTabICL significantly better/worse than its own global baseline (paired t -test over seeds, Holm-corrected across the 12 settings, p<0.05 ).
Fig. 3: PriTabICL-PrivSyn stays closest to the best private method. Each dot: F1 points below the best private method in one setting; bar: mean; right: worst case. Lower and narrower means a safer choice.
TABLE IV: Ablation and sensitivity at ϵ=2 (F1, mean ± std over 10 seeds). F–T denotes the feature–target workload. Rows K=2,4,5 and budget 20/80, 40/60 vary PriTabICL-PrivSyn defaults ( K=3 , 30/70). ↑ / ↓ indicate significantly higher/lower F1 than the complete PriTabICL configuration (paired t -test, Holm-corrected, p<0.05 ).
Fig. 4: Demonstration characteristics under cluster-wise versus global PrivSyn at ϵ=2 , using the same feature–target workload. (a) Query-distance reduction and diversity retained. (b) Changes in label separation and F1 (points). ∗ denotes a significant F1 difference.
Dataset
Clusters
Largest share
Rarer class
Synthetic error cluster / global
Adult
1.8
0.81
4,415
0.087 / 0.073
Airline
2.3
0.63
18,004
0.044 / 0.014
Diabetes
3.0
0.44
8,712
0.239 / 0.225
Phishing
3.0
0.73
2,308
0.013 / 0.003
TABLE V: What DP clustering produces at ϵ=2 (PrivSyn, 10 seeds). Clusters: non-empty clusters and the share of records in the largest. Rarer class: records of the cluster’s rarer class behind the median query. Synthetic error: average class-conditional feature-distribution discrepancy between synthetic and real records, for cluster-wise and global synthesis (lower is better).