TAB-AUDIT: Detecting AI-Fabricated Scientific Tables via Multi-View Likelihood Mismatch

Shuo Huang; Yan Pen; Lizhen Qu

cs.CLMar 20, 2026

TAB-AUDIT: Detecting AI-Fabricated Scientific Tables via Multi-View Likelihood Mismatch

Authors: Shuo Huang, Yan Pen, Lizhen Qu

Organizations: Monash University

Abstract

AI-generated fabricated scientific manuscripts raise growing concerns with large-scale breaches of academic integrity. In this work, we present the first systematic study on detecting AI-generated fabricated scientific tables in empirical NLP papers, as information in tables serve as critical evidence for claims. We construct FabTab, the first benchmark dataset of fabricated manuscripts with tables, comprising 1,173 AI-generated papers and 1,215 human-authored ones in empirical NLP. Through a comprehensive analysis, we identify systematic differences between fabricated and real tables and operationalize them into a set of discriminative features within the TAB-AUDIT framework. The key feature, within-table mismatch, captures the perplexity gap between a table's skeleton and its numerical content. Experimental results show that RandomForest built on these features significantly outperform prior state-of-the-art methods, achieving 0.987 AUROC in-domain and 0.883 AUROC out-of-domain. Our findings highlight experimental tables as a critical forensic signal for detecting AI-generated scientific fraud and provide a new benchmark for future research.

Figures & tables

Model	P	R	F1
BERT-base	72.4	70.8	71.6
SpanBERT	76.9	75.8	76.3
Ours	98.7	98.4	98.6

Figure 1: Compact illustration of the table skeleton–numeric mismatch.

Panel of Figure 2 — Figure 2: Representative paper-level distributions for three observer-based indicators: skeleton likelihood (left), digit-level numeric likelihood (center), and the within-table numeric–skeleton mismatch (right). The key pattern is not merely that fabricated tables can have unusual numbers in isolation, but that their numbers are atypical relative to an otherwise conventional scientific skeleton .

External Baselines
		GPT-4o In-Domain Test				GPT-5.2 External Holdout
Detector	Observer	AUROC	AUPRC	TPR@5%	Real FPR@5%	AUROC	AUPRC	TPR@5%	Real FPR@5%
Binoculars (no training)	shared	0.683	0.701	0.385	0.108	–	–	–	–
DetectGPT-style (no training)	GPT-2	0.397	0.430	0.015	0.020	–	–	–	–
Observer Comparison: TAB-AUDIT (PPL4, LogReg)
TAB-AUDIT (PPL4, LogReg)	GPT-2	0.884	0.804	0.374	0.074	0.855	0.756	0.173	0.065

Table 1: main-results table for fabricated-table detection. The left block reports the fixed in-domain test split on human arXiv papers versus GPT-4o-fabricated papers. The right block reports transfer to the GPT-5.2 external holdout. This adjusted version removes the stricter 1% operating-point columns and keeps the 5% operating point together with threshold-free ranking metrics. Thresholds are calibrated on validation humans.

Figure 3: Paper-context sensitivity of table perplexity. Empirical CDF of $\Delta\log\mathrm{PPL}=\log\mathrm{PPL}_{ctx}-\log\mathrm{PPL}_{only}$ , where $\mathrm{PPL}_{ctx}$ scores table tokens conditioned on the preceding paper content (prefix masked from loss), and $\mathrm{PPL}_{only}$ scores the same table tokens in isolation. Negative values indicate that paper context makes the table more predictable under the observer.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Observer	Baseline	AUROC	AUPRC	TPR@5%	Real FPR@5%	F1
gpt2	DigitLaw+Precision(LogReg)	0.950	0.926	0.703	0.054	0.888
gpt2	DigitsOnlyPPL(LogReg)	0.890	0.818	0.256	0.039	0.838
gpt2	FusedAll(PPL4+Struct+DigitLaw, LogReg)	0.986	0.971	0.856	0.030	0.955
gpt2	MismatchOnly(LogReg)	0.637	0.638	0.144	0.044	0.638
gpt2	Structure+NumCellDensity(LogReg)	0.969	0.939	0.882	0.044	0.944
gpt2	StructureOnly(LogReg)	0.963	0.915	0.851	0.049	0.937

Appendix

Table 2: Full in-domain combination matrix from exp105_comprehensive_main_table . All rows are evaluated on the same fixed paper-level test split of human arXiv papers versus GPT-4o-fabricated papers. Thresholds are calibrated on validation humans. This adjusted version reports the 5% operating point rather than the stricter 1% operating point.

Detector	Observer	AUROC	AUPRC	TPR@5%	Real FPR@5%	F1
GPT-5.2 External Holdout
TAB-AUDIT(PPL4, LogReg)	GPT-2	0.855	0.756	0.173	0.065	–
TAB-AUDIT(PPL4, LogReg)	Qwen	0.808	0.689	0.030	0.055	–
TAB-AUDIT(PPL4, LogReg)	Llama	0.881	0.793	0.096	0.040	–
FusedAll(LogReg)	GPT-2	0.825	0.738	0.061	0.039	0.401
FusedAll(RandomForest)	GPT-2	0.883	0.808	0.218	0.054	0.641

Appendix

Table 3: Supplementary external-holdout transfer results. Holdout metrics are currently available for the canonical TAB-AUDIT observer comparison and for the GPT-2-based fused classifier-family comparison. This adjusted version reports the 5% operating point rather than the stricter 1% operating point. Dashes indicate that a directly comparable holdout run was not produced in the current experiment set.

Subset	Human	AI	Total Papers	Total Tables
Clean benchmark	1012	976	1988	5284
Train split	758	732	1490	–
Validation split	51	49	100	–
Test split	203	195	398	–
GPT-5.2 holdout	203	197	400	1324
Prompt-only baseline	1012	20	1032	3274

Appendix

Table 4: Current benchmark and evaluation-set counts used in the paper. Split-level table counts are omitted where they are not finalized in the saved artifacts.

Stage	Human	AI	Total
Indexed source pool	1118	993	2111
Dropped during filtering	106	17	123
Final usable benchmark	1012	976	1988

Appendix

Table 5: Filtering from the indexed source pool to the final clean benchmark.

Metric	Human	GPT4o	Qwen3-8B	Llama3.1-8B	min $\|d\|$	mean $\|d\|$
Row $\Delta$ -consistency	0.497 $\pm$ 0.433	0.235 $\pm$ 0.390	0.187 $\pm$ 0.359	0.206 $\pm$ 0.367	0.64	0.71
Most-common value rate	0.113 $\pm$ 0.138	0.199 $\pm$ 0.136	0.229 $\pm$ 0.153	0.220 $\pm$ 0.153	0.63	0.72
Strict monotone-column rate	0.190 $\pm$ 0.324	0.418 $\pm$ 0.438	0.445 $\pm$ 0.461	0.560 $\pm$ 0.474	0.59	0.71
Mean Spearman trend strength	0.558 $\pm$ 0.262	0.704 $\pm$ 0.244	0.733 $\pm$ 0.256	0.770 $\pm$ 0.267	0.58	0.68
Row % $\Delta$ -consistency	0.357 $\pm$ 0.374	0.169 $\pm$ 0.339	0.153 $\pm$ 0.296	0.150 $\pm$ 0.315	0.53	0.58
Mean column sortedness	0.754 $\pm$ 0.133	0.811 $\pm$ 0.150	0.831 $\pm$ 0.154	0.864 $\pm$ 0.154	0.40	0.57

Appendix

Table 6: Top LM-free statistical signals (mean

\pm

std over tables). Row

\Delta

-consistency checks whether any triple in a row satisfies

c\approx b-a

; Row %

\Delta

-consistency checks

c\approx(b-a)/|a|\times 100

. Most-common value rate measures numeric repetition. Monotonicity/sortedness quantify over-regular trends. min

|d|

/mean

|d|

report absolute Cohen’s

d

vs. human, aggregated across generators.

Figure 4: Literature-grounded AI paper generation pipeline used to construct the fabricated-paper benchmark.

Figure 5: Table Authenticity Judgment Form.

In-domain (GPT-4o test)
Test Set	Train Source	Observer	AUROC	AUPRC	TPR@5%	Real FPR@5%
in-domain	GPT-4o	GPT-2	0.938	0.936	0.723	0.064
in-domain	Llama3.1	GPT-2	0.902	0.911	0.596	0.043
in-domain	Qwen2.5	GPT-2	0.925	0.930	0.638	0.043
in-domain	GPT-4o	Qwen	0.933	0.932	0.894	0.170
in-domain	Llama3.1	Qwen	0.925	0.918	0.830	0.170

Appendix

Table 7: Full train-source/observer robustness matrix for the canonical TAB-AUDIT (PPL4, LogReg) detector. Rows vary the fabricated-paper source used for supervised training and the observer model used for likelihood scoring. Best AUROC row within each test set is boldfaced.

Experiment	Eval. set	Papers	AUROC	AUPRC	Comment
GPT5.2 Holdout set	Clean test split	398	0.8857	0.8093	203 human / 195 AI papers
Prompt-only baseline	Prompt-only test split	796	0.973	0.886	398 Human vs 398 prompt-only pseudo-papers

Appendix

Table 8: Completed paper-level evaluations that do not involve generator-shift transfer.

Statistic	Count
Total examples	353,266
Unique languages	2,077
Train examples	340,251
Eval examples	6,148
Test examples	6,867
No glottocode	13,428

Appendix

Table 9: Human example (score

0.0995

), condensed from a real ACL-style paper. Unlike the synthetic examples, the table reports corpus statistics rather than a clean “ours beats baseline” comparison.

Model	Coherence Score	Token Consumption
Baseline (ACR)	0.72	1.2M tokens
Proposed Framework	0.83	0.9M tokens

Appendix

Table 10: GPT-4o fabricated-paper example (score

0.9992

). The comparison is compact, neat, and strongly favors the proposed method across all shown dimensions.

System	EN $\rightarrow$ EN	EN $\rightarrow$ ZH	ZH $\rightarrow$ EN	ZH $\rightarrow$ ZH	Avg
Base (no retrieval)	54.2	52.8	50.9	51.6	52.4
RAG	67.5	64.1	62.7	63.4	64.4
RAG + LangControl	68.1	66.8	65.9	66.2	66.8
ReaL-RAG	68.4	67.2	66.1	66.5	67.1

Appendix

Table 11: GPT-5.2 fabricated-paper example (score

0.7504

). This holdout example is more elaborate than the prompt-only table: it uses multiple controlled conditions, averages, and a benchmark-style systems comparison.

Model/System	Dataset	Accuracy (%)
XLM-R (fine-tuned)	Amazon Reviews (EN)	92.3
mBERT (fine-tuned)	Amazon Reviews (EN)	90.8
XLM-R (fine-tuned)	Amazon Reviews (DE)	89.7
mBERT (fine-tuned)	Amazon Reviews (DE)	88.5
XLM-R (fine-tuned)	Twitter Sentiment (ES)	85.4
mBERT (fine-tuned)	Twitter Sentiment (ES)	83.6

Appendix

Table 12: Prompt-only generated example (score

0.5231

). The table is locally plausible, but it remains self-contained and lacks the broader paper-level structure seen in the GPT-4o and GPT-5.2 fabricated-paper examples.

AI-Generated Text Detection

AI-generated text detection develops methods to distinguish human-written, machine-generated, and AI-edited text.

See all 170 papers

SciFigDetect: A Benchmark for AI-Generated Scientific Figure Detection
Apr 9, 2026You Hu, Chenzhuo Zhao, Changfa Mo +2AI-Generated Image Detection Digital Image Forensics
Zhejiang University, Hangzhou, China · Independent Researcher, Beijing, China · University of Oulu, Oulu, FINLAND
Multimodal AI Detection for Math-Heavy Scientific Text
Oct 15, 2026Yixuan Huang, Danish Pruthi, Bhuwan DhingraAI-Generated Text Detection Robustness of Machine-Generated Text Detection
Duke University · Indian Institute of Science
Science or Slop?: Benchmarking and Mitigating Scientific Slop in AI-Generated Papers
Sep 30, 2026Yerim Oh, Young-Jun Lee, Jaewoo Ahn +2AI-Generated Text Detection Scientific Reasoning in Language Models
Seoul National University · University of Minnesota

Apr 9, 2026J/K move · Enter open · S save

SciFigDetect: A Benchmark for AI-Generated Scientific Figure Detection

You Hu, Chenzhuo Zhao, Changfa Mo, Haotian Liu+1

Zhejiang University, Hangzhou, China · Independent Researcher, Beijing, China · University of Oulu, Oulu, FINLAND

AI-Generated Image Detection Digital Image Forensics Synthetic Benchmark Generation

Open paper AskPDF arXiv

Modern multimodal generators can now produce scientific figures at near-publishable quality, creating a new challenge for visual forensics and research integrity. Unlike conventional AI-generated natural images, scientific figures are structured, text-dense, and tightly aligned with scholarly semantics, making them a distinct and difficult detection target. However, existing AI-generated image detection benchmarks and methods are almost entirely developed for open-domain imagery, leaving this setting largely unexplored. We present the first benchmark for AI-generated scientific figure detection. To construct it, we develop an agent-based data pipeline that retrieves licensed source papers, performs multimodal understanding of paper text and figures, builds structured prompts, synthesizes candidate figures, and filters them through a review-driven refinement loop. The resulting benchmark covers multiple figure categories, multiple generation sources and aligned real--synthetic pairs. We benchmark representative detectors under zero-shot, cross-generator, and degraded-image settings. Results show that current methods fail dramatically in zero-shot transfer, exhibit strong generator-specific overfitting, and remain fragile under common post-processing corruptions. These findings reveal a substantial gap between existing AIGI detection capabilities and the emerging distribution of high-quality scientific figures. We hope this benchmark can serve as a foundation for future research on robust and generalizable scientific-figure forensics. The dataset is available at https://github.com/Joyce-yoyo/SciFigDetect.

Abstract

AI-Generated Text Detection

Explore similar work

SciFigDetect: A Benchmark for AI-Generated Scientific Figure Detection

Multimodal AI Detection for Math-Heavy Scientific Text

Science or Slop?: Benchmarking and Mitigating Scientific Slop in AI-Generated Papers