VietAIDetector: An Open-Source Zero-Shot Detector for Vietnamese AI-Generated Text

Trieu Hai Nguyen; Van-Dung Hoang

cs.CLAug 26, 2026

VietAIDetector: An Open-Source Zero-Shot Detector for Vietnamese AI-Generated Text

Authors: Trieu Hai Nguyen, Van-Dung Hoang

Organizations: Nha Trang University, 02 Nguyen Dinh Chieu Street, North Nha Trang 57134, Vietnam · Ho Chi Minh City University of Technology and Engineering, 01 Vo Van Ngan Street, Thu Duc, Ho Chi Minh City, Vietnam

Abstract

Distinguishing AI-generated content from human-written text in Vietnamese remains difficult due to the limited availability of dedicated open-source tools. In this paper, we present VietAIDetector, an open-source software tool designed for practical Vietnamese AI-generated text detection. Operating on a zero-shot scoring engine derived from VietBinoculars~\cite{nguyen2025vietbinoculars}, the software requires no domain-specific retraining and accepts raw text, DOCX files, native PDF documents, and scanned PDFs via integrated optical character recognition. To handle long documents beyond conventional input lengths, the tool combines sliding-window chunking with document-level score aggregation. VietAIDetector offers an interactive Gradio web interface with configurable operating thresholds, including Youden's J statistic, the Closest Point approach, and a low false-positive rate criterion, with results exportable as downloadable PDF reports. Empirical validations confirm that this chunking pipeline mitigates threshold drift on multi-thousand-token documents while maintaining practical runtime throughput under GPU deployment. The tool is publicly available at https://github.com/trieuntu/VietAIDetector.

Figures & tables

Method/ Tool	Open Source	Zero- Shot	GUI	Optimized for Vietnamese	Handles Long Texts	Adversarial Robustness	Input Formats
VietAIDetector	✓	✓	✓	✓	✓	✓ a	✓
Binoculars [ 4 ]	✓	✓	✓	✗	✗	✓	✗
GLTR [ 3 ]	✓	✓	✓	✗	✗	✗	✗
Rank/LogRank [ 3 , 13 ]	✓	✓	✗	✗	✗	✗	✗
RADAR [ 5 ]	✓	✗	✓	✗	✗	✗	✗
DetectGPT [ 7 ]	✓	✓	✗	✗	✗	✗	✗

Table 1: Feature comparison between VietAIDetector and existing tools and methods.

Figure 1: The overall architecture of the VietAIDetector tool.

Figure 2: The VietAIDetector interface for AI-generated text detection using raw text input.

Domain	Dataset file	Field names
Domain	Dataset file	Human	AI
News articles	test	text	google-gemma-3-12b-it-generated_text
Literary works	Gemma-3-12B-VuTrongPhung	text	google-gemma-3-12b-it-hf_generated_text_wo_prompt
Literary works	Sailor2-8B-VuTrongPhung	text	sail-Sailor2-8B-Chat-hf_generated_text_wo_prompt

Table 2: Field names used to extract human-written and AI-generated text from the evaluation datasets.

Figure 3: The VietAIDetector interface showing the detection results for the input text.

Figure 4: The VietAIDetector interface for file upload and detection parameter configuration.

Figure 5: Exploratory comparison on Claude-Sonnet-4.6-News, Gemini-3.7-Flash-News, and GPT-5.6-Luna-News, with 20 documents per dataset. VietAIDetector configurations were selected using these same datasets, so the results do not provide an independent estimate of generalization performance. (a) Average AI score. (b) AI recall under majority voting with configuration

(W=500,O=50)

and

(W=600,O=150)

Dataset	Role	#Docs	Human tokens/doc		AI tokens/doc
Dataset	Role	#Docs	Mean $\pm$ SD	Median [Q1, Q3]	Mean $\pm$ SD	Median [Q1, Q3]
Gemma-3-12B-News-Concat-Dev	Development	592	2,386.4 $\pm$ 240.2	2,369.0 [2,191.8, 2,507.2]	2,712.4 $\pm$ 411.1	2,666.0 [2,396.8, 2,996.2]
Gemma-3-12B-News-Concat-Test	Test	2,358	2,401.4 $\pm$ 255.2	2,362.0 [2,206.5, 2,524.0]	2,703.3 $\pm$ 422.3	2,646.0 [2,382.0, 2,957.5]

Table 3: Summary and token-length distributions of Gemma-3-12B-News-Concat-Dev and Gemma-3-12B-News-Concat-Test. The number of documents (#Docs) is evenly divided between Human and AI classes. PhoGPT BPE token counts are measured after preprocessing, without truncation or special tokens. SD denotes sample standard deviation, and brackets report the first and third quartiles [Q1, Q3].

Condition	Aggregation	AUROC	Macro-F1	FP	FN
Condition	Aggregation	[95% CI]	[95% CI]	FP	FN
Full document	Majority	0.7816 [0.7668, 0.7956]	0.7707 [0.7533, 0.7867]	515	0
Full document	Mean	1.0000 [1.0000, 1.0000]	0.7707 [0.7533, 0.7867]	515	0
Truncated at 768	Majority	0.9572 [0.9491, 0.9652]	0.9571 [0.9490, 0.9652]	100	1
Truncated at 768	Mean	0.9998 [0.9995, 1.0000]	0.9571 [0.9490, 0.9652]	100	1
$(600,0)$	Majority	1.0000 [1.0000, 1.0000]	0.9992 [0.9979, 1.0000]	2	0

Table 4: Document-level performance on Gemma-3-12B-News-Concat-Test with 1,179 Human and 1,179 AI documents at the fixed Youden threshold (

t^{*}\approx 0.928

). Conditions include unsegmented full-document scoring, single-chunk truncation at 768 tokens, and sliding-window chunking

(W,O)

with window size

W

and overlap

O

in tokens. Brackets show 95% source-pair bootstrap confidence intervals. Majority AUROC uses vote fractions

r_{\mathrm{vote}}

and

r_{\mathrm{w\text{-}vote}}

, and mean AUROC uses negative mean scores

r_{\mathrm{mean}}

and

r_{\mathrm{w\text{-}mean}}

. For single-score baselines, majority AUROC is based on binary decisions. FP and FN denote false positives and false negatives, respectively.

Condition	Mean s/doc	P95 s/doc	Docs/s	Chunks/doc	Coverage (%)	Peak GPU memory (GiB)
Full document	0.813	1.213	1.23	1.00	100.00	17.63
Truncated at 768	0.153	0.160	6.52	1.00	30.69	13.95
$(600,0)$	0.412	0.511	2.43	4.64	100.00	15.03
$(650,150)$	0.501	0.634	2.00	5.32	100.00	15.14

Table 5: Computational performance on a single A100 GPU. Time covers document preprocessing and scoring, excluding model loading, warmup, and report generation. P95 represents the 95th percentile of processing time. Coverage is the mean proportion of source tokens retained, and peak GPU memory denotes allocated memory, including model weights.

Model	Condition	Mean-score AUROC	Majority macro-F1	Conditional ASR	ASR count
Model	Condition	[95% CI]	[95% CI]	[95% CI]	ASR count
Gemma-3-12B	Original	0.9999 [0.9997, 1.0000]	0.9814 [0.9714, 0.9914]	N/A	N/A
Gemma-3-12B	Paraphrase 1	0.9983 [0.9969, 0.9993]	0.9729 [0.9600, 0.9843]	0.0171 [0.0057, 0.0314]	6/350
Gemma-3-12B	Paraphrase 2	0.9984 [0.9960, 1.0000]	0.9786 [0.9671, 0.9886]	0.0057 [0.0000, 0.0143]	2/350
Gemma-3-12B	Human-style AI rewrite	0.9914 [0.9840, 0.9969]	0.9614 [0.9457, 0.9743]	0.0400 [0.0200, 0.0629]	14/350
Sailor2-8B	Original	1.0000 [1.0000, 1.0000]	0.9814 [0.9714, 0.9914]	N/A	N/A

Table 6: Paired rewriting evaluation with 350 matched Human/AI pairs per model and condition at the fixed Youden threshold (

t^{*}\approx 0.928

) with

(W,O)=(650,150)

. AUROC is evaluated using negative mean chunk scores, while macro-F1 and conditional ASR follow majority voting. The final column reports the evasion count over originally detected AI documents, and brackets indicate 95% source-cluster bootstrap confidence intervals. N/A denotes not applicable for the unperturbed baseline.

Class	Documents	Pages	Micro-CER	Whitespace-token WER	At generation limit	Failed pages
Class	Documents	Pages	[95% CI]	[95% CI]	(pages)	Failed pages
Human	227	1,015	0.6358 [0.5805, 0.6932]	0.7334 [0.6716, 0.7991]	232	0
AI	227	1,574	0.2944 [0.2565, 0.3339]	0.3312 [0.2887, 0.3749]	94	1
All	454	2,589	0.4528 [0.4193, 0.4879]	0.5187 [0.4808, 0.5570]	326	1

Table 7: OCR transcription quality on the controlled clean-scan subset of Gemma-3-12B-News-Concat-Test. Micro-CER and whitespace-token WER are reported with 95% stratified source-pair bootstrap confidence intervals from 2,000 resamples. The last two columns report pages reaching the maximum generation limit and unrecoverable page failures.

Condition	Aggregation	AUROC			Macro-F1
Condition	Aggregation	Original	Clean scan	Decrease [95% CI]	Original	Clean scan	Change [95% CI]
Full document	Majority	0.7907	0.7841	0.0066 [ $-$ 0.0352, 0.0463]	0.7812	0.7736	$-$ 0.0076 [ $-$ 0.0539, 0.0405]
Full document	Mean	1.0000	0.9847	0.0153 [0.0041, 0.0298]	0.7812	0.7736	$-$ 0.0076 [ $-$ 0.0539, 0.0405]
Truncated at 768	Majority	0.9604	0.9405	0.0198 [ $-$ 0.0044, 0.0441]	0.9603	0.9403	$-$ 0.0200 [ $-$ 0.0444, 0.0044]
Truncated at 768	Mean	0.9994	0.9789	0.0205 [0.0065, 0.0380]	0.9603	0.9403	$-$ 0.0200 [ $-$ 0.0444, 0.0044]
$(600,0)$	Majority	1.0000	0.9966	0.0034 [0.0004, 0.0090]	1.0000	0.8632	$-$ 0.1368 [ $-$ 0.1674, $-$ 0.1069]

Table 8: Paired detection results for original text and clean-scan OCR text at the fixed Youden threshold. The evaluation includes 454 documents in 227 source pairs and uses the four configurations selected before Test evaluation. AUROC scores follow the document-level definitions in Section 3.4 . For single-score baselines, majority AUROC is based on binary decisions. Brackets report pointwise 95% source-pair bootstrap confidence intervals from 2,000 resamples for the paired AUROC decrease (original minus clean scan) and the macro-F1 change (clean scan minus original).

Condition	Aggregation	False Positives (FP)		False Negatives (FN)
Condition	Aggregation	Original	Clean scan	Original	Clean scan
Full document	Majority	95	98	0	0
Full document	Mean	95	98	0	0
Truncated at 768	Majority	17	27	1	0
Truncated at 768	Mean	17	27	1	0
$(600,0)$	Majority	0	61	0	0

Table 9: False-positive and false-negative counts for original and clean-scan OCR text at the fixed Youden threshold across the 227 matched pairs.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Detection thresholds updated in July 2026 using VietAIDetector-Calibration-News-2026-07, which contains 10,000 Human and 10,000 AI records. (a) Youden and closest-point thresholds. (b) TPR@5%FPR threshold.

Chunking		Claude-Sonnet-4.6			Gemini-3.7-Flash			GPT-5.6-Luna			Time
$W$	$O$	AI%	Chunks	Recall	AI%	Chunks	Recall	AI%	Chunks	Recall	(s)
200	50	81.98	14.95	90.0	59.44	15.10	65.0	67.81	16.00	90.0	0.41
200	100	82.24	21.60	95.0	55.62	22.05	55.0	69.13	23.00	95.0	0.54
200	150	82.73	41.80	100.0	57.11	42.60	65.0	69.34	44.40	95.0	1.05
250	50	80.99	11.25	90.0	63.37	11.55	75.0	73.33	12.00	90.0	0.36
250	100	86.38	14.60	100.0	59.10	14.95	65.0	70.52	15.40	95.0	0.45

Appendix

Table 10: Exploratory results on Claude-Sonnet-4.6-News, Gemini-3.7-Flash-News, and GPT-5.6-Luna-News across chunking parameters (

W\in[200,650]

O\in[50,150]

). The Youden threshold is

t^{*}=0.927966

, with

N=20

per dataset.

Figure 7: Development results for 40 configurations on 592 Gemma-3-12B-News-Concat-Dev documents under majority voting at the fixed Youden threshold. Each cell reports macro-F1 above mean chunks per document. Color represents macro-F1 on a restricted scale from 0.994 to 1.000. Outlined cells achieve both macro-F1 and AUROC of 1.0000. Stars identify the fewest-chunk configurations among tied candidates, selected separately for overlapping and non-overlapping windows.

Figure 8: Empirical token-length probability distributions across evaluation and calibration datasets. (a) VietAIDetector-Calibration-News-2026-07 with 10,000 Human and 10,000 AI records. (b) Exploratory long AI benchmarks with 20 documents each for Claude-Sonnet-4.6-News, Gemini-3.7-Flash-News, and GPT-5.6-Luna-News. (c) Gemma-3-12B-News-Concat-Dev with 296 Human and 296 AI documents. (d) Gemma-3-12B-News-Concat-Test with 1,179 Human and 1,179 AI documents. Vertical dashed and dotted lines indicate median token counts for Human and AI classes, respectively.

AI-Generated Text Detection

AI-generated text detection develops methods to distinguish human-written, machine-generated, and AI-edited text.

See all 171 papers

VietBinoculars: A Zero-Shot Approach for Detecting Vietnamese LLM-Generated Text
Sep 30, 2025Trieu Hai Nguyen, Sivaswamy AkileshAI-Generated Text Detection Language Model Robustness
Faculty of Information Technology, Nha Trang University, 02 Nguyen Dinh Chieu Street, North Nha Trang 57100, Vietnam · Faculty of Business Administration, Swiss School of Business and Management Geneva, 12 Avenue des Morgines, Geneva 1213, Switzerland
'Your AI Text is not Mine': Redefining and Evaluating AI-generated Text Detection under Realistic Assumptions
Jun 3, 2026Nils Dycke, Marina Sakharova, Nico Daheim +1AI-Generated Text Detection Generative AI Evaluation
Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science, Technical University of Darmstadt · National Research Center for Applied Cybersecurity ATHENE, Germany · Zuse School ELIZA
READER: Reasoning-Enhanced AI-Generated Text Detection
May 24, 2026Pingfan Su, Kai Ye, Shijin Gong +4AI-Generated Text Detection Large Language Model-Based Text Classification
Department of Statistics, London School of Economics and Political Science · School of Management, University of Science and Technology of China · School of Mathematics, University of Birmingham

Sep 30, 2025J/K move · Enter open · S save

VietBinoculars: A Zero-Shot Approach for Detecting Vietnamese LLM-Generated Text

Trieu Hai Nguyen, Sivaswamy Akilesh

Faculty of Information Technology, Nha Trang University, 02 Nguyen Dinh Chieu Street, North Nha Trang 57100, Vietnam · Faculty of Business Administration, Swiss School of Business and Management Geneva, 12 Avenue des Morgines, Geneva 1213, Switzerland

AI-Generated Text Detection Language Model Robustness Robustness of Machine-Generated Text Detection

Open paper AskPDF arXiv

The rapid proliferation of Large Language Models has intensified the challenge of distinguishing LLM-generated text from human writing in non-English languages. This study introduces VietBinoculars, a zero-shot detection framework coupling PhoGPT-4B observer and performer models with calibrated global decision thresholds. By utilizing specialized Vietnamese BPE tokenization, the method eliminates byte-level fragmentation and probability dilution common in massive multilingual backbones. Evaluated across multi-domain benchmarks, VietBinoculars achieves an area under the ROC curve exceeding 0.99. Under optimal Youden's J thresholds and greedy decoding, detection accuracy reaches at least 98.78%, while significantly outperforming baseline Binoculars, zero-shot detectors, and commercial tools on creative Capybara prompts. Even under a strict false positive rate constraint of 0.06%, the detector maintains F1-scores between 83.15% and 94.70%. Detection performance consistently improves with sequence length, stabilizing at optimal accuracy for passages containing 450 to 550 tokens. Extended stress testing across 48 distinct model-decoding configurations and three post-generation rewriting strategies delineates practical operational boundaries. VietBinoculars exhibits robust resilience against single-pass paraphrasing and human-style revisions, but experiences notable performance degradation under high-entropy sampling and iterative double paraphrasing.

Abstract

AI-Generated Text Detection

Explore similar work

VietBinoculars: A Zero-Shot Approach for Detecting Vietnamese LLM-Generated Text

'Your AI Text is not Mine': Redefining and Evaluating AI-generated Text Detection under Realistic Assumptions

READER: Reasoning-Enhanced AI-Generated Text Detection