NNProxy: Efficient Training-Free Proxy Alignment for Black-Box Zero-Shot LLM-Generated Text Detection
Organizations: State Key Laboratory of Internet of Things for Smart City and the Department of Computer and Information Science, Faculty of Science and Technology, University of Macau, Macao 999078, China · School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China
Abstract
LLM-generated text (LGT) detection is essential for reliable forensic analysis and for mitigating LLM misuse. Existing LGT detectors can generally be categorized into two broad classes: learning-based approaches and zero-shot methods. Compared with learning-based detectors, zero-shot methods are particularly promising because they eliminate the need to train task-specific classifiers. However, the reliability of zero-shot methods fundamentally relies on the assumption that an off-the-shelf proxy LLM is well aligned with the often unknown source LLM, a premise that rarely holds in real-world black-box scenarios. To address this discrepancy, existing proxy alignment methods typically rely on supervised fine-tuning of the proxy or repeated interactions with commercial APIs, thereby increasing deployment costs, exposing detectors to silent API changes, and limiting robustness under domain shift. Motivated by these limitations, we propose the -nearest neighbor proxy (NNProxy), a training-free and query-efficient proxy alignment framework that repurposes the NN language model (NN-LM) retrieval mechanism as a domain adapter for a fixed proxy LLM. Specifically, a lightweight datastore is constructed once from a target-reflective LGT corpus, either via fixed-budget querying or from existing datasets. During inference, nearest-neighbor evidence induces a token-level predictive distribution that is interpolated with the proxy output, yielding an aligned prediction without proxy fine-tuning or per-token API outputs. To improve robustness under domain shift, we extend NNProxy into a mixture of proxies (MoP) that routes each input to a domain-specific datastore for domain-consistent retrieval. Extensive experiments demonstrate strong detection performance of our method.
Figures & tables
| Benchmark | Domains (Topics) | Source LLMs | #HWT | #LGT | Notes |
| Mix8 | arXiv (academic) XSum (news) WritingPrompts (story) PubMedQA (technical QA) | GPT-3.5, GPT-4, Claude-3 Sonnet, Claude-3 Opus, Gemini-1.5 Pro, GPT-4 Turbo, Claude-3.7 Sonnet, Gemini-2.0 Flash | 900 | 3600 | Mix8 builds from the Glimpse [ 28 ] and DNA-DetectLLM [ 54 ] datasets, covering 4 domains and 8 proprietary LLMs. |
| DetectRL | arXiv (academic) XSum (news) WritingPrompts (story) Yelp Reviews (social) | GPT-3.5 Turbo, PaLM-2-Bison, Claude-Instant, Llama-2-70B | 4016 | 4016 | DetectRL [ 34 ] includes prompt-based, paraphrasing, and perturbation attacks on the LGTs, covering 4 domains and 4 LLMs. |
| Setting | GPT-3.5 | GPT-4 | Claude-S | Claude-O | Gemini | Avg. |
| Corpus Source LLM | ||||||
| Claude | 0.943 | 0.867 | 0.916 | 0.937 | 0.797 | 0.897 |
| PaLM | 0.955 | 0.908 | 0.916 | 0.940 | 0.825 | 0.913 |
| Llama | 0.961 | 0.901 | 0.921 | 0.948 | 0.815 | 0.914 |
| GPT-3.5 | 0.968 | 0.932 | 0.937 | 0.952 | 0.857 | 0.933 |
| GPT-4 | 0.984 | 0.954 | 0.966 | 0.975 | 0.888 | 0.953 |
| Prompt ID | Content |
| Prompt 0 | (Empty) |
| Prompt A | You serve as a valuable aide, capable of generating clear and persuasive pieces of writing given a certain context. Now, assume the role of an author and strive to finalize this article. |
| Prompt B | I operate as an entity utilizing GPT as the foundational large language model. I function in the capacity of a writer, authoring articles on a daily basis. Presented below is an example of an article I have crafted. |
| Prompt 1 | [Prompt A] |
| Prompt 2 | [Prompt A] [Prompt B] |
| Prompt 3 | System: [Prompt A] Assistant: [Prompt B] |
AI-Generated Text Detection
AI-generated text detection develops methods to distinguish human-written, machine-generated, and AI-edited text.
See all 170 papers