JudgeStealer: Extracting LLM Judging Capabilities across Evaluation Protocols

Chen Chen; Yaolin Chen; Xuehan Sun; Juan Lin; Xueluan Gong; Yuhang Zheng; Qian Wang; Kwok-Yan Lam

cs.CLAug 27, 2026

JudgeStealer: Extracting LLM Judging Capabilities across Evaluation Protocols

Authors: Chen Chen, Yaolin Chen, Xuehan Sun, Juan Lin, Xueluan Gong, Yuhang Zheng, Qian Wang, Kwok-Yan Lam

Organizations: Nanyang Technological University · Wuhan University

Abstract

Large language model (LLM) judges are increasingly used across various evaluation scenarios, making their judgment capabilities valuable intellectual property. However, black-box access exposes these capabilities to model extraction attacks. Existing extraction methods do not specifically target LLM judges and provide limited support for multiple evaluation protocols under restricted query budgets. In this study, we propose JUDGESTEALER, the first query-efficient model extraction framework for replicating judging capabilities across pointwise scoring, pairwise comparison, and listwise ranking protocols. JUDGESTEALER exploits the strong cross-protocol agreement to acquire pointwise scores and transform them into pairwise and listwise supervisions without additional victim queries. To capture informative judge patterns and improve query efficiency, JUDGESTEALER dynamically selects pointwise inputs based on semantic diversity, predictive uncertainty, and potential judge biases. It further applies score smoothing and multi-protocol review to preserve the ordinal structure of scores and mitigate catastrophic forgetting during surrogate adaptation. Extensive experiments on state-of-the-art LLM-as-a-judge and reward models show that JUDGESTEALER consistently outperforms existing extraction baselines, achieving up to 73.3%, 87.0%, and 71.6% accuracy for pointwise, pairwise, and listwise evaluation, respectively. JUDGESTEALER also remains effective across different sur- rogate model scales, adaptation strategies, and reasoning settings. Moreover, JUDGESTEALER demonstrates robustness against representative extraction defenses.

Figures & tables

Panel of Table 1 — Table 1: A comprehensive comparison of state-of-the-art model extraction attacks.

Figure 1: Cross-protocol Agreement.

Figure 2: Overview of JudgeStealer . In Stage I, JudgeStealer identifies informative instances, submits them to the victim under the pointwise protocol, and trains an initial surrogate model. In Stage II, the pointwise supervision is transformed into pairwise and listwise data, which are used to train the surrogate model. A smoothing mechanism is applied in both stages.

Victim Model	Dataset	Method	Qwen3-1.7B						Llama-3.2-1B
			Pointwise			Pairwise	Listwise		Pointwise			Pairwise	Listwise
			$Acc\uparrow$	$Acc_{\pm 1}\uparrow$	$MAE\downarrow$	$Acc\uparrow$	$Acc\uparrow$	$MAE\downarrow$	$Acc\uparrow$	$Acc_{\pm 1}\uparrow$	$MAE\downarrow$	$Acc\uparrow$	$Acc\uparrow$	$MAE\downarrow$
GPT-5.4	Alpaca	Vanilla	0.2500	0.4865	2.4430	0.5633	0.6300	0.3052	0.1895	0.4180	2.7800	0.6482	0.3855	0.5770
		Lion	0.1110	0.3190	2.5889	0.3740	0.3590	0.4792	0.2045	0.4595	3.1732	0.3635	0.0945	0.6941
		LoRD	0.1470	0.1700	2.7975	0.4680	0.1530	0.8775	0.2005	0.4590	2.9910	0.4340	0.1430	0.9133
		Proxy-KD	0.3005	0.4512	2.5155	0.6240	0.3165	0.5340	0.2218	0.2985	2.5887	0.4190	0.1670	0.7256

Table 2: Experimental results across different datasets, surrogate and victim models.

Figure 3: Model scale and training strategy evaluation.

Dataset	Method	Llama-3.2-1B						Qwen3-1.7B
		Pointwise			Pairwise	Listwise		Pointwise			Pairwise	Listwise
		$Acc\uparrow$	$Acc_{\pm 1}\uparrow$	$MAE\downarrow$	$Acc\uparrow$	$Acc\uparrow$	$MAE\downarrow$	$Acc\uparrow$	$Acc_{\pm 1}\uparrow$	$MAE\downarrow$	$Acc\uparrow$	$Acc\uparrow$	$MAE\downarrow$
Alpaca	Vanilla	0.2748	0.5461	2.0435	0.8291	0.6439	0.3089	0.3283	0.5839	1.6474	0.8520	0.6200	0.3422
Alpaca	Ours	0.3326	0.5670	1.6737	0.8620	0.6506	0.3033	0.3698	0.6443	1.5139	0.8770	0.7094	0.2422
GPT4All	Vanilla	0.2657	0.5409	2.0724	0.8326	0.6500	0.3033	0.2735	0.5120	2.3428	0.8754	0.7344	0.2176
GPT4All	Ours	0.3930	0.6669	1.4035	0.8796	0.7533	0.2056	0.3852	0.6669	1.3880	0.8857	0.7478	0.2057

Table 3: Evaluation results on Chain-of-thought (CoT) reasoning settings.

Surrogate Model	Method	$MAE\downarrow$	$Acc\uparrow$	$Acc@Top\uparrow$
Qwen3-1.7B	Vanilla	1.0567	0.7667	0.7400
	Lion	0.6478	0.7770	0.7575
	LoRD	0.7364	0.6180	0.5203
	Proxy-KD	0.7506	0.5860	0.6495
	Ours	0.6354	0.8356	0.7967
Llama-3.2-1B	Vanilla	1.0566	0.6622	0.6000

Table 4: Evaluation results of reward models.

No.	Setting	Alpaca						GPT4All
		Pointwise			Pairwise	Listwise		Pointwise			Pairwise	Listwise
		$Acc\uparrow$	$Acc_{\pm 1}\uparrow$	$MAE\downarrow$	$Acc\uparrow$	$Acc\uparrow$	$MAE\downarrow$	$Acc\uparrow$	$Acc_{\pm 1}\uparrow$	$MAE\downarrow$	$Acc\uparrow$	$Acc\uparrow$	$MAE\downarrow$
–	Full Setting (Ours)	0.383	0.591	1.776	0.771	0.634	0.300	0.361	0.610	1.611	0.783	0.635	0.295
Sample Selection Mechanism
1	$D\checkmark\,\cdot\,U\checkmark\,\cdot\,B\times$	0.371( $-.012$ )	0.587( $-.004$ )	1.827( $+.052$ )	0.766( $-.005$ )	0.612( $-.022$ )	0.309( $+.009$ )	0.346( $-.015$ )	0.586( $-.024$ )	1.622( $+.011$ )	0.783 ( $+.000$ )	0.623( $-.012$ )	0.303( $+.008$ )
2	$D\checkmark\,\cdot\,U\times\,\cdot\,B\checkmark$	0.371( $-.012$ )	0.575( $-.015$ )	1.858( $+.083$ )	0.764( $-.006$ )	0.610( $-.025$ )	0.313( $+.013$ )	0.351( $-.010$ )	0.581( $-.029$ )	1.733( $+.122$ )	0.783 ( $+.000$ )	0.618( $-.017$ )	0.303( $+.008$ )

Table 5: Ablation studies. JudgeStealer employs full setting with

D\checkmark\,\cdot\,U\checkmark\,\cdot\,B\checkmark

, Adaptive Smoothing and Consolidation.

Figure 4: Performance on different query budgets.

Model	Method	Pointwise			Pairwise	Listwise
		$ACC$	$ACC_{\pm 1}$	$MAE$	$ACC$	$ACC$
Qwen3-1.7B	w/o defense	0.499	0.737	1.2165	0.803	0.538
	w/ defense	0.498	0.730	1.2465	0.801	0.531
Llama-3.2-1B	w/o defense	0.362	0.692	1.7205	0.730	0.426
	w/ defense	0.360	0.688	1.7450	0.723	0.411

Table 6: Evaluation against anti-distillation defense.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

LLMs	Versions	Availability	Access
Response Generation Models
GPT	GPT-4.1, GPT-4o	Proprietary	OpenAI API
Claude	Sonnet 4.5	Proprietary	Anthropic API
Amazon Nova	Nova Micro, Lite, Nova 2 Lite	Proprietary	AWS Bedrock
Llama	Llama 3.1 Instruct 8B, 70B	Open-source	AWS Bedrock
Nvidia	Nemotron Nano 9B, 12B	Open-source	AWS Bedrock

Appendix

Table 7: Target LLM agents and the judge model.

Panel of Figure 5 — Figure 5: Prompt template for pointwise evaluation.

Panel of Figure 6 — Figure 6: Prompt template for pairwise evaluation.

Panel of Figure 7 — Figure 7: Prompt template for listwise evaluation.

Surrogate Model	Alpaca						GPT4All
	Pointwise			Pairwise	Listwise		Pointwise			Pairwise	Listwise
	$Acc\uparrow$	$Acc_{\pm 1}\uparrow$	$MAE\downarrow$	$Acc\uparrow$	$Acc\uparrow$	$MAE\downarrow$	$Acc\uparrow$	$Acc_{\pm 1}\uparrow$	$MAE\downarrow$	$Acc\uparrow$	$Acc\uparrow$	$MAE\downarrow$
Qwen3-0.6B	0.3410	0.5235	2.2005	0.8016	0.5295	0.3911	0.2811	0.5122	2.3100	0.7761	0.5627	0.3557
Qwen3-0.6B LoRA	0.2995	0.4960	2.4135	0.8037	0.5250	0.4023	0.2744	0.5011	2.3400	0.7782	0.5530	0.3683
Qwen3-1.7B	0.3965	0.5690	1.8170	0.7810	0.6250	0.2956	0.3644	0.6010	1.6778	0.7717	0.6057	0.3087
Qwen3-1.7B LoRA	0.3830	0.5905	1.8200	0.7692	0.6105	0.3143	0.3389	0.5711	1.6767	0.7743	0.5890	0.3253

Appendix

Table 8: Results of surrogate models across model scales and adaptation strategies.

Dataset	Setting	Pointwise			Pairwise	Listwise
Dataset	Setting	$Acc\uparrow$	$Acc_{\pm 1}\uparrow$	$MAE\downarrow$	$Acc\uparrow$	$Acc\uparrow$	$MAE\downarrow$
Alpaca	Pointwise only	0.325	0.544	1.883	0.171	0.235	0.634
	Pairwise only	0.025	0.289	3.139	0.832	0.374	0.471
	Listwise only	0.123	0.401	2.629	0.637	0.696	0.240
	JudgeStealer	0.383	0.591	1.776	0.771	0.634	0.300
GPT4All	Pointwise only	0.315	0.572	1.661	0.446	0.180	0.765

Appendix

Table 9: Ablation study of cross-protocol transformation.

Figure 8: Cross-protocol agreement on less capable judges.

Model	$W$	$\tau_{W}$
Llama-3.2-1B	0.9370	0.9223
Qwen3-1.7B	0.9240	0.9223

Appendix

Table 10: Evaluation against the anomaly detector.

Role	Model	Method	Avg. Z-score
Victim	Qwen3-32B	Watermarked	6.9173
Surrogate	Qwen3-1.7B	No defense	-1.0170
	Qwen3-1.7B	With defense	-0.2191
	Llama-3.2-1B	No defense	-1.1081
	Llama-3.2-1B	With defense	-0.2559

Appendix

Table 11: Evaluation against ownership tracing defense.

Model Extraction Attacks

Model extraction attacks infer a machine-learning model’s behavior, architecture, parameters, or hidden assets from queries and system leakage.

See all 48 papers

Feb 6, 2026

FairJudge: An Adaptive, Debiased, and Consistent LLM-as-a-Judge

Existing LLM-as-a-Judge systems suffer from three fundamental limitations: limited adaptivity to task- and domain-specific evaluation criteria, systematic biases driven by non-semantic cues such as position, length, format, and model provenance, and evaluation inconsistency that leads to contradictory judgments across different evaluation modes (e.g., pointwise versus pairwise). To address these issues, we propose FairJudge, an adaptive, debiased, and consistent LLM-as-a-Judge. Unlike prior approaches that treat the judge as a static evaluator, FairJudge models judging behavior itself as a learnable and regularized policy. From a data-centric perspective, we construct a high-information-density judging dataset that explicitly injects supervision signals aligned with evaluation behavior. Building on this dataset, we adopt a curriculum-style SFT-DPO-GRPO training paradigm that progressively aligns rubric adherence, bias mitigation, and cross-mode consistency, while avoiding catastrophic forgetting. Experimental results on multiple internal and public benchmarks show that FairJudge consistently improves agreement and F1, reduces non-semantic biases, and outperforms substantially larger instruction-tuned LLMs. All resources will be publicly released after acceptance to facilitate future research.

Bo Yang, Lanfei Feng, Yunkui Chen +3

College of Computer Science, Zhejiang University, Hangzhou, China · School of Software Technology, Zhejiang University, Hangzhou, China

LLM Evaluation LLM-as-a-Judge Language Model Bias Evaluation Bias in LLM-as-a-Judge

Ask PDF

Oct 12, 2026

StableJudge: A Lightweight Adversarial Defense Framework for LLM-as-a-Judge

Large language models (LLMs) are increasingly used as judges to assign scalar scores that drive downstream decision-making, including tasks such as paper review and recommendation. However, LLM-as-a-Judge pipelines are vulnerable to score-hijacking prompt injection embedded in evaluated content, which corrupts scoring signals and distorts induced rankings. Existing defenses often enforce robustness by modifying the judge model itself, making safety dependent on a specific model's scale and training and requiring repeated effort when the judge is updated or replaced. We propose StableJudge, a lightweight defense framework that externalizes prompt-injection mitigation as a reusable pre-scoring capability. StableJudge centers on a compact 0.13B Sentry that localizes injected spans and performs targeted sanitization prior to evaluation, and is trained using adversarially evolving attacks to improve robustness beyond a fixed prompt set. To stabilize measurement, StableJudge derives deterministic, fine-grained scalar scores from the judge's output distribution. Across held-out scientific abstracts and out-of-distribution attack variants, StableJudge improves score recovery under attack while largely preserving general-purpose performance, demonstrating that robust LLM-based judging can be achieved by decoupling safety from judge scale via reusable defense capabilities. Code and data will be released.

Anning Hu, Yangyu Li, Yuqi Huang +2

Shanghai Jiao Tong University

LLM-as-a-Judge Prompt Injection Defense Adversarial Robustness of Language Models Prompt Injection Attacks on LLMs

Ask PDF

Aug 18, 2026

SESSE: Sketch, Expand, Sort, Summarize, Evaluate -- LLM-as-Judge Evaluation via Structured Decomposition

LLM-as-judge evaluation reduces response quality assessment to a single holistic A/B preference choice, providing no mechanism to isolate which quality dimensions drove the preference or distinguish model errors from genuine label ambiguity. We propose SESSE (Sketch, Expand, Sort, Summarize, Evaluate), a training-free framework that decomposes holistic judgment into structured sub-questions mined directly from the judge's own error cases; requiring no oracle responses, task-specific rubrics, or fine-tuning. On RewardBench (n=1,000), SESSE achieves near-parity with the chain-of-thought baseline and is competitive with RISE-Judge-32B (92.7%), a fine-tuned specialist, while remaining fully training-free. Per-criterion vote evidence provides an interpretable audit trail for diagnosing label ambiguity and judge failure modes unavailable from a single holistic output token.

Dae Lee, Mihai Delgeanu, Adel Youssef

Apple

LLM Evaluation LLM-as-a-Judge

Ask PDF

Abstract

Model Extraction Attacks

Explore similar work

FairJudge: An Adaptive, Debiased, and Consistent LLM-as-a-Judge

StableJudge: A Lightweight Adversarial Defense Framework for LLM-as-a-Judge

SESSE: Sketch, Expand, Sort, Summarize, Evaluate -- LLM-as-Judge Evaluation via Structured Decomposition