Prior work uses linear probes on benchmark prompts as evidence of evaluation awareness in large language models. Because evaluation context is typically entangled with benchmark format and genre, it is unclear whether probe-based signals reflect context or surface structure. We test whether these signals persist under partial control of prompt format using a controlled 2x2 dataset and diagnostic rewrites. We find that probes primarily track benchmark-canonical structure and fail to generalize to free-form prompts independent of linguistic style. Thus, standard probe-based methodologies do not reliably disentangle evaluation context from structural artifacts, limiting the evidential strength of existing results.
Figures & tables
Figure 1: Overview of our approach. (a) Standard probes train on benchmark vs. chat prompts, where format and context are confounded. (b) Our 2×2 design crosses format and context independently, enabling isolation of each factor.
Format
Evaluation Context
Deployment Context
Benchmark
Bench-Eval
Bench-Deploy
(Structured)
(MMLU, ARC, GSM8K)
(ShareGPT rewritten to
Standard Evaluation
benchmark format)
Casual
Casual-Eval
Casual-Deploy
(Free-form)
(MuTual, 1st turn)
(ShareGPT, 1st turn)
Conversational Proxy
Standard Deployment
Table 1: The 2×2 Dataset Matrix. We cross usage context with prompt format to disentangle structural cues from evaluation intent.
Intended Evaluation
Intended Deployment
Training Regime
Bench-Eval
Casual-Eval
Casual-Deploy
Bench-Deploy
Standard (Single)
98.8%
0.0%
0.0%
94.5% (FP)
Paired (Decorrelated)
98.5%
100.0%
0.0%
7.2%
Table 2: Probing Results (Layer 24). Values indicate the percentage of prompts classified as Evaluation . Standard training conflates format with context, leading to high false positives on Bench-Deploy . Paired training restores correct classification.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 2: Length distributions (in characters) across the four datasets. Casual-Deploy is histogram-matched to Bench-Eval . Bench-Deploy is slightly longer due to formatting overhead, while Casual-Eval (1st turn) is naturally shorter.
Condition
Before cleanup
After cleanup
Bench-Deploy (MCQ rewrite)
55–70%
94–95%
Bench-Deploy Formal
51–68%
88–91%
Appendix
Table 3: Effect of removing unintended answer leakage. Values indicate % classified as Evaluation. Removing leakage makes the probe more likely to flag the prompt as Eval, isolating the format effect.
Layer
B-Eval
C-Eval
C-Deploy
B-Deploy
8
99.8%
0.0%
0.0%
94.3%
16
99.5%
0.0%
0.0%
94.3%
24
98.8%
0.0%
0.0%
94.5%
30
99.2%
0.0%
0.0%
94.6%
31
99.1%
0.0%
0.0%
94.6%
Appendix
Table 4: Standard (Single-Contrast) Training Results across Layers. Values indicate % classified as Eval. The standard probe consistently relies on format, incorrectly flagging Bench-Deploy as Evaluation and ignoring Casual-Eval .
Layer
B-Eval
C-Eval
C-Deploy
B-Deploy
8
98.8
98.6
0.0
7.2
16
97.7
99.1
0.0
4.8
24
98.5
100.0
0.0
7.2
30
99.0
99.5
0.0
7.2
31
98.7
99.5
0.0
6.0
Appendix
Table 5: Paired Training Results across Layers. Values indicate % classified as Eval. The paired probe consistently learns Context (Evaluation) vs. Format (Benchmark).
Dataset
Single Probe
Paired Probe
AdvBench (520)
0%
0%
HarmBench standard (200)
0%
0%
HarmBench contextual (100)
0%
0%
HarmBench copyright (100)
0%
0%
Appendix
Table 6: Probe classification on AI Safety Datasets (Layer 24). All safety prompts are classified as Deployment.
Evaluation awareness in language models studies how models recognize testing or monitoring and whether that recognition changes their answers, safety, or strategic behavior.