Fast and Accurate Probing of In-Training LLMs' Downstream Performances

Zhichen Liu; Tianle Lun; Zhibin Wen; Hao An; Yulin Ou; Jianhui Xu; Hao Zhang; Wenyi Fang

cs.LGApr 1, 2026

Fast and Accurate Probing of In-Training LLMs' Downstream Performances

Authors: Zhichen Liu, Tianle Lun, Zhibin Wen, Hao An, Yulin Ou, Jianhui Xu, Hao Zhang, Wenyi Fang,

Organizations: Southern University of Science and Technology · Huawei

Abstract

The paradigm of scaling Large Language Models (LLMs) in both parameter size and test time has pushed the boundaries of AI capabilities, but at the cost of making the traditional generative evaluation paradigm prohibitively expensive, therefore making the latency of LLM's in-training downstream performance evaluation unbearable. However, simple metrics like training loss (perplexity) are not always correlated with downstream performance, as sometimes their trends diverge from the actual task outcomes. This dilemma calls for a method that is computationally efficient and sufficiently accurate in measuring model capabilities. To address this challenge, we introduce a new in-training evaluation paradigm that uses a lightweight probe for monitoring downstream performance. The probes take the internal representations of LLM checkpoints (during training) as input and directly predict the checkpoint's performance on downstream tasks measured by success probability (i.e., pass@1). We design several probe architectures, validating their effectiveness using the OLMo3-7B's checkpoints across a diverse set of downstream tasks. The probes can accurately predict a checkpoint's performance (with avg. AUROC $>$ 0.75), have decent generalizability across checkpoints (earlier predicts later), and reduce the computation latency from $\sim$ 1 hr (using conventional generative evaluation method) to $\sim$ 3 min. In sum, this work presents a practical and scalable in-training downstream evaluation paradigm, enabling a more agile, informed, and efficient LLM development process.

Figures & tables

Figure 1: Workflow comparison between the current generative evaluation paradigm and the proposed probing evaluation paradigm for in-training downstream monitoring. Current paradigm suffers from high evaluation cost and excessive latency, resulting in delayed feedback to training progress. In contrast, probe can bypasses the generation process during the evaluation phase, thereby enabling rapid and timely assessment.

Figure 2: Demonstration of the working process of in-training downstream performance evaluation framework in training stage, and the structure of two lightweight probe models. Training would be applied only in specific checkpoints. Once a probe is trained, it could maintain predictability in the future checkpoints.

Dataset	Domain	N-shot	Stage	Sampling Budget	Train Size	Test Size
MMLU	Knowledge	5	Pre/Post	8/4	10k	14k
GSM8K	Math	8	Pre/Post	8/4	7.5k	1.3k
MATH	Math	4	Pre/Post	8/4	6k	500
BBH	Reasoning	3	Pre/Post	8/4	5.2k	1.4k
HumanEval	Code	0	Pre/Post	8/4	100	64
MBPP	Code	3	Pre/Post	8/4	710	250

Table 1: Summary of datasets used for probe training and evaluation. N-shot denotes the number of in-context exemplars. Stage indicates the applicable training phase: Pre (Pre-training) and Post (Post-training/Instruction Tuning). Sampling Budget (

n

) refers to the number of samples generated to estimate the ground-truth Pass@1 score in pre/post stage.

	Metric	MMLU	GSM8k	MATH	BBH	HumanEval	MBPP	Avg.
Loss fit	AUROC ( $\uparrow$ )	0.4588	0.6189	0.6177	0.5219	0.7048	0.5549	0.5795
Loss fit	MSE ( $\downarrow$ )	0.2322	0.1239	0.0399	0.1880	0.1407	0.1755	0.1500
Linear	AUROC ( $\uparrow$ )	0.6019	0.5036	0.4714	0.7309	0.5966	0.5818	0.5810
Linear	MSE ( $\downarrow$ )	0.1780	0.1310	0.0406	0.1534	0.1773	0.1686	0.1415
Submodel	AUROC ( $\uparrow$ )	0.6737	0.7827	0.7776	0.8014	0.9163	0.7822	0.7890
Submodel	MSE ( $\downarrow$ )	0.1492	0.0962	0.0333	0.1302	0.0810	0.1395	0.1049

Table 2: Estimation fidelity on Pre-training Checkpoints. The models are trained and evaluated on the OLMo-3-Base-800k checkpoint. (

\uparrow

: higher is better;

\downarrow

: lower is better).

	Train Ckpt (k)	Test Ckpt (k)
	Train Ckpt (k)	200	400	600	800	1000	1200	1400	last
Submodel	200	0.7639	0.7346	0.7523	0.7633	0.7504	0.7379	0.7414	0.7513
	400	–	0.7491	0.7654	0.7764	0.7647	0.7561	0.7554	0.7657
	600	–	–	0.7773	0.7850	0.7697	0.7617	0.7628	0.7685
	800	–	–	–	0.7890	0.7719	0.7695	0.7680	0.7731
	1000	–	–	–	–	0.7728	0.7608	0.7630	0.7687

Table 3: Forward Transferability Analysis. We report AUROC scores where probes are trained on earlier checkpoints (rows) and tested on future checkpoints (columns).

Figure 3: Cumulative time consumption for Probe Evaluation vs. Generative Evaluation on pre-training checkpoints (OLMo-3-Base).

Num of Layers	MMLU	GSM8k	MATH	BBH	HumanEval	MBPP	Avg.
8-layer	0.6079	0.6663	0.6921	0.7894	0.8612	0.5911	0.7013
16-layer	0.6937	0.7463	0.7636	0.7963	0.9184	0.7166	0.7725
Full (32-layer)	0.6737	0.7827	0.7776	0.8014	0.9136	0.7822	0.7890

Table 4: Impact of Input Layer Depth. Comparison of Submodel probes trained on the first

k

layers versus the full model (32 layers).

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Table 5: Comprehensive AUROC Results for OLMo-3-base across Six Datasets (Train ckpt

\times

Test ckpt)

Table 6: OLMo-3-Instruct Results

Table 7: OLMo-3-Think AUROC Results (Total 9 Datasets)

Absolute Error	HumanEval	MATH	BBH	MMLU	GSM8k	MBPP
Probe	0.0263	0.0009	0.0447	0.0159	0.0081	0.0952
Subset	0.2305	0.0012	0.1221	0.0320	0.009	0.0761

Appendix

Table 8: Comparison between probe evaluation and subset sampling on absolute error (smaller is better).

Metrics	Datasets	HumanEval	Math	BBH	MMLU	GSM8K	MBPP
AUROC	Subset	0.8946	0.7766	0.7373	0.6824	0.7232	0.6251
AUROC	Fullset	0.9163	0.7776	0.8014	0.6737	0.7827	0.7822
MSE	Subset	0.1035	0.0354	0.1520	0.1566	0.1167	0.1692
MSE	Fullset	0.0810	0.0333	0.1302	0.1492	0.0962	0.1395

Appendix

Table 9: Comparison between probe trained on subset and fullset.

Panel of Figure 4 — Figure 4: Cumulative time consumption for Probe Evaluation vs. Generative Evaluation on post-training checkpoints for Instruct and Think models.

	data collection	probe training
base	5.9h	2.5h
instruct	35.5h	2.6h
think	112h	2.6h

Appendix

Table 10: Detailed time cost of probe preparation.

LLM Evaluation

Large language model evaluation measures what models can do, how reliably they do it, and whether benchmark scores predict real-world performance.

See all 2,408 papers

May 18, 2026

Forecasting Downstream Performance of LLMs With Proxy Metrics

Progress in language model development is often driven by comparative decisions: which architecture to adopt, which pretraining corpus to use, or which training recipe to apply. Making these decisions well requires reliable performance forecasts, yet the two commonly used signals are fundamentally limited. Cross-entropy loss is poorly aligned with downstream capabilities, and direct downstream evaluation is expensive, sparse, and often uninformative at early training stages. Instead, we propose to construct proxy metrics by aggregating token-level statistics, such as entropy, top-k accuracy, and expert token rank, from a candidate model's next token distribution over expert-written solutions. Across three settings, our proxies consistently outperform loss- and compute-based baselines: 1) For cross-family model selection, they rank a heterogeneous population of reasoning models with mean Spearman Rho = 0.81 (vs. Rho = 0.36 for cross-entropy loss); 2) For pretraining data selection, they reliably rank 25 candidate corpora for a target model at roughly

10{,}000\times

less compute than direct evaluation, pushing the Pareto frontier beyond existing methods; and 3) for training-time forecasting, they extrapolate downstream accuracy across an

18\times

compute horizon with roughly half the error of existing alternatives. Together, these results suggest that expert trajectories are a broadly useful source of signal for assessing model capabilities, enabling reliable performance forecasting throughout the model development life cycle.

Arkil Patel, Siva Reddy, Marius Mosbach +1

Mila – Quebec AI Institute & McGill University Canada · ServiceNow Research · Periodic Labs

Language Model Pretraining LLM Evaluation Language Model Performance Prediction

Ask PDF

Jan 30, 2026

Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry

Large language models (LLMs) are widely used as reference-free evaluators via prompting, but this "LLM-as-a-Judge" paradigm is costly, opaque, and sensitive to prompt design. In this work, we investigate whether smaller models can serve as efficient evaluators by leveraging internal representations instead of surface generation. We uncover a consistent empirical pattern: small LMs, despite with weak generative ability, encode rich evaluative signals in their hidden states. This motivates us to propose the Semantic Capacity Asymmetry Hypothesis: evaluation requires significantly less semantic capacity than generation and can be grounded in intermediate representations, suggesting that evaluation does not necessarily need to rely on large-scale generative models but can instead leverage latent features from smaller ones. Our findings motivate a paradigm shift from LLM-as-a-Judge to Representation-as-a-Judge, a decoding-free evaluation strategy that probes internal model structure rather than relying on prompted output. We instantiate this paradigm through INSPECTOR, a probing-based framework that predicts aspect-level evaluation scores from small model representations. Experiments on reasoning benchmarks (GSM8K, MATH, GPQA) show that INSPECTOR substantially outperforms prompting-based small LMs and closely approximates full LLM judges, while offering a more efficient, reliable, and interpretable alternative for scalable evaluation. The code and data are available at: https://github.com/zhuochunli/Representation-as-a-judge

Zhuochun Li, Yong Zhang, Ming Li +8

Ping An Technology (Shenzhen) Co., Ltd. · University of Pittsburgh · University of Maryland, College Park +1

LLM Evaluation LLM-as-a-Judge Small Language Models Language Model Probing

Ask PDF

Sep 25, 2025

Anticipatory Evaluation of Language Models: Predicting LLM Benchmark Scores from Descriptions

Modern ML research increasingly faces an evaluation bottleneck: building benchmarks and running experiments is costly, slowing iteration. We ask how much can be inferred about evaluation outcomes before experiments, given only a natural-language description of the task and setup. We call this setting anticipatory evaluation. We introduce EVALCAST, a literature-grounded resource of 2,290 description-performance pairs from 1,519 arXiv papers, including a test set of papers published after the model's knowledge cutoff. We evaluate two tasks: predicting absolute performance through regression and comparing benchmark difficulty through pairwise prediction. Experiments with frontier LLMs show that accurate regression remains challenging, while pairwise predictions can recover relative performance ordering under matched settings. Comparable performance on post-cutoff and streaming evaluations suggests that the observed signal is not solely attributable to previously reported results. Overall, EVALCAST provides a resource and evaluation framework for studying anticipatory evaluation and its potential use in prioritizing benchmarks.

Jungsoo Park, Ethan Mendes, Gabriel Stanovsky +1

Georgia Institute of Technology, Atlanta, Georgia · The Hebrew University of Jerusalem, Jerusalem, Israel

Language Model Performance Prediction LLM Evaluation

Ask PDF

Abstract

LLM Evaluation

Explore similar work

Forecasting Downstream Performance of LLMs With Proxy Metrics

Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry

Anticipatory Evaluation of Language Models: Predicting LLM Benchmark Scores from Descriptions