cs.LGJun 16, 2026

TuneAhead: Predicting Fine-tuning Performance Before Full Training Begins

Authors: Yuxiang LuoHaonan LongChen WangQiqi DuanXiaotian LinYanwei XuYuyu LuoWeikai Yang+1 more

Abstract

Fine-tuning large language models (LLMs) is compute-intensive and error-prone: model performance depends sensitively on data quality and hyperparameter choices, and naïve runs can even degrade model performance. This raises a practical question:can we predict fine-tuning performance before committing to a full training run? We present TUNEAHEAD, a lightweight framework for pre-hoc prediction of fine-tuning performance. TUNEAHEAD encodes each candidate run as a meta-feature vector that combines static dataset descriptors with dynamic probe features from a short standardized probe. A predictor maps these features to performance estimates, while SHAP-based attributions provide interpretable diagnostics that reveal which specific features drive the prediction. Across 1,300+ fine-tuning runs on Qwen2.5-7B-Instruct, TUNEAHEAD consistently outperforms strong baselines such as Early-Stop Extrapolation and ProxyLM. On a held-out test set of 370 runs, TUNEAHEAD achieves an RMSE of 1.47 percentage points and places 95.1% of predictions within +3/-3 percentage points of the true score. These accurate continuous predictions support practical go/no-go screening policies that can reduce unnecessary full fine-tuning while retaining most promising runs.

Explore similar work

CardsList
  1. TopoTuner: Topological Finetuning of Large Language Models

    Jul 18, 2026Abdulkadir Erol, Yash Mahajan, Vepaul Hariprashad +4FinetuningTuning

  2. torchtune: PyTorch native post-training library

    May 20, 2026Mark Obozov, Maxime Griot, Joseph Cummings +8PytorchLLM Post-Training Methods