stat.MLNov 9, 2026

Prediction with Foundation Models under Deployment Budgets

Authors: Zhaohui Xu, Xiaofei Xu, Doudou Zhou

Organizations: Department of Statistics and Data Science, National University of Singapore · School of Mathematics and Statistics, Wuhan University

Abstract

Foundation models can supply predictions for a downstream task that draw on knowledge from extensive pretraining, but they differ in task-specific accuracy and query cost. We develop a framework that improves prediction for the task under a budget on the average query cost per input, using training data to learn which models to query and how to combine their outputs with features available without querying. The first method queries the same models for every input; the second lets them vary across inputs and uses Q-learning to decide which model to query next and when to stop. For the first method, we show that several individually inaccurate outputs can jointly outperform the best single one. It attains the minimax rate when the regression function depends on the outputs only through models within the budget. Otherwise, any method within the budget can be forced to a slower rate, which the first method attains. For the second method, we show when varying the queried models across inputs strictly improves on querying the same models for every input, and bound its prediction risk and average cost in finite samples. In simulations and a multimodal laptop price application, both methods outperform existing methods.

Figures & tables

Explore similar work

CardsList
  1. Scout Before You Route: Predicting Task Success and Token Count from a Small LLM Prefill

    Nov 6, 2026Aristides Milios, Christopher PalLLM Output Length PredictionLLM Routing

  2. Learning to Fine-tune Foundation Models under Resource Limitations

    Jul 12, 2026Thomas Tsouparopoulos, Iordanis KoutsopoulosFine-TuningLLM Fine-Tuning

  3. Embedding Foundation Model Predictions in Discrete-Choice Models with Structural Guarantees

    Jun 24, 2026Yingshuo Wang, Xian Sun, Yanhang Li +2Tabular Foundation ModelsMonotonicity Constraints