Modern LLMs scale at test-time, e.g. via repeated sampling, where inference cost grows with model size and the number of samples. This creates a trade-off that pretraining scaling laws, such as Chinchilla, do not address. We present Train-to-Test (T2) scaling laws that jointly optimize model size, training tokens, and number of inference samples under fixed end-to-end budgets. T2 modernizes pretraining scaling laws with pass@k modeling used for test-time scaling, then jointly optimizes pretraining and test-time decisions. Forecasts from T2 are robust over distinct modeling approaches: measuring joint scaling effect on the task loss and modeling impact on task accuracy. Across eight downstream tasks, we find that when accounting for inference cost, optimal pretraining decisions shift radically into the overtraining regime, well-outside of the range of standard pretraining scaling suites. We validate our results by pretraining heavily overtrained models in the optimal region that T2 scaling forecasts, confirming their substantially stronger performance compared to pretraining scaling alone. Finally, as frontier LLMs are post-trained, we show that our findings survive the post-training stage, making T2 scaling meaningful in modern deployments.
Figures & tables
Figure 1: Our T2 scaling laws combine Chinchilla scaling for pretraining with pass@ k modeling for test-time scaling via repeated sampling to obtain optimal pretraining allocations subject to a test-time scaling budget. T2 recommends overtraining compared to Chinchilla.
Figure 2: Optimal pretraining forecasts predicted by both T2 approaches, compared to Hoffmann et al. (2022) . (Left) Optimal tokens per parameter (including the 20 tokens per parameter rule of thumb used by practitioners), (Middle) Optimal model sizes. (Right) Optimal training set sizes. Both T2 approaches forecast extreme overtraining.
Figure 3: T2 scaling across all of our evaluation tasks. Both approaches improve monotonically over Chinchilla scaling, while Chinchilla exhibits non-monotonic scaling in Ctrain .
Figure 4
Figure 5: T2 overtraining findings survive post-training. The optimal frontier is slightly subdued compared to base models, which is consistent with Springer et al. (2025) .
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Approach 1 IsoFLOP profiles across different scaling budgets for all eight tasks.
Figure 7: Approach 2 IsoFLOP profiles across different scaling budgets for all eight tasks.
Figure 8: Overall checkpoint scaling grid. Each cell reports the number of tokens per parameter. Orange cells are overtrained checkpoints we created.