Authors: Thomas Simon Foster, Bassel Al Omari, Tingchen Fu, Thomas Mann, Carl Domond, Lucia Cipolina-Kun, Bhavul Gauri, Muna Aghamelu, Alexander D. Goldie, Eryk Helenowski, Jean-Christophe Gagnon-Audet, Alberto Pepe, Saba Nazir, Daniel Izcovich, Noam Levi, Rishi Hazra, Karen Hambardzumyan, Nicolas Baldwin, Xian Li, Martin Josifoski, Paris Giampouras, Masoud Jalili Sabet, Anya Sims, Hela Momand, Tatiana Shavrina, Despoina Magka, Jason Weston, Yulin Wang, Anirudh Goyal, João Henriques, Yoram Bachrach, Emily McMilin, Jakob Nicolaus Foerster
Organizations: FAIR at Meta · University of Oxford · University College London
AI research agents (AIRA) can now carry machine learning experiments from proposal through implementation and evaluation. Yet progress on frontier tasks is throttled by the cost of evaluations that can consume days of GPU time. When an agent can propose far more candidates than it can afford to run, progress depends on its research preference: how it allocates a fixed execution budget across many candidates. We introduce AI Research Preference Models (RPMs) that predict which candidate solution is most promising, without paying the cost of running them all. We build RPMs from frozen pretrained language models in two variants: an inference-only model that reasons over candidate plans, code, and previously executed solutions, and an agentic model that additionally runs small-scale pilot experiments. Integrated into the AIRA-dojo research agent and evaluated on the machine learning research benchmark AIRS-Bench, the two variants increase the average normalized score from 0.684 to 0.711 and 0.729, respectively. Both reach the unguided agent's 24-hour performance in roughly 15 hours, using less than two-thirds of its execution budget, and together yield new state-of-the-art results on two AIRS-Bench tasks.
Figures & tables
Figure 1: Overview of an AI research agent with Research Preference Model (RPM) augmented child creation. Each node represents a solution within the search tree, labeled with its evaluation score. The agent (1) selects a promising parent node from the active tree, (2) generates a batch of candidate child solutions, (3) utilizes the RPM to select the most promising candidate, using the full context of previously executed solutions and (4) executes and scores only the selected candidate to expand the tree, to reduce the compute overhead of running unpromising candidates ( Section 3.3 ).
Figure 2: Evaluation of RPM-augmented AIRA-dojo. Average normalized scores over time (left) and final performance (right) for AIRA-dojo with RPM-augmented child selection on AIRS-Bench , with error bars indicating the 95% confidence intervals. Dashed lines indicate the No-RPM baseline and immediate validation or test oracles. The Inference-only RPM (orange) yields steady gains over the baseline (yellow). While per-step proxy experiments slow its early trajectory, the Agentic RPM (green) leverages this compute overhead to surpass other methods.
Figure 3: Offline evaluation of Inference-only RPM scaling properties. Increasing search-tree context provided gives improved predictive accuracy (left). Allocating a higher reasoning budget yields better selection (middle). Expanding the candidate suggestion pool reliably improves both the oracle-selection limit and the RPM’s selection performance (right), measured by the average difference between the selected solution’s score and the average score of all candidates.
Figure 4: What strategy the Agentic RPM uses. The Agentic RPM frequently runs simplified versions of the original candidates. Agents save time by running reduced cross-validation split counts (left) and sub-sampling the training data (middle). Both agents employ similar training adaptations, such as removing ensembling, reducing epochs and lowering the batch size.
Figure 5: How the Agentic RPM’s strategy affects performance. Increasing the time budget leads to better performance, but with diminishing returns (leftmost). Using more cross-fold validation splits does not correlate with better selection accuracy (middle left). Using full data brings an advantage over using sub-sampled data (middle right). Most training adaptations improve selection accuracy, except for the removal of the learning rate scheduler and simplification of the data augmentation approach (rightmost).
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Accuracy (%)
GPT-5
64.66
Claude Opus 4.8
67.44
Gemini 3.1 Pro
67.40
Ensembling (majority vote)
68.04
Ensembling (arbiter, Claude 4.8)
69.35
Appendix
Table 1: Selection accuracy of the Inference-only RPM across different LLM backbones and ensembling configurations
Figure 9: Reasoning category frequency and selection accuracy impact. Mention frequency of reasoning categories across reasoning traces (left) and resulting selection accuracy when mentioned versus omitted (right). Citing correctness, pretrained backbones, or prior evidence improves accuracy, whereas citing architecture alone degrades performance.
Figure 10: Context citation frequency and selection accuracy impact. Mention frequency of distinct context metrics across reasoning traces (left) and resulting selection accuracy across citation counts (right). Selection accuracy increases monotonically as more context metrics are cited.
Figure 11: The performance of three ablation variants at offline evaluation using GPT-5 (leftmost) and their average tool calling count of bash (middle left), python (middle right), submit_solution (rightmost). bash is seldom used by any variant. The w/o feedback variant calls python and submit_solution more frequently than other variants.
Figure 12: The average execution time distribution of pilot experiments across offline evaluations. Compared with w/o overstating variant (left) and w/o feedback variant (middle), the original Agentic RPM produces more time-consuming pilot experiments. Without GPU access (right), the average execution time is further increased.
Figure 16: Selection Advantage vs. Final Normalized Score. Final normalized score plotted against step-wise selection advantage across individual seed runs. Large diamonds denote mean values ( ±95% CI). Selection advantage strongly correlates with end-to-end performance (Pearson r=0.55,p=0.0007 ; Spearman ρ=0.56,p=0.0004 ), with Agentic RPM achieving both higher selection advantage and final score than Inference-only RPM and Random selection.
Figure 18: Final Node Selection performance across candidate pool sizes. Average normalized test score as a function of the top- N validation candidate pool size. Inference-only RPM judges (GPT-5 and Qwen3.6-27B) beat random selection but degrade as N increases, becoming progressively worse than the Validation Oracle (AIRA-dojo’s default selection strategy). The Inference-only RPM with the stronger GPT-5 LLM backbone outperforms that with the Qwen backbone.