SNAPPerf: Software-Aware Tabular Foundation Model for Retraining-Free Performance Prediction of Configurable Systems
Organizations: National Institute of Standards and Technology · Université Grenoble Alpes, CNRS
Abstract
Modern software systems expose hundreds of configuration options whose combinations can substantially affect execution time, memory and energy usage. Measuring software performance resulting from a configuration set requires running a realistic workload. Since the configuration space grows exponentially, exhaustive benchmarking is infeasible. Prior work therefore trains models on a small set of measured configurations to predict the performance of the rest. Yet these models are specific to a particular software version, hardware platform, and workload, so changes require new measurements and retraining, and such changes are becoming more frequent with coding agents. We challenge this per-system training assumption with SNAPPERF (Software-aware, No-retraining, Adaptive Prediction), a sample-efficient tabular foundation model specialized for configurable software. SNAPPERF is pretrained once on synthetic data. Each new update only requires a few measured configurations as context to predict the rest without any per-system training or hyperparameter tuning. Across 39,866 configurations from 12 real-world systems, spanning 6 domains and 4 performance metrics, SNAPPERF reduces prediction error by 38% over the prior state of the art at the same measurement budget, and by 25% over the best general-purpose tabular foundation model. It matches the state of the art's accuracy with 54% fewer measurements on average. These results suggest that knowledge about software can be encoded in pretraining data rather than in per-system models, opening a new research direction in software-aware pretraining.
Figures & tables
| Quality mode | Threads | Constant bitrate | … | Encoding time |
| best | 1 | no | … | 60.8 s |
| good | 2 | yes | … | 22.3 s |
| real-time | 4 | no | … | 5.7 s |
| … 121 measured of 2,736 configurations | ||||
| best | 4 | yes | … | ? |
| good | 1 | no | … | ? |
| Method | Median MRE | Avg. rank | Top group (of 60) |
| Specialized foundation model [(ours)], no per-system training | |||
| SNAPPerf | 2.80 | 1.08 | 55 |
| General-purpose foundation models, no per-system training | |||
| Nori | 3.80 | 2.43 | 0 9 |
| TabICL | 4.43 | 2.72 | 0 8 |
| TabPFN | 4.13 | 3.15 | 0 4 |
Tabular Foundation Models
Tabular foundation models learn reusable representations or in-context prediction skills for structured data such as spreadsheets and databases.
See all 209 papers