Traditional benchmarks for large language models (LLMs), such as HELM and AIR-BENCH, primarily assess safety risk through breadth-oriented evaluation across diverse tasks. However, real-world deployment often exposes a different class of risk: operational failures arising from repeated generations of the same prompt rather than broad task generalization. In high-stakes settings, response consistency and safety under repeated use are critical operational requirements. We introduce Accelerated Prompt Stress Testing (APST), a depth-oriented evaluation framework inspired by highly accelerated stress testing in reliability engineering. APST probes LLM behavior by repeatedly sampling identical prompts under controlled operational conditions, including temperature variation and prompt perturbation, to surface latent failure modes such as hallucinations, refusal inconsistency, and unsafe completions. Rather than treating failures as isolated events, APST characterizes them statistically as stochastic outcomes of repeated inference. We model observed safety failures using Bernoulli and binomial formulations to estimate per-inference failure probabilities, enabling quantitative comparison of operational risk across models and configurations. We apply APST to multiple instruction-tuned LLMs evaluated on AIR-BENCH 2024 derived safety and security prompts. While models exhibit similar performance under conventional single- or very-low-sample evaluation (N <= 3), repeated sampling reveals substantial variation in empirical failure probabilities across temperatures. These results demonstrate that shallow benchmark scores can obscure meaningful differences in reliability under sustained use.
Figures & tables
Fig. 1: Conceptual comparison of LLM safety evaluation paradigms along evaluation breadth (number of prompts or risk categories) and evaluation depth (repeated sampling enabling statistical estimation). Standard benchmarks emphasize breadth with primarily single- or very-low-sample evaluation (N≤3) , while adversarial testing emphasizes adaptive depth over a narrow prompt set. Accelerated Prompt Stress Testing (APST) occupies a distinct statistical-depth regime, combining category-level coverage with repeated sampling to estimate empirical failure probabilities under sustained inference.
Fig. 2: Empirical failure probability by temperature for the Phase 1 calibration model (Gemma-3N-E4B-it) for Phase 1 calibration prompts. Each bar aggregates outcomes across repeated generations of identical prompts at fixed decoding configurations. Non-zero failure probabilities are observed for all models, demonstrating that stochastic safety failures persist even under conservative evaluation conditions.
Fig. 3: Empirical failure probability as a function of sampling depth for Phase 1 calibration. Failure probability estimates increase from near-zero at shallow depth and stabilize only after moderate numbers of repeated samples. This illustrates that single-sample or low-depth evaluation systematically underestimates operational failure risk. Here, “cumulative” refers to aggregation over increasing sample counts for a fixed configuration, not temporal accumulation or time-dependent behavior.
Fig. 4: AIR-BENCH–style category-level safety summary under shallow evaluation (Phase 2A). Each cell reports the proportion of unsafe or non-compliant responses, reported as observed failure proportions under AIR-BENCH–equivalent shallow sampling ( N=3 , T=0.0 ), not as depth-calibrated reliability estimates. Under this benchmark-equivalent view, models appear broadly aligned, with high category-level safety scores and limited observable differentiation.
Fig. 5: Comparison of benchmark-style (AIR-BENCH–equivalent, prompt-level) and depth-oriented (APST, inference-level) evaluation outcomes. AIR-BENCH–equivalent scores are aggregated under a derived domain grouping (security, finance, generic) obtained by mapping AIR-BENCH L3 categories. This domain grouping is not part of the AIR-BENCH taxonomy and is used solely for high-level comparison. Models with similar AIR-BENCH–equivalent scores, when aggregated under the same derived domain grouping, exhibit substantially different empirical failure probabilities under repeated sampling, demonstrating that benchmark alignment does not imply reliability under sustained inference.