Reliability is a user-specific property of large language models
Organizations: Wuhan University, Wuhan, China. · University of Warwick, Coventry, United Kingdom.
Abstract
Public benchmarks support comparisons of large language model (LLM) capabilities, yet users can experience the same highly ranked model differently. Drawing on traditional software reliability, we argue that the reliability of an LLM service should be assessed for specified users and conditions of use. This requires more than a capability score: evaluation evidence must support a claim about failure during future operation. We distinguish testing to discover faults from testing to assess reliability and identify ten modelling considerations behind such claims. We propose a six-workstream research roadmap linking claim specification, dynamic operational profiles, graded outcomes and noisy judgements, local probabilistic robustness, interaction dependence, and statistical inference with reassessment. Local probabilistic robustness addresses the behavioural variation hidden when different concrete inputs are treated as the same task. Published findings and an illustrative customer-support example motivate the roadmap; its integrated validation remains an open research task. Reliability claims complement, but do not replace, separate safety and security arguments. Capability scores should therefore be complemented by versioned operational claims that state whose requirements are covered, under which conditions, and with what uncertainty.
Figures & tables
| Dimension | Coin toss | IEC 61508 safety software | DL classifier | Autonomous vehicle | LLM service |
|---|---|---|---|---|---|
| Claim / metric | Probability of heads; probability of all heads in the next tosses | PFD; probability of failure-free operation over the next demands | Misclassification probability per input | Crash, fatality or disengagement probability per scenario or trip; event rate per mile or hour | Failure probability or expected severity per user task or conversation; specified-event rate per response, token or hour |
| Statistical unit / exposure | One toss | One demand on the safety function | One prediction on a sampled input | One scenario or trip; or continuous distance or time | One bounded task, possibly multiround; or cumulative exposure such as service time or number of tokens |
| Failure / severity | Tails when heads are required; binary | Failure to perform the specified safety function; consequences assessed through hazard analysis | Invalid class or decision; binary, class-weighted or loss-valued | Crash, unsafe manoeuvre, near miss or disengagement; consequence-based severity | Hallucination, unsafe response, privacy loss or unresolved task; ordinal or continuous severity, potentially user-dependent |
| Oracle / observation | Direct physical observation | Specification, plant state, logs and test oracle | Reference or expert label and task rule; labels may be wrong | Telemetry, event reconstruction, safety rules and review | Rules or human/model judgement of recorded evidence; disagreement and systematic bias are possible |
| Stochastic process | Bernoulli sequence | Fixed-version Bernoulli demands; a state-dependent sequence if reset is incomplete | Binary failure or categorical outcomes | Bernoulli by scenario or trip; Poisson, NHPP, survival or recurrent-event models by exposure | Binary, ordinal or loss-valued task outcomes; hierarchical, hidden-state or recurrent-event models for interaction |
| Dimension | Coin toss | IEC 61508 safety software | DL classifier | Autonomous vehicle | LLM service |
|---|---|---|---|---|---|
| i.i.d. / dependence | Plausible if the mechanism is stable and tosses reset | Independent draws from a stable OP, fixed configuration and specified resets; otherwise model dependence | Duplicates and user/acquisition clusters may induce dependence; class-mix shift affects representativeness | Adjacent miles, routes, weather, scenarios and versions can be correlated | Turns, users, task families, retrieval snapshots, context, routing and memory may induce clusters and state dependence |
| Operational profile | Known trial mechanism | Demand distribution justified for the plant and mission | Deployment input distribution; a public dataset is only a proxy, with sparse coverage of high-dimensional regions | ODD plus probabilities over scenarios, routes, weather, traffic and driver behaviour | Dynamic distribution over users, intents, languages, histories, contexts, retrieval and serving conditions |
| Local probabilistic robustness | Normally not applicable | Input tolerances and neighbouring states addressed through requirements and robustness analysis | Label-preserving perturbations around an input | Nearby scenario parameters, trajectories, sensors and behaviours | Conditional model for task-preserving wording, format, language, context order and valid retrieval variation within one modelled demand |
| Estimator / uncertainty / evidence | MLE or Bayesian estimate; exact or asymptotic interval | Rare-event bounds, Bayesian or conservative inference, stopping rules and non-test assurance evidence | Weighted or stratified estimate; sampling, label, shift and calibration uncertainty | Rate, survival or Bayesian model; exposure, reporting, simulation and transfer uncertainty | OP-weighted and hierarchical inference; sampling, oracle, output, dependence, , rare-event and drift uncertainty |
| Version / claim lifetime | Stability is assumed | Bound to software, hardware, plant state and demand profile | Model, preprocessing, labels and user distribution can change | Full stack, maps, ODD, traffic, weather and reporting evolve | Model, prompt, decoding, retrieval, filters and routing can drift; judge and OP changes also affect the assessment |
| Assessment element | Illustrative specification |
|---|---|
| Users and service | Customer population using billing-support service , which retrieves approved information and recommends escalation but cannot change accounts. |
| One demand | One billing-dispute conversation, ending with an answer, escalation, abandonment or a predefined interaction limit. The rubric specifies how each endpoint is scored. |
| Failure and severity | Severity-3-or-higher failure: materially incorrect advice, private-account disclosure or falsely reporting resolution, assessed against explicit severity anchors. |
| OP and local variation | gives demand frequencies; gives frequencies of valid wording, formatting and context variations within a demand. Both need operational justification. |
| Measurement | Expert-audited cases calibrate the rules or model judge. Judge error and rater differences enter the reliability assessment. |
| Dependence and reset | Reset conversation context between tasks. Record shared customer, incident, retrieval and serving conditions, and assess any remaining outcome dependence. |
LLM Reliability
Large language model reliability studies whether outputs remain correct, calibrated, robust, and controllable across prompts, contexts, and deployment conditions.
See all 866 papers