When Compliance Data Masquerades as Evaluation: Measurement Validity for Deployed AI Systems
Organizations: Dept. of Computing Sciences Texas A&M University–Corpus Christi Corpus Christi, TX, USA · Dept. of Computer Science and Engineering University of California, Riverside Riverside, CA, USA · EECS Department University of Missouri Columbia, MO, USA
Abstract
We argue that a recurring failure in the evaluation of deployed AI systems occurs when data collected for operational monitoring or regulatory compliance are interpreted as if they were designed for comparative evaluation. Automated driving provides a concrete example of this problem. U.S. disengagement and crash-reporting regimes produce valuable operational evidence, but differences in reporting scope, exposure, deployment domain, event capture, and comparator construction limit the safety claims that can be supported from these measurements alone. We frame this issue as a measurement-validity problem in AI evaluation rather than as a transportation-specific data limitation. We argue that comparative claims about deployed AI systems require alignment between the intended capability, measured outcome, exposure opportunity, deployment domain, data-generation process, and evaluation comparator. Using automated-driving safety evaluation as a case study, we propose an evaluation contract that makes these assumptions explicit before operational data are interpreted as evidence of comparative performance. The broader implication is that data useful for monitoring deployed AI systems are not automatically valid benchmarks for evaluating them.
Figures & tables
| Regime | Measures | Cannot establish alone |
|---|---|---|
| California DMV | Testing mileage and disengagement events | Comparable system safety across operators |
| NHTSA SGO | Reported crash incidents | Exposure-normalized risk comparison |
| Retrospective studies | Outcome rates using exposure and benchmarks | General conclusions beyond matched domains |
| Failure mode | Evaluation contract requirement |
|---|---|
| Construct mismatch | Define the capability being evaluated and ensure that the measured outcome represents the intended construct. |
| Exposure mismatch | Specify the opportunity over which performance is measured, including an appropriate denominator or interaction population. |
| Domain mismatch | Define the deployment distribution and the scope over which the evaluation claim applies. |
| Capture mismatch | Document how observations are generated, classified, and made available for analysis. |
| Governance mismatch | Specify data access, comparator construction, transparency requirements, and evaluation independence. |
AI Safety Evaluation
AI Safety Evaluation studies how to test whether AI systems avoid harmful behavior and remain reliable across users, tasks, and deployment settings.
See all 143 papers