An AI audit record is useful only if its durability and trust boundary are explicit. Returning a guarded decision before any durable write minimizes latency, but it cannot guarantee that evidence survives an immediate crash. We rebuild RuntimeGuard-AI around this constraint. The resulting research prototype binds each deterministic policy decision to the exact policy source, commits a privacy-minimizing record at a caller-selected synchronization boundary, and returns an Ed25519-signed receipt that states whether that boundary completed. After restart, the engine validates framed records, manifests, shard placement, sequence continuity, and replay identity. A separate attestation path groups committed records into chained, signed Merkle epochs that an auditor verifies with an externally obtained key. On an Apple M4 Pro at four worker threads and 2,048-byte prompts, buffered signed evidence reaches 27,193 requests/s with 141.9 microseconds median latency. Per-record data and full synchronization reduce throughput to approximately 242 requests/s and raise median latency to 16.0 ms. Sealing a 100,000-record signed epoch takes 97.0 ms. The result is a measured durability-latency trade-off, not a "free" asynchronous audit path. The prototype does not prove model execution, prevent a compromised signer from forking history, or establish legal conformity.
Figures & tables
Figure 1: RuntimeGuard-AI V2 lifecycle. The synchronous path returns only after its selected host synchronization boundary. Epoch construction and audit verification operate on committed records outside that path.
Figure 2: Commit and recovery states. No data- or full-sync request reaches Acknowledged before a complete frame and its configured synchronization operation.
Mode
p50
p99
Throughput
Policy only
0.938
3.146
1,717,597.126
Buffered evidence
141.875
238.145
27,192.808
Data sync
16,018.646
33,963.302
240.123
Full sync
16,009.136
29,700.826
244.416
Table 1: Inline component results at 4 threads and 2,048-byte prompts. Latency is in microseconds; throughput is requests/s.
Figure 3: Inline latency and throughput from canonical raw observations. Log axes expose the orders-of-magnitude separation between policy evaluation, buffered evidence, and per-record synchronization.
Figure 4: Canonical epoch-operation and recovery scaling. Every plotted value is derived from manifest-verified raw CSV files.
Runtime-governance evidence often collapses materially different events into one audit record: a decision may be made, an instruction dispatched or received, an action may or may not execute, and a resulting state may or may not be observed. This paper presents AIREP, a vendor- and model-independent protocol for per-decision AI runtime evidence. Its current wire model separates evidence into four artifact families: Decision, Control, Execution, and Effect. Artifacts use closed core schemas, explicit identities and digests, declared scope limits, RFC 8785 canonical JSON, domain-separated SHA-256 hashing, and pure Ed25519 signatures. A three-level assurance model distinguishes structural/hash consistency (AIREP-Core), verifier-accepted authorship (AIREP-Authenticated), and independently anchored chain-head freshness and non-truncation relative to an accepted witness (AIREP-Witnessed); these classes do not establish event truth. A structured reconciler preserves failure, missing evidence, unevaluated prerequisites, and indeterminate outcomes as distinct states. The released beta includes a four-family first-party producer, Python and Node reference-verification paths, adversarial/lifecycle corpora, and reproducible validation. Post-release first-party Hermes and LightEval integration exercises preserve explicit evidence boundaries without claiming adoption or interoperability. Independent implementation evidence exists separately for a v0.1.2 producer and a v0.2 consumer/verifier; because they target different frozen versions, they do not establish same-version producer-to-consumer interoperability. AIREP remains experimental.
AI systems are increasingly used to assist consequential decisions in regulated domains such as auditing, finance, and healthcare. This creates a recurring obligation: an organization must be able to reconstruct, after the fact, which evidence informed a given conclusion, and to show that the record of that reasoning was not altered. Existing tools address related but distinct problems - model observability, drift monitoring, governance reporting - and are built for the machine-learning engineer operating a system, not the reviewer who must trace one specific conclusion back to its supporting evidence. We present AuditWeave, a lightweight Python library, with no runtime dependencies, that records the steps of AI-assisted and data-transformation workflows into a single append-only, hash-chained ledger. A small, system-agnostic event vocabulary spans both retrieval-augmented generation (RAG) pipelines and tabular/lakehouse transformations, so a conclusion that draws on both can be traced end-to-end through one record. Within a sealed ledger, any modification, reordering, insertion, or deletion of events is detectable through chain verification. We describe the design and evaluate recording overhead, scalability, and tamper-detection correctness on the reference implementation. The integrity guarantees cost tens of microseconds per event, and, as the hash-chain construction implies, verification flagged every injected mutation across four mutation classes over 2,000 randomized trials.
Long-horizon agents execute thousands of actions, resulting in sequential failures rather than isolated errors. When a coding agent deletes a production database or a prompt injection spreads across agents, the incident raises questions of causality, authority, and non-repudiable third-party verification. The Agent Flight Recorder captures each agent action as a structured, canonically serialized event binding eight semantic fields from intent through execution to provenance. Hash chaining and Merkle batching provide tamper evidence and compact inclusion proofs. For cross-organizational disputes where no party's infrastructure qualifies as neutral ground, periodic on-chain anchoring of epoch roots lets any verifier with the disclosed payload and Merkle proof check the record independently, without pre-agreeing on a trusted intermediary. The on-chain footprint is minimal: each anchor stores a 32-byte epoch root and a back-pointer, and no event content touches the chain. We evaluate the system across five cumulative ablation configurations on synthetic agent workloads. The full system adds ~48 microseconds median per-event latency and 512 bytes per event. L2 anchoring costs $2.30 per 100K events at 100-event epochs. The full integrity stack detects edit, delete, reorder, and fork tampering at 100% with zero false positives. Structured forensic queries achieve 1.0 precision on guardrail and delegation lookups where unstructured text search yields 0.013 and 0.077 respectively.
Laurent Bindschaedler, Quentin Botha, Christoph Siebenbrunner
Max Planck Institute for Software Systems · Research Institute for Cryptoeconomics, Vienna University of Economics and Business