FCP: Formal Context Protocol for Verifying LLM-Generated Artifacts
Abstract
When a language model writes both an artifact and the tests meant to check it, a passing test is weak evidence: one misreading of the request yields a faulty artifact and a check that accepts it. We present the Formal Context Protocol (FCP), which checks AI-generated artifacts against the request itself. FCP separates conformance (does the artifact satisfy a formal reading of a requirement?), which model checkers and SMT solvers decide, from fidelity (does the reading mean what the prose means?), which has no decision procedure but is refutable by a concrete witness. Eight ways a proof can falsely certify a requirement each become a mechanical check that returns a witness, and what no machine can settle becomes concrete questions for the request's author. Adapters place hardware, text, agents and code on the same engines. On four hardware designs, trusting AI-written assertions certified 23% of violated requirements, and FCP certified none. On ten code tasks, code from three language models passed its own tests and violated the request every time. Trusting the tests certified all 33 violations, and FCP certified three. Agreement among models, readings or tests did not imply correctness.
Figures & tables
| Situation in which a proof of certifies falsely | Mechanical witness returned by FCP | |
|---|---|---|
| V | never triggers on under , or holds on every behavior that allows | proof that the trigger is unreachable; proof of on (tautology) |
| E | excludes behaviors of itself (assumptions may restrict only the environment) | reachable state with no legal non-reset input (non-receptiveness) |
| F | differs from the meaning of | behavior accepted by one independent reading and rejected by another, in the context of the other requirements |
| C | the requirements contradict each other | input sequence on which no output sequence meets every guarantee (unrealizability) |
| U | the request does not determine some output | two conforming behaviors with equal history that differ in an output |
| W | the properties are too weak to notice a change of | observable mutant of that no property kills |
| Level | Meaning and attached evidence |
|---|---|
| ASSURED | proved by two engines; trigger witnessed; assumptions receptive; an independent reading is an equivalent contract (in context, § 4 ) |
| PROVED-DISPUTED | proved, and an independent reading differs; the simplest distinguishing behavior is attached |
| PROVED-CONFLICTED | proved, and the requirement belongs to a minimal conflict; the conflicting stimulus is attached |
| VACUOUS | trigger unreachable under the assumptions (proof attached), or the property is a tautology |
| ENV-UNSOUND | the assumptions are not receptive; a blocking state is attached |
| FAILED | counterexample replayed in simulation and triaged to an artifact deviation or a conflict |
| Domain | Tasks | Artifacts checked | Readings |
|---|---|---|---|
| Hardware | 4 IP specifications; 13 planted defects | reference RTL, 28 single-edit RTL defects, 4 AI-generated designs | gold; 2 Sonnet per design |
| Prompt | 1 launch email, 10 instructions | 3 responses (Haiku, Sonnet, Opus); an LLM judge as baseline | gold; 2 Opus |
| Agent | 1 returns desk, 7 tools, 11 rules | 10 scaffolds; 18 runs of 3 models | gold; 2 Opus |
| Code | 10 tasks; 10 conflicts, 23 ambiguities | per task: 3 model implementations, a reference, 8 planted bugs; a refactoring in 9 tasks | gold; 2 Opus; 21 variants |
| Self-check : AI writes assertions, the prover reports PASS | FCP | |
|---|---|---|
| Violated requirements certified as holding | 56 / 244 (23%) | 0 / 244 |
| Injected formalization faults that certify a buggy design | 71 / 161 | 0 / 161 , all 161 flagged |
| AI formalizations equivalent to intent (all well-formed and lint-clean) | 44 / 98 (45%) | n/a (FCP does not consult gold) |
| Disagreements between the two AI formalizers that were real errors | n/a | 45 / 45 |
| Injected RTL defects detected (gold readings) | 28 / 28 | 28 / 28, all replayed |
| Acceptance rule for a requirement | Accepted on reference RTL (readings) | False assurance on buggy RTL |
|---|---|---|
| Self-check: the formalizer’s reading is proved | 92 / 104 | 56 / 244 |
| All readings proved, with no comparison | 77 / 104 | 10 / 244 |
| Equivalent in a minimal sufficient context, and that context proved | 64 / 104 | 5 / 244 |
| Equivalent in the full context, and every other requirement proved under both readings | 31 / 104 | 0 / 244 |
| Strict: equivalent without context | 7 / 104 | 0 / 244 |
| Code | Own tests | Violates (all readings) | First failing call (replayed in a fresh process) |
| Haiku | 134 pass | R-SHIP-1 | checkout(5000,0,False) (5000, 498) |
| Sonnet | 196 pass, 1 xfail | R-MONO-1 | checkout(4999,0,False) (4999, 499), then checkout(5000,0,False) (5000, 0) |
| Opus | 745 pass | R-SHIP-1 | checkout(4998,0,False) (4998, 2) |
| Planted bugs flagged in the planted requirement: 7 of 7 (C1 to C5, C7, C8). C6 (float round() , half a cent to even) is correctly not flagged: every reading allows either neighbor. | |||
| False assurance on the models’ code: the self-check baseline (own tests pass) certifies 3 of 3 violations, FCP none. | |||
| Task | Adapter | Conflict | Before code | Choice | Agree on | Own tests | Bugs | Certify |
|---|---|---|---|---|---|---|---|---|
| Card masking | strings | last four digits vs. at most a quarter | 92 s, 1 call | a quarter: 3 digits shown | every call | 38, 146, 119 b | 8/8 | 3/3, 0 |
| Rate limiter | object | under the limit vs. 10 s penalty | 9 s, 5 calls | penalty | 175,528 calls | 48, 43, 43 b | 7/8 | 3/3, 0 |
| Student pricing | change | $75 shipping vs. no change for non-students | 29 s, 1 call | $50 for non-students | 3.6 M calls | 22, 29, 45 | 8/8 | 3/3, 0 |
| Payment retry | effects: service | keep trying vs. 3 attempts | 67 s, 3 errors | 3 attempts | 50,000 runs | 21, 36, 105 | 8/8 | 3/3, 0 |
| Atomic save | effects: files | no temporary file vs. keep it on failure | 18 s, 1 failed move | keep the file | see a | 18, 37, 33 b | 8/8 | 3/3, 0 |
| Invoice lines | records | line rounding vs. order rounding | 21 s, 2 items | discount summed per line | every call | 19, 35, 23 b | 8/8 | 3/3, 0 |
| Task | Scope | Pairs translated | Verdicts | Proved | Fail in both | Missed | s per pair |
|---|---|---|---|---|---|---|---|
| Checkout | 1,355,700 calls | 30 / 36 | 260 | 198 | 62 | 0 | 6 to 7 |
| Card masking | 131,613 calls | 36 / 39 | 240 | 161 | 75 | 4 | 6 to 97 |
| Rate limiter | 175,528 calls | 39 / 39 | 208 | 58 + 7 | 140 | 3 | 28 to 734 |
| Student pricing | 3,601,728 calls | 36 / 39 | 204 | 139 | 65 | 0 | 24 to 27 |
| Payment retry | 50,000 runs | 39 / 39 | 364 | 293 | 71 | 0 | 1 to 2 |
| Atomic save | 1,441 runs | 36 / 39 | 276 | 208 | 68 | 0 | 0.3 to 0.5 |
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Construct | Meaning |
|---|---|
| Types | Booleans; bit-vectors; integers with statically computed ranges; bounded sequences (strings, lists) of at most slots |
| Expressions | exact arithmetic, comparisons, Boolean connectives on one-bit operands, conditionals, , , , ASCII character tests, over with |
| Past time | , disabled before the current reset segment; , , |
| Patterns | invariant , implies , next , within , hold_until , bounded_wait , inorder , after_reset ; each monitor flags the minimal bad prefix |
| State and quantifiers | ghost variables with reset values and update rules; rigid variables, universally quantified over a range; named definitions |
| Item roles | property (guarantee), assumption, scenario (cover), don’t-care; each item names the requirement it formalizes |
| Design | Spec lines | RTL lines | Gold reqs. | RTL defects | Request defects |
|---|---|---|---|---|---|
| SFIFO-01 synchronous FIFO (depth 5) | 86 | 53 | 21 | 8 | 4 |
| RRA-4 round-robin arbiter | 53 | 44 | 12 | 7 | 4 |
| BRG-1 command-to-APB bridge | 84 | 61 | 21 | 8 | 3 |
| HUB-2 two-port APB hub (with BRG-1) | 45 | 124 | 13 | 5 | 2 |
| Fault operator | Faults | Self-check FA | FCP FA | Flagged by |
|---|---|---|---|---|
| placeholder consequent ( … || true ) | 50 | 30 | 0 | vacuity or tautology, peer |
| reset-polarity error in the trigger | 27 | 18 | 0 | vacuity, peer |
| masking assumption ( assume !trigger ) | 27 | 20 | 0 | vacuity, peer, receptiveness |
| dropped conjunct | 17 | 2 | 0 | peer, vacuity |
| next cycle replaced by same cycle | 25 | 1 | 0 | failure on reference, peer |
| bound one cycle too loose | 11 | 0 | 0 | peer |
| Scaffold | ID1 | ID2 | ELIG | CONF | TOT | BIG | ONCE | HUM | END | frame |
|---|---|---|---|---|---|---|---|---|---|---|
| guarded kernel | E | E | E | E | E | E | E | E | B | ok |
| ReAct loop | D | D | D | D | D | D | D | D | D | ok |
| G1 confirmation not tied to the order | E | E | E | D | E | E | E | E | B | ok |
| G2 amount not checked against it | E | E | E | D | E | E | E | E | B | ok |
| G3 total from any order’s lookup | E | E | E | E | D | E | E | E | B | ok |
| G4 handoff after the third failure | E | D | E | E | E | E | E | E | B | ok |
| Task | Rules | Planted conflict (number of planted ambiguities) |
|---|---|---|
| Checkout | 7 | free shipping from $50 vs. the charge never drops as items are added (4) |
| Card masking | 4 | keep the last four digits vs. never show more than a quarter, for 12 to 15 digits (2) |
| Rate limiter | 5 | allow below 3 attempts in 60 s vs. deny for 10 s after a denial (3) |
| Student pricing | 4 | free shipping from $75 vs. nothing changes for non-students (2) |
| Payment retry | 7 | keep trying until the charge succeeds vs. at most 3 attempts (2) |
| Atomic save | 7 | leave no temporary file vs. keep it if os.replace fails (2) |
LLM Reliability
Large language model reliability studies whether outputs remain correct, calibrated, robust, and controllable across prompts, contexts, and deployment conditions.
See all 866 papers