cs.ARNov 15, 2026

FCP: Formal Context Protocol for Verifying LLM-Generated Artifacts

Authors: Shashank

Abstract

When a language model writes both an artifact and the tests meant to check it, a passing test is weak evidence: one misreading of the request yields a faulty artifact and a check that accepts it. We present the Formal Context Protocol (FCP), which checks AI-generated artifacts against the request itself. FCP separates conformance (does the artifact satisfy a formal reading of a requirement?), which model checkers and SMT solvers decide, from fidelity (does the reading mean what the prose means?), which has no decision procedure but is refutable by a concrete witness. Eight ways a proof can falsely certify a requirement each become a mechanical check that returns a witness, and what no machine can settle becomes concrete questions for the request's author. Adapters place hardware, text, agents and code on the same engines. On four hardware designs, trusting AI-written assertions certified 23% of violated requirements, and FCP certified none. On ten code tasks, code from three language models passed its own tests and violated the request every time. Trusting the tests certified all 33 violations, and FCP certified three. Agreement among models, readings or tests did not imply correctness.

Figures & tables

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. MCPGen: Benchmarking LLMs on Executable MCPWorkflow Development

    Sep 20, 2026Yingxuan Yang, Jiaqi Liu, Lirui Guan +4LLM EvaluationLLM Code Generation

  2. Judging Is Not Enumerating: Silent Omissions in LLM-Authored Acceptable Sets

    Aug 2, 2026Wenhui Chen, Jianlin Chen, Ziyao Lin +2LLM EvaluationLLM-as-a-Judge

  3. The Price of an Explanation: What Demanding Verifiable Artifacts Costs, and What It Buys Back

    Nov 19, 2026Edgar A. Duéñez-Guzmán, Eduardo Dueñez, Edgar Said Hernandez Sanchez +1Program SynthesisLanguage Model Reasoning Evaluation