LAAF: A Layered Accountability Architecture Framework for LLM Applications

Prachi Chaturvedi; Shahnawaz Ahmad; Ehsan Nowroozi; Muhammad Waqas; George Loukas; Alireza Jolfaei; Lucas Cordeiro; Pierre Dantas

cs.AIAug 27, 2026

LAAF: A Layered Accountability Architecture Framework for LLM Applications

Authors: Prachi Chaturvedi, Shahnawaz Ahmad, Ehsan Nowroozi, Muhammad Waqas, George Loukas, Alireza Jolfaei, Lucas Cordeiro, Pierre Dantas

Organizations: School of Computer Science Engineering and Technology, Bennett University, Greater Noida, India · Centre for Sustainable Cyber Security (CS2), University of Greenwich, United Kingdom · School of Electrical Engineering, Computing & Mathematical Sciences, Curtin University, Australia · Systems and Software Security (S3), University of Manchester, United Kingdom

Abstract

Large Language Models (LLMs) operate in hospitals, courtrooms, banks, and public service desks, where fluent, confident outputs are treated as authoritative even when ungrounded or incorrect. When such an output contributes to harm, who is answerable, and through what mechanisms can responsibility be traced, explained, and acted upon? Following PRISMA guidance, five databases were searched from January 2022 to March 2026 against four review questions; of 4,512 records identified, 122 primary studies were included, together with 12 regulatory and standards documents analysed as primary sources. The review consolidates a sociotechnical account of accountability as an actor-forum relation resolved into five dimensions, and synthesises mechanisms across four families: technical controls, human oversight, organisational governance, and documentation and traceability, each with a maturity assessment. The corpus is read through a four-layer classification device spanning provenance, application logic, human oversight, and governance and redress, cross-cut by traceability, role clarity, and continuous monitoring. Both are mapped onto the EU AI Act, whose high-risk obligations have applied since 2 August 2026, the NIST AI RMF with its Generative AI Profile, ISO/IEC 42001, and sectoral guidance in healthcare, consumer finance, education, and the public sector. Four persistent gaps emerge: under-specification of human oversight, absence of shared accountability metrics, disciplinary disconnection, and limited empirical evaluation, alongside five structural tensions that no surveyed instrument resolves. The review closes by consolidating the classification device into an integrated accountability architecture, LAAF, with cybersecurity aligned to the OWASP LLM Top 10 (2025); it is a synthesis of the surveyed evidence rather than a validated artefact.

Figures & tables

Figure 1: Evolution of Large Language Models, 2017–2026, in five eras: foundation (2017–2018), scaling (2019–2020), scaling breakthroughs (2021–2022), efficiency and democratisation (2022–2024), and multimodal and reasoning specialisation (2024–2026). The periodisation is the authors’ construction, derived from the release history of the models named.

Figure 2: Taxonomy of LLM harms grouped by the lifecycle stage at which they originate. Arrows indicate that the phases are interdependent, an unaddressed harm at one stage resurfacing at the next; harms accumulate across the lifecycle, which is why accountability must be a lifecycle property.

Figure 3: Emergence of hallucinations in LLM pipelines. Absent grounding, logical verification, and temporal consistency checking permit unguarded probabilistic decoding to produce the five recurring categories of hallucinated output.

Figure 4: Hallucination-detection decision tree. Branching on two conditions, whether external reference material is available and whether the deployer has white-box access to internal model states, yields four method families: black-box reference-free, white-box internal-state, retrieval alignment, and retrieval combined with an internal check.

Actor type	Actor	Contribution	Accountability surface	Answerable at
Internal	Foundation-model provider	Training data, base weights, alignment, release	Model card, evaluation reports, systemic-risk disclosures	L1
Internal	Fine-tuning party	Domain adaptation, safety patches	Fine-tuning records, drift documentation	L1
Internal	Application developer	Prompt design, retrieval, tools, guardrails	Application logs, design rationale, retrieval provenance	L2
Internal	Deploying organisation	Policy, roles, review obligations, override authority	Governance policy, role allocation, audit trails	L2-L4
Internal	Domain professional	Prompt phrasing, interpretation, professional judgement	Review records, sign-off, duty of care	L3
External	End user or affected party	Reliance on output, downstream action	Terms of use, reported harms, complaint and contestation records	Forum, not a layer

Table 1: Actors, contributions, and accountability surfaces in LLM deployments. Identifies each actor’s contribution and the artefact through which they can be held answerable; the takeaway is that accountability is a relation across actors, not a property of the model. The regulator is listed as an external actor whose entry in the final column records the instruments through which it holds others answerable, rather than an artefact for which it is itself answerable. The final column names the LAAF layer (Section 9.2) at which each actor becomes answerable; the affected party and the regulator are forums to which the stack answers, not positions within it.

Figure 5: PRISMA 2020 flow of the review process, reporting exclusions at each stage, and reconciling with the per-database counts of Table 4 .

Ref.	Focus	Concept	Mech.	Reg.	Sector	Cyber	Differentiation from this survey
[ 43 ]	Hallucination taxonomy	P	Y	N	N	N	We integrate hallucination into an accountability framework
[ 76 ]	Accountability concept	Y	N	N	N	N	We add mechanism synthesis + LAAF construction
[ 112 ]	Attributability gap	Y	N	N	N	N	We extend to LLMs with regulatory mapping
[ 70 ]	LLM auditing (3-layer)	P	Y	P	N	N	LAAF covers full lifecycle incl. cybersecurity
[ 34 ]	EU AI Act analysis	P	N	Y	P	P	We bridge legal and technical perspectives
[ 90 ]	Risk governance for AI	P	P	P	N	N	We update to LLM context with GenAI Profile

Table 2: Comparative analysis of related surveys and contributions. Each prior work is scored on five comparison criteria: Concept, Mechanisms, Regulatory mapping, Sector cases, Cybersecurity, plus its differentiation; the dense pattern demonstrates the integration gap. (Y = covered, P = partial, N = absent), Y denotes that the work treats the criterion as a structured component with a dedicated section, table, or framework element; P that it is discussed without being developed; N that it is absent.

Dataset / resource	Domain	Scale	Application
HaluEval [ 58 ]	Hallucination detection	35,000 annotated samples	Evaluating detection methods
TruthfulQA [ 61 ]	Factuality	817 questions, 38 categories	Truthful generation
FactScore [ 66 ]	Long-form factuality	Atomic-fact annotations	Fine-grained factuality
RAGTruth [ 74 ]	RAG hallucination	18,000 word-level annotations	Retrieval-augmented evaluation
BOLD [ 19 ]	Fairness	23,679 prompts	Open-ended fairness
HolisticBias [ 94 ]	Bias coverage	600 descriptors, 13 axes	Bias measurement / audit

Table 3: Benchmark datasets and resources for accountability evaluation. Catalogues the principal public resources and their accountability use; the takeaway is that current benchmarks measure outputs, not accountability mechanisms.

Stage	ACM	IEEE	ScienceDirect	Springer	Wiley	Total
Identified (keyword)	612	1,847	1,108	832	113	4,512
After duplicate removal	553	1,648	989	565	70	3,825
After title screening	187	432	261	174	24	1,078
After abstract screening	92	248	178	109	15	642
After intro/conclusion review	45	121	89	64	7	326
Final inclusion	24	41	32	22	3	122

Table 4: Selection and screening results from digital libraries. The per-database PRISMA cascade; the takeaway is the progressive elimination from 4,512 to 122, reconciling exactly with Figure 5.

Figure 6: Conceptual structure of accountability in LLM applications. An actor is answerable to a forum, is judged against standards, explains conduct through a defined process, and faces implications; the relation holds only when all five elements are present.

Dimension	Central question	Failure mode when missing
Disclosure and justification	Can actors explain and justify decisions?	Disclosure without consequence
Oversight	Can humans meaningfully review and override?	Ceremonial sign-off, automation bias
Organisational / governance	Are roles and pathways embedded institutionally?	Aspiration without operationalisation
Moral/professional	Are human duties of care preserved?	Responsibility displaced onto the model
Enforcement/redress	Are failures investigated and sanctioned?	Rhetorical accountability without consequence

Table 5: Five dimensions of accountability and failure modes. States the central question each dimension answers and what breaks when it is missing; the takeaway is that accountability emerges only from the interaction of all five.

Figure 7: Taxonomy of accountability mechanisms for LLM applications: four families, each subdividing into specific mechanisms; no family suffices alone.

Family	Specific mechanisms	Primary contribution	Maturity	Regulatory anchor
Technical controls	RAG, guardrails, uncertainty/abstention, watermarking, logging	Attach outputs to evidence; mark/contain risk	RAG & logging high; guardrails/UQ medium; watermarking emerging	EU AI Act Arts. 11–12, 15; ISO/IEC 42001
Human oversight	In-the-loop, on-the-loop, escalation, override	Place substantive judgement in the path	Medium (mostly simulated)	EU AI Act Art. 14; NIST Manage
Organisational	Roles, committees, audit/red team, incident response, management systems	Embed answerability institutionally	Medium	EU AI Act Arts. 9, 17, 27–73; ISO/IEC 42001
Documentation	Model/system cards, datasheets, evaluation reports, output provenance	Make decisions investigable post-hoc	Cards high; output provenance emerging	EU AI Act Annex IV; NIST Map/Measure

Table 6: Accountability mechanism families (consolidated). Each family’s primary contribution, maturity, and regulatory anchor; the takeaway is complementarity, each family serves different accountability dimensions, so LAAF integrates rather than privileges any one. Maturity is assigned from the three signal counts: high where all three are well represented, medium where two are, and emerging otherwise.

Figure 8: Regulatory and standards landscape timeline. No instrument in scope was published before January 2023, so the timeline begins later than the review window of January 2022 to March 2026.

Sector	Principal guidance	Deployment patterns	Mechanisms emphasised
Healthcare	FDA AI/ML SaMD; EMA reflection paper	Clinical decision support, diagnostics	Predetermined change control, validation, surveillance
Consumer finance	CFPB Circular 2023-03; EBA	Credit decisioning, advisory chatbots	Output traceability, reason-giving, contestability
Education	US Dept. of Education; UNESCO	Tutoring, assessment, personalisation	Transparency, instructor oversight, privacy
Public sector	OECD; EU AI Act Annex III	Service delivery, benefits, hiring	Risk classification, oversight, contestability

Table 7: Sectoral guidance relevant to LLM-based applications. Per sector, the principal guidance, deployment patterns, and the mechanisms each emphasises; the takeaway is that sector calibration is required even under a common horizontal framework.

Dimension	EU AI Act	NIST AI RMF	ISO/IEC 42001	Sectoral
Legal status	Binding, penalties	Voluntary, influential via procurement and state reference	Voluntary, certifiable	Mixed: binding where statutory, advisory otherwise
Scope	Horizontal	Horizontal	Horizontal	Sector-specific
Human oversight	Mandatory (Art. 14)	Govern and Manage; GenAI Profile actions	Operational controls	Mandatory in regulated sectors
Documentation/logging	Mandatory (Arts. 11–12, Annex IV)	Map/Measure	Documented information	Sector-specific records
Post-deployment monitoring	Mandatory (Arts. 72–73)	Measure and Manage	Performance evaluation	Post-market surveillance
Cybersecurity	Mandatory (Art. 15)	Map, Measure, Manage; GenAI Profile actions	Annex controls	Sector-specific

Table 8: Comparative analysis of regulatory and standards instrument types. Aligns the four instrument types across legal status, scope, and the four core obligations of Section 7.1; the takeaway is convergence on common requirements, which in principle allows one architecture to serve several jurisdictions, though the design and audit cost of doing so has not been measured empirically.

Persistent gap	Where documented	Requirement imposed on LAAF
Under-specification of human oversight	1.2, 7.1, 8.5	Specify what, when, with what information, with what authority
Absence of shared accountability metrics	7.5, 8.5	Define layer-specific accountability indicators
Disciplinary disconnection	1.2, 6, 7	Integrate mechanisms from the technical and governance literatures
Limited empirical evaluation	Sections 6–8	Articulate evaluation criteria and validation pathways

Table 9: Persistent gaps and resulting requirements for LAAF. Each gap is documented to specific sections and converted into a concrete requirement imposed on the framework; the takeaway is that the gaps are specific and tractable, not diffuse.

Tension	Where most visible	How LAAF addresses it
Transparency vs. competitive/security	Documentation, watermarking, security	Separate internal/external disclosure surfaces in Layer 4
Automation vs. substantive oversight	Oversight, decision support	Distinguish review regimes; require explicit justification
Standardisation vs. contextual fit	Audit, standards, sectoral	Sector-agnostic architecture with sector specialisations
Traceability vs. privacy	Provenance, logging, privacy	Governed provenance and audit assets in Layer 4
Deployment speed vs. governance maturity	Capability vs. risk governance	Incremental adoption by layer, with the accountability properties forgone at each stage stated explicitly

Table 10: Structural tensions and how LAAF addresses them. The five trade-offs any deployment must navigate, with the LAAF design response; the takeaway is that accountability is a navigation problem, not a checklist.

Figure 9: The Layered Accountability Architecture Framework (LAAF): four layers spanned by three cross-cutting properties (traceability, role clarity, continuous monitoring), ordered by distance from the model. Evidence flows upward, constraining the layer above; binding the layer below.

Layer	Accountability function	Principal artefacts	Named accountable actor
L1 Foundation Model & Provenance	Establish verifiable identity of model and training basis	Model card, training-data record, version, systemic-risk disclosures	Foundation-model provider; fine-tuning party
L2 Application Logic & Guardrails	Ground generation, enforce policy, mark uncertainty, log behaviour	Retrieval pipeline, guardrails, uncertainty scores, abstention rules, output provenance	Application developer; deploying organisation
L3 Human Oversight & Review	Place substantive human judgement in the path	Review workflow, escalation pathway, override authority, regime selector	Deploying organisation; domain professional
L4 Governance, Audit & Redress	Allocate roles, govern, audit, respond to incidents	Role allocation, governance committee, audit, incident response, management system	Deploying organisation (corporate entity)

Table 11: The four layers of LAAF: accountability function, principal artefacts, and named accountable actor per layer.

Property	Layer 1	Layer 2	Layer 3	Layer 4
Traceability	Model/version ID, training-data record	Output provenance, retrieval citations, uncertainty	Review record, override record	Audit record, incident record, minutes
Role clarity	Provider & fine-tuner named	Developer & deployer named	Reviewer & override authority named	Named individuals per governance role
Continuous monitoring	Update notifications, risk reassessment	Logging hooks, drift detection	Review/override pattern analysis	Incident cycles, audit cycles, improvement loop

Table 12: Cross-cutting properties and their realisation per layer. Shows the concrete artefact realising each property in each layer; the takeaway is that the properties stitch the four layers into a single auditable chain.

Layer	EU AI Act	NIST	ISO/IEC 42001	OWASP LLM Top 10 (2025)	Layer indicator
L1	Ch. V, Annex IV	Map	Design/development controls	LLM03 Supply Chain; LLM04 Data & Model Poisoning	Completeness of model/training disclosures
L2	Arts. 11–12, Annex IV	Map, Measure	Operational controls	LLM01 Prompt Injection; LLM02 Sensitive Info Disclosure; LLM05 Improper Output Handling; LLM06 Excessive Agency; LLM08 Vector & Embedding Weaknesses & LLM09 Misinformation	Proportion of outputs with citations & uncertainty
L3	Art. 14	Manage	Roles & competence	LLM07 System Prompt Leakage	Override rate, review time, regime justification
L4	Arts. 9, 17, 72–73	Govern (all)	Leadership, planning, evaluation, improvement	LLM10 Unbounded Consumption (org level); all categories	Incident closure rate, audit-cycle completion

Table 13: Mapping of LAAF layers to regulatory and cybersecurity requirements, with indicators. Per layer, the EU/NIST/ISO anchors, the OWASP LLM Top 10 (2025) categories, and a measurable indicator; the takeaway is that LAAF is regulator-mappable and security-integrated at every layer, with concrete metrics.

LAAF element	Healthcare	Finance	Public sector
L1 priorities	Clinical safety in training data	Fair lending in training data	Democratic neutrality in training
L2 guardrails	Clinical accuracy, citation grounding	Discriminatory features, data leakage	Political balance, source verification
L3 reviewer	Clinician with medical authority	Credit officer with adverse-action knowledge	Civil servant with verification duty
L4 governance	Clinical-AI lead + hospital quality	Credit risk + fair lending + consumer protection	Digital service + DPO + accountability officer
Regulatory anchors	EU AI Act high-risk, FDA, EMA	CFPB, EBA, EU AI Act Annex III	OECD, EU AI Act Annex III
Sector-specific risk	Diagnostic error, patient harm	Discriminatory denial, financial harm	Citizen rights, democratic harm

Table 14: LAAF sector specialisations across three case studies. Shows how each layer specialises across healthcare, finance, and public sector while the four-layer/three-property structure stays constant; the takeaway is sector-agnostic structure with sector-calibrated content.

Ref.	Framework	Layered	Oversight	Cyber	Reg. map	Sector	Roadmap	Eval. criteria
[ 70 ]	Three-layered LLM audit	$\checkmark$	P	$\times$	P	$\times$	$\times$	$\times$
[ 3 ]	NIST AI RMF	$\times$	$\checkmark$	P	P	$\times$	$\times$	P
[ 4 ]	NIST GenAI Profile	$\times$	$\checkmark$	$\checkmark$	P	P	$\times$	$\times$
[ 46 ]	ISO/IEC 42001	$\checkmark$	$\checkmark$	P	P	$\times$	$\times$	P
[ 10 ]	Transparency Index	$\times$	$\times$	$\times$	P	$\times$	$\times$	$\times$
[ 90 ]	Risk governance	P	P	$\times$	$\times$	$\times$	$\times$	$\times$

Table 15: Comparative analysis of LAAF against existing governance frameworks. Each framework is scored on the seven criteria listed at the head of this subsection, where

\checkmark

denotes the criterion is addressed as a design element, P that it is discussed without being operationalised, and × that it is absent. To our knowledge, LAAF is the only entry covering all seven. Scores are author-assigned against this rubric without a second independent rater, which Section 11 records as a limitation. The Eval. column records whether evaluation criteria are articulated, not whether the framework has been empirically validated; no entry in the table, including LAAF, reports empirical validation in deployment. Layered denotes decomposition of accountability into ordered strata each with a distinct accountable actor, rather than into process phases or clause groups.

AI Accountability

AI accountability concerns how responsibility for AI decisions and harms is assigned, evidenced, challenged, and remedied.

See all 157 papers

LLM-FACETS: A Privacy-Preserving Framework for Evaluating LLM Transparency and Accountability
May 29, 2026Tom Lucas, Alessio Buscemi, Alfredo Capozucca +2LLM Evaluation AI Accountability
Luxembourg Institute of Science and Technology (LIST), Luxembourg, University of Luxembourg, Luxembourg, and Scriptor Artis, France · Luxembourg Institute of Science and Technology (LIST), Luxembourg · University of Luxembourg, Luxembourg +1
A Scoping Review of LLM-as-a-Judge in Healthcare and the MedJUDGE Framework
Apr 3, 2026Chenyu Li, Zohaib Akhtar, Mingu Kwak +13LLM-as-a-Judge Clinical Language Model Evaluation
Department of Biomedical Informatics, School of Medicine, University of Pittsburgh, Pittsburgh, PA, USA · Department of Health Information Management, School of Health and Rehabilitation Sciences, University of Pittsburgh, Pittsburgh, PA, USA · OpenCura, Health Innovation Consortium +6
Generated Context versus Governed State: Functional Conditions for Accountable Longitudinal Clinical Reasoning
Aug 14, 2026Augusto Bernardo Pissarra, Victor Lorena de Farias SouzaAI Accountability Clinical Reasoning
MyndwareMed

May 29, 2026J/K move · Enter open · S save

LLM-FACETS: A Privacy-Preserving Framework for Evaluating LLM Transparency and Accountability

Tom Lucas, Alessio Buscemi, Alfredo Capozucca, German Castignani+1

Luxembourg Institute of Science and Technology (LIST), Luxembourg, University of Luxembourg, Luxembourg, and Scriptor Artis, France · Luxembourg Institute of Science and Technology (LIST), Luxembourg · University of Luxembourg, Luxembourg+1

LLM Evaluation AI Accountability LLM Auditing LLM Reliability

Open paper AskPDF arXiv

Assessing whether Large Language Models outputs are factually grounded, epistemically calibrated, and methodologically reproducible is a prerequisite for responsible AI deployment. Yet auditing LLMs remains inaccessible to non-technical practitioners: existing tools require programming expertise and non-trivial environment setup, and cloud-hosted platforms transmit evaluation data to external services, creating barriers for domain experts and compliance officers legally responsible for AI oversight. We introduce LLM-FACETS (LLM FActuality Cross-EvaluaTion System): an open-source framework with a browser-accessible interface and a plugin architecture, structured around three practitioner profiles (technical experts, domain experts, compliance officers) that mirror the stakeholder categories identified in the EU AI Act and the NIST AI Risk Management Framework. The architecture makes data flows explicit: deterministic metrics (BLEU, ROUGE, BERTScore) run entirely within the self-hosted server with no outbound transmission; LLM-judge metrics contact external APIs explicitly, with users retaining full credential control. The framework operationalizes transparency through three mechanisms: token-level log-probability visualization for epistemic uncertainty, multi-judge consensus to mitigate judge bias, and RAG Triad metrics (Faithfulness, Answer Relevance, Context Relevance) to detect and localize hallucinations. A plugin architecture allows any new metric or dataset to be integrated without modifying the evaluation pipeline. The open-source implementation enables cross-checking across multiple metrics targeting the same property, ensuring reproducibility and decoupling AI accountability from the teams building the systems assessed. We verify the framework through cross-validation of 18 metric implementations against canonical reference libraries.

Abstract

AI Accountability

Explore similar work

LLM-FACETS: A Privacy-Preserving Framework for Evaluating LLM Transparency and Accountability

A Scoping Review of LLM-as-a-Judge in Healthcare and the MedJUDGE Framework

Generated Context versus Governed State: Functional Conditions for Accountable Longitudinal Clinical Reasoning