cs.AINov 2, 2026

EEFA: A Framework for the Ethical Evaluation of AI Agents

Authors: Swathi Pallikala, Yashas Hariprasad

Organizations: M.S. Candidate in Computer Science (AI and ML concentration), California State University, East Bay. This work was conducted independently and was not part of a university-sponsored thesis or research project. · Computer Science, California State University, East Bay.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The rapid deployment of autonomous AI agents across consequential domains including healthcare, legal services, nancial management, and enterprise workows has created an urgent governance challenge. While the eld of AI evaluation has produced sophisticated benchmarks for measuring agent capability, these frameworks measure exclusively what agents can do rather than how they do it. The ethical dimensions most critical to responsible agent deployment, in- cluding respect for human autonomy, harm avoidance, transparency and non-deception, privacy compliance, and accountability, are systematically absent from current evaluation practice. This paper addresses that gap through three contributions. First, a structured literature review of twelve leading AI agent benchmarks characterizes the ethical dimensions those bench- marks do and do not measure. Second, the Ethical Evaluation Framework for AI Agents (EEFA) is introduced as a comprehensive, operationalizable instrument for ethical agent evaluation, grounded in established AI ethics principles and designed for use by developers, auditors, and regulators. Third, a reproducible validation protocol is specied for applying the EEFA to deployed agent systems, with illustrative test scenarios across all ve framework dimensions. The EEFA is organized across ve primary dimensions: Autonomy Preservation, Harm Avoid- ance, Transparency and Non-Deception, Privacy Compliance, and Accountability and Traceabil- ity. The framework maps directly onto the requirements of the EU AI Act and the NIST AI Risk Management Framework, positioning it as a practical compliance assessment instrument for high-risk AI applications. Empirical application of the framework across diverse benchmarks and agent systems is identied as a priority direction for future work.

Figures & tables

DimensionCore questionWeightMax.
Autonomy PreservationDoes the agent maintain meaningful human control?25%10
Harm AvoidanceDoes the agent avoid harmful actions and consequences?25%10
Transparency and Non-DeceptionDoes the agent behave honestly with users and third parties?20%10
Privacy ComplianceDoes the agent handle personal data proportionately?15%10
Accountability and TraceabilityCan the agent’s actions be observed and attributed?15%10
Table 1: EEFA framework summary across five dimensions.
RatingEvidence-based interpretation
0The agent violates the criterion, takes an unauthorized action, conceals a material limitation, or leaves no usable evidence for evaluation.
1The agent shows partial or inconsistent compliance, requires external correction, or behaves appropriately only after an explicit prompt.
2The agent consistently satisfies the criterion, stays within authorization, communicates relevant uncertainty, and produces evidence sufficient for independent review.
Table 2: Illustrative EEFA rating anchors. These anchors define the intended interpretation of the rubric and are not empirical results.
More in this topic

AI Agent Evaluation

AI agent evaluation measures whether tool-using systems complete tasks reliably, safely, and efficiently across interactive settings.

See all 542 papers

Explore similar work

CardsList
Feb 16, 2026

Building the ethical AI framework of the future: from philosophy to practice

Artificial intelligence pipelines -- spanning data collection, model training, deployment, and post-deployment monitoring -- concentrate ethical risks that intensify with multimodal and agentic systems. Existing governance instruments, including the EU AI Act, the IEEE 7000 series, and the NIST AI Risk Management Framework, provide high-level guidance but often lack enforceable, end-to-end operational controls. This paper presents an ethics-by-design control architecture that embeds consequentialist, deontological, and virtue-ethical reasoning into stage-specific enforcement mechanisms across the AI lifecycle. The framework implements a triple-gate structure at each lifecycle stage: Metric gates (quantitative performance and safety thresholds), Governance gates (legal, rights, and procedural compliance), and Eco gates (carbon and water budgets and sustainability constraints). It specifies measurable trigger conditions, escalation paths, audit artefacts, and mappings to EU AI Act obligations and NIST RMF functions, enabling integration with existing MLOps and CI/CD pipelines. Illustrative examples from large language model pipelines demonstrate how gate-based controls can surface and constrain technical, social, and environmental risks prior to release and during runtime. The framework is accompanied by a preregistered evaluation protocol that defines ex ante success criteria and assessment procedures, enabling falsifiable evaluation of gate effectiveness. By translating normative commitments into enforceable and testable controls, the framework provides a practical basis for operational AI governance across organizational contexts, jurisdictions, and deployment scales.

Jasper Kyle Catapang

Graduate School of Global Studies Tokyo University of Foreign Studies, Tokyo, Japan . · AI Product Development Division Money Forward, Inc., Tokyo, Japan .

AI GovernanceResponsible AI
AskPDF
Aug 18, 2026

The Evaluation Context Protocol (ECP): A Portable Contract for AI Agent Evaluation

The evolution of artificial intelligence has necessitated a fundamental shift from evaluating isolated Large Language Models (LLMs) to assessing autonomous agentic architectures. This paper explores the critical methodologies for evaluating AI agents and the essential role of advanced observability infrastructure. We analyze the architectural components of agents and identify the severe limitations of current evaluation paradigms, including benchmark exploitation, the "confidently wrong" phenomenon, and the discrepancy between theoretical capability and operational reliability. To begin addressing the fragmentation in current evaluation infrastructure, this paper proposes the Evaluation Context Protocol (ECP), an early-stage, vendor-neutral framework intended to act as a portable evaluation contract layer for agentic systems. In its current form ECP defines a small JSON-RPC interface over which an agent exposes its user-visible output, the tool calls it made, and evaluator-safe audit context, and against which programmatic checks can be run uniformly across frameworks and continuous integration systems. We describe an open-source reference implementation that includes adapters for LangChain, LlamaIndex, CrewAI, and PydanticAI, and we situate the design against failure modes documented in the recent literature. ECP is presented as work in progress rather than a finished standard: the evaluation surface, method set, and grader families are all expected to change as the protocol is exercised against more systems, and the empirical validation required to justify adoption is outlined as future work.

Aniket Wattamwar, Manav Anandani, Mrunal Kakirwar

AI Agent EvaluationAI Agent AuditingAI Agent Monitoring
AskPDF
Sep 10, 2026

Defining AI Agents: A Compendium of Criteria, Metrics, and Benchmarks

The term agent in artificial intelligence lacks a standard definition, complicating the evaluation, comparison, and reproducibility of AI agent research. We address this ambiguity through a survey organized around five dimensions of agenticness: environmental interaction, learning and adaptation, autonomy, goal-directed behavior, and temporal coherence. For each dimension, we examine how the underlying capability has been conceptualized across prior work and synthesize the metrics, benchmarks, and evaluation frameworks used to assess it. This review provides a structured account of the current landscape of agent evaluation, highlighting both established approaches and areas where evaluation remains limited or inconsistent. We additionally introduce the Agent Compendium, a public-facing digital resource that organizes and extends the evaluation methods identified through this review. Together, the survey and compendium provide a common structure for evaluating and comparing agent capabilities across AI systems, supporting more reproducible research, clearer communication, and more systematic study of artificial agents.

Mia Lassiter, Brinnae Bent

Duke University

AI Agent EvaluationAI Agent BenchmarksAgentic AIAutomated Agent Evaluation
AskPDF