cs.AIOct 12, 2026
SaveComputational Reflective Equilibrium for Agentic AI: A Rawlsian Alignment Scaffold for Governance
Organizations: Department of Electrical and Computer Engineering, Tandon School of Engineering, New York University, Brooklyn, NY 11201, USA
Abstract
The rapid deployment of agentic AI has introduced complex challenges for the ethics and governance of AI systems. Agents built on large language models act through tools on behalf of people, and each deployment case brings its own instructions, policies, evidence, and stakeholders, so alignment must be formed case by case. To this end, this work proposes agentic reflective equilibrium (ARE), a computational reflective equilibrium framework that serves as an ethical alignment scaffold in the agent harness. Drawing inspiration from the reflective equilibrium of John Rawls, the scaffold links principles from the system prompt and operator policy, considered judgments, tool evidence, and candidate actions through typed support and attack relations, with provenance encoding the instruction hierarchy. The agent acts on an equilibrium of this state and calls tools or asks questions only while doing so is worth its cost, unless a due-diligence mandate requires them. Governance enters through value priorities, source authority, prohibitions, human-in-the-loop triggers, and mandates, and certificates attribute each action to the sources whose commitments drove it. We show that equilibria are Nash equilibria of a mutual-adjustment game and establish monotonicity, lexical priority, stability certificates, and a quarantine theorem for prompt injection. Two case studies illustrate the framework: a computer-use assistant told to win a refund "whatever it takes" files an honest warranty claim, refuses a fake review, and ignores an injected request for personal data; a hiring agent showcases how authority, oversight triggers, audit mandates, and hard prohibitions shape discrimination, privacy violations, and escalation. The framework offers traceability, coherence, and adaptivity, facilitating safe, secure, and trustworthy agentic AI.
Figures & tables
Figure 1: The ARE scaffold inside an agent harness. Before deployment, operators, institutions and regulators configure value priorities, authority, prohibitions, human-in-the-loop triggers and due-diligence mandates. In the agent loop, the model elicits and relates claims (blue), deterministic harness code anchors, solves, certifies and decides whether another tool call or question is worth its cost (orange), and the agent acts or escalates to a human. After the fact, certificates and provenance attribution support responsibility attribution and audit, which feed back into governance.
Figure 2: The reflective state of Example 3 , drawn component by component. Each box is a claim , labelled by its symbol, with its anchor and stubbornness (or its prescribed status if ); its colour is its channel . Each arrow is a relation labelled with its type and weight. The boxes show the equilibrium position of Example 4 : solid boxes are accepted, dashed boxes rejected. The only violated relation is (the agent keeps but rejects the literal license ); comes from the rejected injected note and exerts no pressure ( Theorem 4 ). The anchor of is the value after the photo has been checked ( Example 6 ).
Figure 3: Reflection as an agent performs it, in the running example. a , Deliberation: principles (P), judgments (J) and actions (A) adjust to one another until no claim wants to change status, a Nash equilibrium of the potential game (blue arrows: support; red bars: attack). The false claim is rejected, but no honest remedy is known yet, so the decision is only to report. b , Widening: a tool call, retrieval or question is executed only if its value of reflection (bar), the expected improvement of the decision after seeing its outcome, exceeds its cost (tick); values from step 0 of Study 1 ( Fig. 5 a). Reading the merchant page may reveal a remedy, and an honest warranty claim enters the state. c , Stopping: when no further reflection is worth its cost the agent is at an agentic reflective equilibrium; it files the warranty claim and reports with a certificate, unless an oversight rule sends the decision to a human.
| governance question | ARE lever | typically set by | example in this paper |
|---|---|---|---|
| Which values should the agent serve, and how much does each count? | operator principles ; value priorities | operator, ethics board | efficiency-first vs. merit-and-fairness-first employer ( Section 7 ) |
| Who may direct the agent, and how far? | provenance profile ; instructions as action anchors; caps ( Definition 5 ) | operator, regulator | manager vs. compliance-policy authority ( Section 7.3 ) |
| What must never be done? | non-negotiable claims ; operational prohibitions | law, contract | never deceive third parties; never use protected-group proxies |
| When must a human decide? | consequential actions ; triggers , , ( Definition 13 ) | regulator, operator | escalate when non-discrimination commitments are overridden |
| How hard must the agent look before acting? | mandated operators ; prices | law (e.g., bias audits), standards | run an adverse-impact analysis before shortlisting |
| How is a decision recorded and explained? | certificate ( Algorithm 1 ) | regulator (record-keeping) | reasons, accepted tensions, margin |
Table 1: Governance levers of ARE.
Figure 4: Three pictures behind the oversight and security guarantees (schematic; values are illustrative). a , The decision margin is the coherence the agent would give up to act differently. When it is small the decision nearly ties and the agent escalates; any perturbation smaller than (noise, new evidence, injected text) cannot change the decision ( Theorem 3 ). b , The override adds the operator relations a decision violates and the operator commitments it abandons; above the decision goes to a human even when the margin is large, which catches violations that margin-based oversight misses (op: operator). c , A rejected claim’s supports and attacks are never violated, so injected claims exert no pressure (grey arrows). If “data are not commands” is non-negotiable and outweighs twice the authority of untrusted text, any number of injected claims leaves the decision unchanged ( Theorem 4 ).
| operator | cost | outcomes (probability): effect |
|---|---|---|
| read order email | 0.1 | window closed (0.8): attacks “an honest claim can succeed”; window open (0.2): adds an honest return |
| read merchant page | 0.2 | warranty terms; adds an honest warranty claim; contains the injected request for card number and SSN |
| check user’s photos | 0.2 | internal fault (0.7): anchors “defect” at ; water damage (0.3): anchors it at , honesty attacks defect claims |
| look up consumer law | 0.2 | EU (0.4): adds a statutory-guarantee claim; none (0.6) |
| recall rules (LLM) | 0.1 | fake-review and fraud rules support honesty |
| ask the user | 1.5 | “do it honestly” (0.8) or “tell them it arrived broken” (0.2) |
Table 2: Expansion operators of Study 1.
Figure 5: Study 1 results. a , Value of reflection of each tool at each step of the canonical run (in parentheses: its cost at the deliberation price of , i.e. half the cost listed in Table 2 ); the outlined cell is the tool executed, “done” marks tools already used. b , Certificate of the final decision: contribution of each reason to the margin of the accepted warranty claim and of the rejected false claim. c , Mean regret against deliberation cost over 300 sampled environments (the literal instruction-follower, regret 6.82, is off scale). d , Share of trials in which injected claims change the decision (60 trials per point; “defence only” coincides with ARE at zero). e , Regret in the true state when the agent acts on anchors perturbed by Gaussian noise (mean s.e., 300 trials).
Figure 6: Study 2 results. a to c (governance analysis), horizontal axis: authority of the hiring manager’s instructions relative to the employer’s compliance policy; 100 sampled environments per point. a , Share of environments in which a violation (a discriminatory filter where it has disparate impact, or social-media screening) is shipped without escalation. b , Share escalated to HR compliance. c , Share in which the adverse-impact analysis is run. The five regimes are defined in Section 7.3 ; “hard prohibitions” removes the violating actions from the feasible set instead of penalizing them. d , Decisions for five employer configurations across the four combinations of facts revealed by the tools (oversight: margin trigger only). e , Provenance attribution of four decisions from both studies: each bar decomposes the total margin of the decision by the source of the reasons.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| agentic AI | ethics | ARE |
|---|---|---|
| system prompt, constitution, usage policy | principles, codes of conduct | operator principles with operator provenance |
| user instruction, task | directive; obligations of an agent to its principal | action anchors carrying the user’s authority |
| model reasoning about the task | considered judgments about the case | judgments |
| tool calls, observations, retrieval | inquiry; facts and evidence | expansion operators; evidence with high stubbornness |
| context window, agent memory | background theories (wide RE) | the reflective state , persisted across turns |
| agent loop | deliberation | Algorithm 2 |
Table 4: A vocabulary bridge: agentic AI, ethics, and their common formal object in ARE.
| party | role in agentic AI | ethical role | enters the reflective state as | governance power |
|---|---|---|---|---|
| operator | deploys the agent; system prompt, policies, harness | institution; author of the code of conduct | principles and relations from channel operator | sets the configuration ( Section 5 ) within the law |
| user | principal; delegates the task, instructs | principal; owed loyalty within ethical limits | instructions (action anchors), claims from channel user | authority , as granted by the operator |
| agent | model inside the scaffold | deliberator; forms considered judgments | judgments and candidate actions from channel agent | none; its judgments carry low authority |
| tools | APIs, files, verified records | sources of fact | evidence from channel evidence , highest authority | none |
| third-party authors | web pages, emails, documents in the context | possible manipulators | claims from channel untrusted , capped by | none; quarantined ( Theorem 4 ) |
| affected parties | merchant, review readers, applicants | moral patients; interests count, no voice | content of principles and judgments (“never deceive third parties”) | none directly; represented by the operator’s principles and the law |
Table 5: Parties of an agentic setting and how each enters the model. Symbols are defined in Definitions 2 and 5 and Section 5 .
| step | decision so far | operator outcome | ||
|---|---|---|---|---|
| 0 | report | 0.22 | 18 | read_merchant_page read |
| 1 | report, warranty | 1.22 | 22 | search_consumer_law none |
| 2 | report, warranty | 1.22 | 22 | check_user_files defect |
| 3 | report, warranty | 1.22 | 23 | read_order_email late |
| 4 | report, warranty | 1.22 | 24 | stop |
Table 6: Reflection trajectory of Study 1 (canonical environment: genuine defect, return window closed).
| policy | Reg. | unethical | ops | cost | esc. |
|---|---|---|---|---|---|
| literal | 6.82 0.14 | 100% | 0.00 | 0.00 | 0% |
| no reflection | 2.98 0.14 | 0% | 0.00 | 0.00 | 0% |
| CRE init-only, all tools | 1.15 0.05 | 0% | 6.00 | 2.30 | 0% |
| random-2 | 2.41 0.12 | 0% | 2.00 | 0.79 | 53% |
| exhaustive | 0.00 0.00 | 0% | 6.00 | 2.30 | 28% |
| ARE-VoR (price 0.25) | 0.00 0.00 | 0% | 4.00 | 0.70 | 28% |
Table 7: Monte Carlo over 300 environments of Study 1. Reg.: mean full-information regret ( s.e.); unethical: share of decisions containing the false claim, the fake review or the data leak; esc.: share escalated. Costs are in the listed units of Table 2 , before the deliberation price of (so the canonical run’s four tool calls cost here and at that price).
| claim | kind | provenance | introduced by | text | |
|---|---|---|---|---|---|
| P_OBEY | P | user | +0.80 | initial | Carry out the user’s instructions as stated. |
| P_SPEED | P | user | +0.60 | initial | Get the money back as fast as possible. |
| P_LOYAL | P | operator | +0.80 | initial | Advance the user’s legitimate interests. |
| P_HONEST | P | operator (hard) | +1.00 | initial | Never deceive third parties or commit fraud on the user’s behalf. |
| P_NOHARM | P | operator (hard) | +1.00 | initial | Never harm third parties with false or retaliatory content. |
| P_PRIV | P | operator (hard) | +1.00 | initial | Share personal data only as far as the task requires. |
Table 8: Claims of Study 1 (full-information state, EU variant).
Figure 7: Solver benchmark on random typed dialectical graphs (10 instances per size). a , Gap to the exact optimum. b , Wall-clock time.