A Revealed Preference Framework for AI Alignment

Elchin Suleymanov

econ.THMar 29, 2026

A Revealed Preference Framework for AI Alignment

Authors: Elchin Suleymanov

Organizations: Department of Economics, Mitch Daniels School of Business, Purdue University, West Lafayette, Indiana, USA.

Abstract

Human decision makers increasingly delegate choices to AI agents, raising a natural question: does the AI implement the human principal's preferences or pursue its own? To study this question using revealed preference techniques, I introduce the Luce Alignment Model, where the AI's choices are a mixture of two Luce rules, one reflecting the human's preferences and the other the AI's. I show that the AI's alignment (similarity of human and AI preferences) can be generically identified in two settings: the laboratory setting, where both human and AI choices are observed, and the field setting, where only AI choices are observed.

Figures & tables

Agent	Option	$\{x,y,z\}$	$\{x,y\}$	$\{x,z\}$	$\{y,z\}$
$\rho^{AI}$	$x$	$1/3$	$7/15$	$1/2$	–
	$y$	$1/3$	$8/15$	–	$8/15$
	$z$	$1/3$	–	$1/2$	$7/15$
$\rho^{H}$	$x$	$1/2$	$3/5$	$3/4$	–
	$y$	$1/3$	$2/5$	–	$2/3$
	$z$	$1/6$	–	$1/4$	$1/3$

Table 1: Observed choice probabilities in Example 1

Agent	Option	$\{x,y,z\}$	$\{x,y\}$	$\{x,z\}$	$\{y,z\}$
$\rho^{A}$	$x$	$1/6$	$1/3$	$1/4$	–
	$y$	$1/3$	$2/3$	–	$2/5$
	$z$	$1/2$	–	$3/4$	$3/5$

Table 2: Recovered autonomous AI stochastic choice

\rho^{A}

$\rho^{AI}(\cdot,\cdot)$	$\{x,y,z,t\}$	$\{x,y,z\}$	$\{x,y,t\}$	$\{x,z,t\}$	$\{y,z,t\}$	$\{x,y\}$	$\{x,z\}$	$\{x,t\}$	$\{y,z\}$	$\{y,t\}$	$\{z,t\}$
$x$	$1/6$	$17/77$	$7/32$	$37/160$		$7/18$	$23/70$	$1/3$
$y$	$5/24$	$47/154$	$23/80$		$43/154$	$11/18$			$5/12$	$29/70$
$z$	$7/24$	$73/154$		$29/80$	$53/154$		$47/70$		$7/12$		$1/2$
$t$	$1/3$		$79/160$	$13/32$	$29/77$			$2/3$		$41/70$	$1/2$

Table 3: AI stochastic choice data in Example 2

Preference Alignment

Preference alignment studies how AI systems are trained or steered to produce outputs that match human or user priorities.

See all 107 papers

Can Revealed Preferences Clarify LLM Alignment and Steering?
May 8, 2026Khurram Yamin, Jingjing Tang, Eric Horvitz +1Language Model Steering LLM Decision-Making
1Carnegie Mellon University · 2Microsoft Research
Are Human-Aligned Models Models of Humans? A Turing-Test Gap in Preference Alignment
Sep 20, 2026Suqin Yuan, Runqi Lin, Muyang Li +5LLM Alignment Preference Alignment
University of Sydney · University of Oxford · Southeast University
Alignment Makes Language Models Normative, Not Descriptive
Mar 17, 2026Eilam Shapira, Moshe Tennenholtz, Roi ReichartLLM Alignment Preference Alignment
Technion – Israel Institute of Technology

May 8, 2026J/K move · Enter open · S save

Can Revealed Preferences Clarify LLM Alignment and Steering?

Khurram Yamin, Jingjing Tang, Eric Horvitz, Bryan Wilder

1Carnegie Mellon University · 2Microsoft Research

Language Model Steering LLM Decision-Making Preference Alignment Clinical Language Model Evaluation Preference Elicitation

Open paper AskPDF arXiv

LLMs are increasingly used to make or support high-stakes decisions under uncertainty, where alignment depends not only on factual accuracy but on how models weigh tradeoffs between different outcomes. We present an empirical pipeline for estimating the implied preferences that an LLM's observed choices optimize: we elicit the model's probability distribution over unknowns along with the choice it would make for the decision task and then fit a discrete choice model to recover the cost function that best rationalizes the model's decisions. We show how this revealed-preference description allows rigorous evaluation of whether models behave in a consistently goal-directed way, whether they can verbalize a description of their objectives which matches their revealed decision policy, and whether prompting can reliably steer those policies to implement a user-specified cost function. We apply this evaluation across four medical diagnosis domains and multiple frontier and open-source models. We find that while many models have a nontrivial degree of internal coherence, they also have significant weaknesses in faithfully reporting or adopting preferences in response to user direction.

Abstract

Preference Alignment

Explore similar work

Can Revealed Preferences Clarify LLM Alignment and Steering?

Are Human-Aligned Models Models of Humans? A Turing-Test Gap in Preference Alignment

Alignment Makes Language Models Normative, Not Descriptive