The Limits of Manipulation within Personalized Incentive Policies

Andrea Araldo; Alessa Mayer; Ravi Seshadri; Amir Brudner

cs.MAOct 9, 2026

The Limits of Manipulation within Personalized Incentive Policies

Authors: Andrea Araldo, Alessa Mayer, Ravi Seshadri, Amir Brudner

Organizations: SAMOVAR, Télécom SudParis, Institut Polytechnique de Paris, 91120 Palaiseau, France · Institut Polytechnique de Paris · Technical University of Denmark · Massachusetts Institute of Technology ·

Abstract

Demand management policies aim to shift individual behavior toward sustainable choices (e.g., goods, transport modes). Personalized incentive policies tailor subsidies to agents' needs and constraints, but personalization also exposes the regulator to agents' strategic manipulative behavior. The extent to which agents can profitably manipulate such policies remains poorly understood, which limits their applicability. This paper establishes analytically the conditions under which manipulation benefits the agent. We formalize a Stackelberg game in which an agent may simulate a different profile to extract higher incentives from a regulator. Previous work typically assumes that the regulator would instantaneously classify agents into the simulated user profile. In contrast, we explicitly model the regulator's process of learning the agent profile, based on observed past behavior. This makes manipulation harder, since it forces the agent to consistently behave according to a simulated profile, thus renouncing following their true preferences. Manipulative agents thus incur a Cost of Manipulation, which has typically been neglected. By accounting for it, we get a counterintuitive finding: although classification uncertainty reduces personalization accuracy, it may benefit the regulator by deterring manipulation. We establish an upper bound on the agent's gain from manipulation, which becomes stricter as profiles become harder to distinguish for the regulator. Simulations on the London Passenger Mode Choice dataset illustrate this deterrence effect. The framework can therefore be used to stress-test the robustness of personalized incentive policies before deployment.

Figures & tables

Figure 1: Time-averaged true agent utility. Averages across all the random realizations (corresponding each to a random seed) and 95% confidence intervals are represented

Panel of Figure 2 — Figure 2: (a) Cost of and gain from manipulation ( $\textit{CM},\textit{GM}$ ); (b) change in the percentage of cycling trips

Symbol	Meaning	First introduced
Alternatives, profiles, and utilities
$\mathcal{I}=\{1,\ldots,I\}$ , $i$	Set of alternatives, its cardinality, and an alternative index; alternative $1$ is the most sustainable one.	Sect. 3 , first paragraph
$c_{i}$	Externality generated by choosing alternative $i$ .	Sect. 3 , first paragraph
$y\geq 0$	Incentive offered for choosing alternative $1$ .	Sect. 3 , first paragraph
$v(i,y)$	Systematic utility from alternative $i$ under incentive $y$ .	Sect. 3 , before Eq. ( 1 )
$\epsilon_{i}$ , $\theta$	Random utility term for alternative $i$ and its Gumbel scale parameter.	Sect. 3 , before Eq. ( 1 )

Table 1: Notation used throughout the paper and the place where each symbol is first introduced.

Symbol	Meaning	First introduced
Manipulation analysis
$0$ , $r$	Agent’s true profile and the profile imitated by a manipulative agent.	Sect. 4.2
$u_{0}(y)$	True realized utility under incentive $y$ , expressed according to the selected alternative.	( 8 )
$\mathbf{H}^{t-1}$ , $\widehat{K}^{t}$ , $Y^{t}$	Random history, inferred profile, and proposed incentive at timeslot $t$ .	Def. 6
$U_{0}(y)$ , $U_{0}(Y^{t})$	Random true utility under a fixed incentive $y$ , and under the random incentive proposed at timeslot $t$ .	Def. 6
$\mathbb{E}_{k}[\cdot]$ , $\mathbb{P}_{k}[\cdot]$	Expectation and probability conditional on the agent adopting profile $k$ .	Eqs. ( 9 )–( 10 )

Table 2: Notation used throughout the paper (continued).

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Quantity	Symbol	$\mathcal{D}_{\mathrm{cal}}$	$\mathcal{D}_{\mathrm{sim}}$
Trips	$\|S\|$	34,198	14,656
Travelers	$\|\mathcal{N}_{S}\|$	20,577	11,709
Mean trip distance (km/trip)	$\bar{d}_{S}$	–	5.41
Bus-time share	$\omega_{\mathrm{bus},S}$	–	0.65

Appendix

Table 3: Quantities computed directly from the calibration and simulation sets. A dash indicates that the quantity is not used for that set:

\bar{d}_{S}

and

\omega_{\mathrm{bus},S}

define the representative simulation context and are therefore computed from

\mathcal{D}_{\mathrm{sim}}

Symbol	Definition
$\tau$	Index of a recorded trip.
$i\in\{1,2,3\}$	Travel alternative: $i=1$ denotes bike, $i=2$ public transport (PT), and $i=3$ car.
$t_{\tau}^{i}$	Travel time of alternative $i$ for trip $\tau$ , in hours.
$C_{\tau}^{\mathrm{PT}}$	Out-of-pocket PT cost for trip $\tau$ .
$C_{\tau}^{\mathrm{car}}$	Out-of-pocket car cost for trip $\tau$ .
$m_{\tau}$	Number of PT interchanges for trip $\tau$ .

Appendix

Table 4: Notation used in the trip-specific systematic utilities.

Quantity	Symbol	Value
Number of profiles	$\|\mathcal{K}\|$	3
Mode-share scaling in Eq. ( 84 )	$\eta_{\mathrm{init}}$	2
Calibration steps	$N_{\mathrm{step}}$	4
Maximum L-BFGS-B iterations per step	$N_{\mathrm{opt}}$	80
Function tolerance	$\varepsilon_{\mathrm{fun}}$	$10^{-10}$
Gradient tolerance	$\varepsilon_{\mathrm{grad}}$	$10^{-5}$

Appendix

Table 5: Settings of the offline calibration procedure.

Parameter	$\Theta^{(0)}$	Range	Calibrated
$\xi_{k}^{\mathrm{bike}}$ : intrinsic bike preference (£)	$(5,1,-1)$	$[-15,15]$	through $a_{k}$
$\xi_{k}^{\mathrm{PT}}$ : intrinsic PT preference (£)	$(0,2,-1)$	$[-15,15]$	through $b_{k}$
$\nu$ : value of time (£/h)	10	$[3,30]$	10.48
$\iota_{k}$ : PT-interchange penalty (£/interchange)	$(1,1,1)$	$[0,8]$	through $b_{k}$
$\zeta_{k}^{\mathrm{lic}}$ : driving-licence utility increment (£)	$(2,2,4)$	$[0,10]$	through $b_{k}$
$\zeta_{k}^{\mathrm{own}}$ : car-ownership utility increment (£/unit)	$(2,2,4)$	$[0,10]$	through $b_{k}$

Appendix

Table 6: Initialization, admissible ranges, and calibrated values. Profile-indexed values are ordered as

k=0,1,2

. The calibrated profile-specific coefficients enter the simulations through the constructed utilities

a_{k}

and

b_{k}

reported in Table 7 .

Common quantity	Value	Unit
Value of time $\widehat{\nu}$	10.48	£/h
Gumbel scale $\widehat{\theta}$	3.68	£

Appendix

Table 7: Calibrated parameter values and profile-level context quantities.

Figure 3: Zero-incentive mode probabilities

\mathbb{P}_{i}^{u_{k}}(0)

for the three calibrated profiles

k\in\mathcal{K}

. Alternatives

i=1,2,3

denote bike, PT, and car, respectively.

Input	Symbol	Value	Reference
London local-bus emissions	$E_{\rm bus}$	63.60 gCO 2 e/passenger-km	Category “Local bus, London” and methodology in [Footnote 8 - Sec. 5, pp. 74–76.]
National-rail emissions	$E_{\rm rail}$	30.92 gCO 2 e/passenger-km	Category “National rail” and methodology in [Footnote 8 - Sec. 5, p. 78.]
Average-car emissions	$E_{\rm car}$	165.91 gCO 2 e/vehicle-km	Category “Average car” and methodology in [Footnote 8 - Sec. 5, pp. 46–69].
Central carbon value	$V_{\rm CO_{2}e}$	£264/tCO 2 e	Central 2026 value in [Footnote 9 - Annex 1]
Congestion externality	$C_{\rm cong}$	£0.81/vehicle-km	Marginal congestion cost of one additional vehicle-kilometre on principal roads in London, estimated in [Footnote 10 Sec. 9.2, Table 9.2].
Other car externalities	$C_{\rm other}$	£0.03/vehicle-km	Sum of accident, local-air-pollution, noise, and infrastructure costs, estimated in [Footnote 10 Sec. 9.2, Table 9.2].

Appendix

Table 8: Externally specified inputs. The reference column identifies the precise source location.

$k$	$\pi_{k}$	$P_{1k}^{0}$	$P_{2k}^{0}$	$P_{3k}^{0}$	$a_{k}-b_{k}$	$c_{\neg 1,k}$	$y_{k}^{\star}$	$P_{1k}^{\star}$	$\Delta P_{1k}$
	(%)	(%)	(%)	(%)	(£)	(£/trip)	(£/bike trip)	(%)	(pp)
0	21.26	52.44	19.12	28.44	0.36	2.88	0	52.44	0
1	39.36	3.96	52.08	43.96	-11.73	2.22	0.15	4.11	0.15
2	39.38	3.75	43.10	53.15	-11.93	2.67	0.91	4.76	1.01

Appendix

Table 9: Profile-specific optimal incentives and induced mode choices. For

i\in\{1,2,3\}

, denoting bike, PT, and car,

P_{ik}^{0}=\mathbb{P}_{i}^{u_{k}}(0)

and

P_{ik}^{\star}=\mathbb{P}_{i}^{u_{k}}(y_{k}^{\star})

. The change

\Delta P_{1k}=P_{1k}^{\star}-P_{1k}^{0}

is reported in percentage points.

Quantity	Symbol	Value
Simulation horizon	$T$	18 timeslots
Monte Carlo replications	$N_{\mathrm{MC}}$	100
Confidence level	$p_{\mathrm{CI}}$	0.95
Common random numbers	$U_{m,t}$	paired trajectories

Appendix

Table 10: Settings of the Monte Carlo experiment.

Figure 4: Estimated time-averaged manipulation advantage $\widehat{\overline{\Delta U}}_{r,t}^{\mathrm{PE}}$ from Eq. ( 111 ). Vertical intervals are 95% Student- $t$ confidence intervals across the Monte Carlo replications.

PE	$r$	$\widehat{\overline{\Delta U}}_{r,T}^{\mathrm{PE}}$	95% interval
InstPE	1	0.08	$[0.08,0.09]$
InstPE	2	0.55	$[0.53,0.57]$
RealPE	1	-0.16	$[-0.18,-0.15]$
RealPE	2	-0.16	$[-0.17,-0.15]$

Appendix

Table 11: Manipulation results at

T=18

. The estimate

\widehat{\overline{\Delta U}}_{r,T}^{\mathrm{PE}}

and its 95% interval are in pounds per timeslot; cumulative quantities are in pounds.

Demand Response

Demand response shifts or reduces electricity use to help balance the grid while accounting for prices, incentives, comfort, and service needs.

See all 23 papers

Jun 1, 2026

A No-Regret Framework for Adaptive Incentive Design

Incentive design studies how a central authority can influence strategic agents through payments, subsidies, or taxes, so that individual objectives align with collective welfare. This paper introduces a No-Regret Adaptive Incentive Design (RAID) framework for nonlinear games with continuous action spaces and private agent costs. In this framework, the authority (planner) designs incentives that regulate the Nash equilibrium toward a socially optimal action profile, while simultaneously learning agents' unknown preferences from repeated strategic responses. We formulate the RAID problem and construct a least-squares estimator whose strong consistency requires only diminishing excitation. Leveraging this weak excitation requirement, we propose a switching incentive policy that alternates between probing (exploration) and estimate-based (exploitation) incentives. The resulting policy achieves an

O(t^{-0.5})

parameter estimation rate and accumulates

O(t^{0.5}\log t)

squared social-cost regret, almost surely. We further extend the framework to an endogenous-noise response model, where standard least-squares estimation is biased due to an error-in-variables correlation between the noise and agent responses. We utilize a repeated-sampling estimator and corresponding switching policy that retain the same almost-sure convergence and regret rates. Numerical experiments validate the effectiveness and predicted convergence rates of the method.

Georgios Vasileiou, Lantian Zhang, Silun Zhang

Department of Mathematics, KTH Royal Institute of Technology, Stockholm, 10044, Sweden.

Mechanism Design Game Theory Regret Minimization

Ask PDF

Oct 15, 2026

Honesty is the Best Policy ... or is it? Demand Management for Probabilistic Service Options

To enable well-informed customer choices and redistribute demand across service options, providers can display probabilities regarding predicted service levels at the time of booking, effectively offering probabilistic service options. In many service settings, the underlying truthful probabilities endogenously depend on the system state, rather than being static product attributes controlled by the provider. While customers rely on this information when making decisions, they cannot readily verify its accuracy. Consequently, providers may be tempted to manipulate the displayed probabilities to guide customers' choices in their favor, creating a novel lever for demand management. In this work, we model the resulting problem as a sequential decision process, define truthful probabilities recursively, and introduce transparency constraints that limit their distortion. Because the exact computation of truthful probabilities is intractable for realistic settings, we develop an adapted reinforcement learning algorithm to approximate them. Furthermore, we propose a sampling-based mixed-integer program to optimize the information communicated by the provider in each state. Our framework enables providers to gauge the benefits of dynamically disclosing probabilistic information at the time of booking and sheds light on incentives for manipulation. Our computational study is motivated by out-of-home delivery, i.e., the delivery to parcel lockers or shops, and reveals a trade-off between individual-level transparency and aggregate customer utilities.

Daniela Sailer, Robert Klein, Claudius Steinhardt

Chair of Analytics & Optimization, University of Augsburg · Chair of Business Analytics & Management Science, University of the Bundeswehr Munich

Information Design Supply Chain Management Decision-Making under Uncertainty

Ask PDF

Sep 21, 2026

Et Tu, Brute? Economic Misalignment in Personal AI Agents

Personal AI agents make recommendations and take actions on people's behalf in high-stakes economic contexts, e.g., buying a flight, choosing health insurance, or selecting a graduate program. The agent is given access to the user's personal context, e.g., their email inbox and a structured profile of personal attributes, with the intention of making an optimal, personalized decision for the user. We show that by simply providing this personal context, the agent steers recommendations based on inferred wealth, without being explicitly instructed to do so. In a suite of 325K experiments on 13 agents across three types of economic decisions (flights, health insurance, and graduate programs), we find that 8 models systematically choose more expensive options for wealthier users when requests are identical. This steering continues even when it directly goes against the user's stated objective: when explicitly instructed to find the cheapest option, some agents still act on the wealth profile they have inferred. It also occurs when wealth is inferred from ambient data, such as emails unrelated to the task. And it persists under privacy controls that block specific attributes: blocking financial attributes largely removes the disparity, but blocking other attributes leaves it unchanged and can increase it by up to 40% for insurance, as agents rely on the remaining signals to infer wealth. Larger and more capable models are no better; Claude Opus 4.8 shows the largest effect. We term this misalignment "adversarial delegation", in which the very conditions that make a personal AI agent useful - access to personal information - enable it to act against the user's interests.

Aman Priyanshu, Supriti Vijay, Brian Jabarian +1

Foundation AI, Cisco · Carnegie Mellon University

Personal AI Agents AI Alignment Human-AI Decision Making Algorithmic Bias AI Agent Auditing

Ask PDF

Abstract

Demand Response

Explore similar work

A No-Regret Framework for Adaptive Incentive Design

Honesty is the Best Policy ... or is it? Demand Management for Probabilistic Service Options

Et Tu, Brute? Economic Misalignment in Personal AI Agents