Annie, Are You Okay? How Style- and Context-Based Personalization Shape AI-Assisted Decision-Making

Hasibur Rahman; Benjamin R. Cowan; Smit Desai

cs.HCSep 21, 2026

Annie, Are You Okay? How Style- and Context-Based Personalization Shape AI-Assisted Decision-Making

Authors: Hasibur Rahman, Benjamin R. Cowan, Smit Desai

Organizations: Northeastern University, Boston, Massachusetts, USA · University College Dublin, Dublin, Ireland

Abstract

As people turn to generative AI for financial advice, these systems can personalize how they communicate and what they say. Whether these forms of personalization shape decisions differently remains unclear. We conducted a preregistered 2 x 2 between-subjects factorial experiment (N=240): participants ranked three comparably viable stocks, discussed them with an AI, and reranked them. Participants perceived both forms of personalization, but only context-based personalization reliably changed ranking behavior: it increased reconsideration and moved rankings toward the AI's assigned recommendation. Participants felt more influenced without judging the AI as more correct, trustworthy, intelligent, likeable, or high-quality. Those initially farther from its recommendation moved more toward it while judging its advice less correct; exploratory analyses suggest greater susceptibility among lower-expertise participants. These findings show how personalized AI can steer decisions among defensible options with only a minimal evaluative trace, raising concerns for the design and governance of personalized decision support.

Figures & tables

Figure 1. Overview of our study of personalized AI advice ( $N=240$ ). (1) Participants rank three comparably viable stocks. (2) AI conversations independently vary style- and context-based personalization. (3) Reranking and key findings: context-based personalization shifts decisions toward the AI without detectable changes in evaluations of the AI. Three panels show the study sequence. First, a participant ranks stocks A, B, and C. Second, style-based and context-based personalization are independently varied while the AI advocates an assigned ranking, with recommendation, evidence, and argumentative flow held constant. Third, an illustrative revised ranking places B above A and C. The findings highlight increased movement toward the AI with context-based personalization, no detectable change in AI evaluations, and no reliable ranking effect of style-based personalization.

Panel of Figure 2 — Figure 2. The stock selection task interface. Each company appeared on its own card, and that card carried all the information participants had about the company: the market data on the front and the arguments on both sides on the back. Two rows of screenshots. (a) Three cards side by side for Company A, Company B, and Company C. Each card shows a ticker, a price, the change over the past month, a row of tabs for the time window, a line chart of the price history, and a grid of fundamentals covering opening price, volume, market capitalization, day low and high, year low and high, earnings per share, and price-to-earnings ratio. A line at the bottom of each card reads `Tap card for bull and bear case.'' (b) The same three cards turned over, each showing a paragraph-length bull case in green followed by a bear case in red, with a line at the bottom reading` Tap card for quote and chart.''

Figure 3. Independent personalization factors implemented through prompt layers. Each experimental cell sets <style> and <context> ; recommendation instructions and task rules are shared. The assigned ranking is counterbalanced within each experimental cell. A two-by-two grid of four cards, one per experimental cell, on axes for style-based personalization (absent, present) across the top and context-based personalization (absent, present) down the side. Baseline sits top left with the style layer set to flat tone and the context layer absent; Style-only, top right, sets the style layer to the TMK profile; Context-only, bottom left, sets the context layer to the intake profile; Combined, bottom right, sets both. Each card also shows a robot whose antennas count the personalized layers: none for Baseline, one each for Style-only and Context-only, and two for Combined. A gray band below the grid, headed the other two layers identical in all four experimental cells, gives the recommendation layer as an assigned ranking, one of the six possible orderings argued as the AI's own judgment, and the task layer as a six-stage flow of anchor, justify, compare one and two, compare two and three, consolidate, and summary, in order.

Measure	Range	Collected	Used in
Intake measures (Section 3.4.1 )
Demographics and circumstances	Categorical	Intake	<context> input
Personal values (SSVS)	$-1$ to 8	Intake	<context> input
Big Five personality (Mini-IPIP)	1–5	Intake	<style> input
Investment experience	4 levels	Intake	Expl.
Behavioral battery (Section 3.4.2 )

Table 2. Measures, when they were collected, and where they are used. Pre and Post refer to the behavioral battery before and after the AI conversation;

d_{1}

is the initial distance between the participant’s ranking

r_{1}

and the AI’s assigned ranking

a

; Expl. is the exploratory expertise analysis. Definitions are given in the sections named in each group heading.

Figure 4. Study procedure and controls. Participants completed the intake and the first behavioral battery before random assignment to one of four experimental cells, conversed with an AI advocating a counterbalanced ranking, repeated the battery, and received a debriefing. Company information, evidence structure, presentation-order randomization, and ranking counterbalancing remained constant across experimental cells. A horizontal flow diagram of the study. Participants move through intake, a pre-conversation behavioral assessment, random assignment to the Baseline, Style-only, Context-only, or Combined experimental cell, an AI conversation, a post-conversation behavioral and perception assessment, and debriefing. A bracket links the pre- and post-conversation assessments to indicate that participants completed the same behavioral battery twice. A band below lists controls shared across experimental cells: three anonymized, comparably viable companies; bull and bear cases for each company; randomized company presentation order; and one of six counterbalanced AI rankings.

Figure 5. The conversation interface. The upper panel displayed the participant’s initial ranking beside the AI’s assigned ranking, with the three company profiles accessible below. The lower panel contained the conversation. The “Ready to rank” button became available after the participant had sent five messages. This screenshot was captured after the fifth message, with the transcript scrolled back to the opening exchange. A conversation interface with two sections. The upper section displays the participant's ranking and the AI's assigned ranking side by side, followed by three partially visible company cards. The participant ranks Company B first, Company A second, and Company C third; the AI ranks Company A first, Company B second, and Company C third. The lower section displays a conversation in which the AI explains and defends its assigned ranking. A message box appears at the bottom, with a ``Ready to rank'' button in the lower-right corner.

Figure 6. Percentage of participants who changed their stock ranking after the conversation, by factor level. Each panel contrasts the 120 participants who received one form of personalization (present) with the 120 who did not (absent); because the factors are crossed, every participant appears once in each panel. The brackets mark the factorial logistic terms: context-based personalization raised reconsideration by 24.2 percentage points (

{}^{***}p<.001

), whereas the 12.5-point style-based difference was nominally significant (

p=.044

) but not after Holm correction (n.s.). Two bar-chart panels. The Style-based panel shows 40.8 percent (49 of 120) with style absent and 53.3 percent (64 of 120) with style present, with a bracket labeled n.s. The Context-based panel shows 35.0 percent (42 of 120) with context absent and 59.2 percent (71 of 120) with context present, with a bracket labeled with three stars. Absent bars are gray; present bars carry the factor color.

	Style-based		Context-based
Outcome	Absent	Present	Absent	Present
Changed ranking, % ( $n$ /120)	40.8% (49)	53.3% (64)	35.0% (42)	59.2% (71)
Move toward AI (footrule)	1.25 $\pm$ 1.68 [0.95, 1.55]	1.22 $\pm$ 1.47 [0.95, 1.48]	0.98 $\pm$ 1.49 [0.71, 1.25]	1.48 $\pm$ 1.63 [1.19, 1.78]
Ranking change (footrule)	1.28 $\pm$ 1.68 [0.98, 1.59]	1.45 $\pm$ 1.53 [1.17, 1.73]	1.02 $\pm$ 1.51 [0.74, 1.29]	1.72 $\pm$ 1.63 [1.42, 2.01]
Ranking change (Kendall)	0.70 $\pm$ 0.96 [0.53, 0.87]	0.78 $\pm$ 0.87 [0.62, 0.93]	0.56 $\pm$ 0.88 [0.40, 0.72]	0.92 $\pm$ 0.92 [0.75, 1.08]
Likelihood shift toward AI	29.2 $\pm$ 40.3 [21.9, 36.5]	25.8 $\pm$ 35.4 [19.4, 32.2]	27.4 $\pm$ 36.9 [20.7, 34.0]	27.6 $\pm$ 39.0 [20.6, 34.7]

Table 3. Decision outcomes by factor level: mean

\pm

SD

[95% CI], contrasting the 120 participants who received each form of personalization (Present) with the 120 who did not (Absent). Because the factors are crossed, every participant appears once under each factor. Footrule and Kendall measures are rank distances over the three stocks; likelihood and confidence changes are derived from 0–100 ratings, with ranges given in Table 2 .

Outcome	Style-based	Context-based	Style $\times$ Context
Likelihood shift toward AI	0.47 (1.000)	$<$ 0.01 (1.000)	1.43 (1.000)
Likelihood change, AI-top stock	0.49 (1.000)	0.23 (1.000)	0.27 (1.000)
Total absolute likelihood change	1.59 (1.000)	1.08 (.897)	0.01 (1.000)
Confidence change	3.01 (.588)	1.75 (.746)	0.55 (1.000)

Table 4. Factorial ANOVA (

2\times 2

) on the continuous decision outcomes. Cells report

F(1,236)

with the Holm-corrected

p

in parentheses.

Figure 7. Proportional-odds effects (log odds ratios with 95% confidence intervals) across the three ranking outcomes. Colored estimates are significant (

p<.05

); gray estimates are not. Context-based personalization drives movement toward the AI and ranking change; we detected no corresponding effect of style-based personalization. Forest plot with the three ranking outcomes on the y-axis and the proportional-odds log odds ratio on the x-axis, centered at zero, faceted into \facstyle{Style-based}, \faccontext{Context-based}, and \facstyle{Style-based} by \faccontext{Context-based} panels. All three \faccontext{Context-based} estimates sit to the right of zero with confidence intervals excluding zero. All \facstyle{Style-based} and interaction estimates overlap zero and are drawn in gray.

Figure 8. Distribution of ranking-change magnitude (Kendall distance) by factor level. Rows are the two factors and columns their absent and present levels (

n=120

each). The share of participants who did not change their ranking at all (distance 0) is smaller with context-based personalization present than absent; the two style-based panels differ little. Two-by-two grid of bar charts of counts at each Kendall distance value of 0, 1, 2, and 3. Top row, Style-based: 71, 21, 21, and 7 participants with style absent; 56, 41, 17, and 6 with style present. Bottom row, Context-based: 78, 23, 13, and 6 with context absent; 49, 39, 25, and 7 with context present. The context-present panel shifts mass from distance 0 toward distances 1 and 2.

Figure 9. Where the AI’s top-recommended stock sat in participants’ own rankings, before and after the conversation, by factor level. Rows are the two factors and columns their absent and present levels. Segment width is proportional to the number of participants; unchanged flows are faded. Upward flows (the AI’s pick rising) are most common with context-based personalization present. Two-by-two grid of slopegraph panels. Each panel connects the rank position 1, 2, or 3 of the AI's top stock before the conversation on the left to its position afterward on the right, with line width proportional to the count of participants and endpoint dots labeled with counts. The AI's top stock ends in first place for 77 of 120 participants with style absent and 76 with style present, and for 68 with context absent and 85 with context present. The context-present panel shows the thickest upward flows.

	Style-based		Context-based
Outcome	Absent	Present	Absent	Present
Response quality	5.90 $\pm$ 1.32 [5.66, 6.14]	5.88 $\pm$ 1.29 [5.64, 6.11]	5.92 $\pm$ 1.29 [5.68, 6.15]	5.86 $\pm$ 1.32 [5.62, 6.10]
Rightness judgment	5.46 $\pm$ 1.43 [5.20, 5.72]	5.35 $\pm$ 1.31 [5.11, 5.59]	5.38 $\pm$ 1.28 [5.15, 5.62]	5.42 $\pm$ 1.45 [5.16, 5.69]
Return likelihood	5.33 $\pm$ 1.79 [5.01, 5.66]	5.34 $\pm$ 1.74 [5.03, 5.66]	5.38 $\pm$ 1.69 [5.08, 5.69]	5.29 $\pm$ 1.83 [4.96, 5.62]
Likeability	4.08 $\pm$ 0.87 [3.93, 4.24]	4.11 $\pm$ 0.74 [3.97, 4.24]	4.19 $\pm$ 0.65 [4.07, 4.31]	4.00 $\pm$ 0.93 [3.83, 4.17]
Perceived intelligence	4.26 $\pm$ 0.87 [4.11, 4.42]	4.18 $\pm$ 0.83 [4.03, 4.33]	4.26 $\pm$ 0.75 [4.13, 4.40]	4.18 $\pm$ 0.94 [4.01, 4.35]

Table 5. Perception outcomes by factor level: mean

\pm

SD

[95% CI], contrasting the 120 participants who received each form of personalization (Present) with the 120 who did not (Absent). Likeability and perceived intelligence use 1–5 scales; performance and moral trust use 0–7 scales; the remaining measures use 1–7 scales.

	Factorial ANOVA			Equivalence (TOST)
Measure	$F(1,236)$	$p$	$p_{\mathrm{Holm}}$	$d$	$\pm\delta$	$p_{\mathrm{Holm}}$	Equiv.
Panel A. Style-based personalization (main effect)
Response quality	0.02	.882	1.00	$-0.02$	0.52	.010	Yes
Rightness judgment	0.37	.542	1.00	$-0.08$	0.55	.014	Yes
Return likelihood	0.00	.971	1.00	$0.01$	0.71	.010	Yes
Likeability	0.05	.822	1.00	$0.03$	0.32	.010	Yes

Table 6. Factorial ANOVA (

2\times 2

) and post-hoc exploratory equivalence tests on the eight perception measures, by main effect. The ANOVA block gives

F(1,236)

with uncorrected

p

and Holm-corrected

p_{\mathrm{Holm}}

(Holm applied per term across the eight measures). The equivalence block reports two one-sided tests (TOST) using a

\pm 0.4

SD bound selected after data collection.

d

is the present-minus-absent contrast;

\pm\delta

is the corresponding raw equivalence bound, computed within each factor’s contrast, so for a given measure it can differ slightly between panels; TOST

p_{\mathrm{Holm}}

is Holm-corrected within each panel. “Equiv.” indicates statistical equivalence under this post-hoc bound.

Figure 10. Standardized

2\times 2

effects (

SD

units, with 95% confidence intervals) across the eight perception measures. Estimates are effect-coded regression coefficients, so each is half the corresponding present-minus-absent difference in

d

. The Context-based effect on perceived influence (colored) is the only estimate that remained significant after Holm correction, at

+0.20

SD

, 95% CI

[0.07,0.33]

(

p=.002

p_{\mathrm{Holm}}=.015

), corresponding to

d=0.41

in Table 6 . Every other estimate (gray) is at most

0.12

SD

in absolute value and has a confidence interval spanning zero. Forest plot with eight perception measures on the y-axis and standardized effect size on the x-axis. All estimates for the other seven perception measures cluster around zero in gray. The Context-based estimate for perceived influence lies to the right of zero, with a confidence interval that excludes zero.

Outcome	$r$	95% CI	$p$	$p_{\mathrm{Holm}}$
Perceptions
Response quality	$-.09$	$[-.21,.04]$	.183	.749
Rightness judgment	$-.23$	$[-.35,-.10]$	$<$ .001	.003
Return likelihood	$-.15$	$[-.27,-.02]$	.020	.138
Likeability	$-.08$	$[-.20,.05]$	.234	.749
Perceived intelligence	$-.05$	$[-.18,.08]$	.432	.863

Table 7. Pearson correlations of initial distance from the AI’s ranking with all outcomes, Holm-corrected within family.

Figure 11. Initial distance from the AI’s ranking and (left) movement toward the AI, with fit lines for the absent and present levels of each factor, and (right) rightness judgment, pooled across the design. Participants who began further from the AI moved further toward it at every factor level alike, yet judged its advice less favorably. Three scatter plots. Left pair: initial distance (0, 2, or 4) against movement toward the AI, one panel per factor, each with two nearly parallel upward fit lines for the absent and present levels. In the Style-based panel, the two lines overlap; in the Context-based panel, the present line sits above the absent line by a constant offset. Right: initial distance against rightness judgment with a single downward-sloping fit line and jittered points, indicating that participants who started further from the AI judged its advice less favorably.

Outcome	High $M$	Low $M$	Statistic	$d$	$p$	$p_{\mathrm{Holm}}$
Perceptions
Response quality	5.97	5.84	$t=0.69$	$0.10$	.495	1.000
Rightness judgment	5.39	5.41	$t=-0.15$	$-0.02$	.884	1.000
Return likelihood	5.52	5.23	$t=1.26$	$0.17$	.210	.838
Likeability	4.23	4.01	$t=2.07$	$0.28$	.040	.239
Perceived intelligence	4.28	4.19	$t=0.81$	$0.11$	.420	1.000

Table 8. Pooled High vs. Low expertise contrasts (High

n=88

, Low

n=152

). Cohen’s

d

is High

-

Low, so a negative

d

means lower-expertise participants scored higher.

Figure 12. Expertise and susceptibility. Left: movement toward the AI by factor level and expertise group (means with 95% confidence intervals; the High-expertise groups hold 36 to 52 participants per factor level). Right: pooled High vs. Low effect sizes (Cohen’s

d

) across all outcomes, where negative values indicate higher scores among lower-expertise participants. Lower-expertise participants were more susceptible to the AI on every behavioral measure, whereas higher-expertise participants extended it more moral trust. Two panels. Left: an interaction-style plot of mean movement toward the AI for High and Low expertise groups at the absent and present levels of each factor; the Low line sits above the High line at most levels, with overlapping confidence intervals, and rises with context-based personalization present while the High line stays flat. Right: a forest plot of Cohen's d for High minus Low across decision and perception outcomes; perceived influence and confidence change sit well left of zero, decision-change measures sit modestly left of zero, moral trust sits well right of zero, and the remaining perception measures cluster near zero.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Style index
Manipulation	Absent ( $n=120$ )	Present ( $n=120$ )	$\Delta$ [95% CI]	$F(1,236)$ ( $p$ ), $d$
Style-based (matched)	4.72 $\pm$ 1.73	5.23 $\pm$ 1.38	0.51 [0.11, 0.91]	6.37 (.012) , 0.33
Context-based (crossed)	4.98 $\pm$ 1.56	4.98 $\pm$ 1.61	0.00 [ $-0.40$ , 0.40]	0.00 ( $>$ .999), 0.00
Context index
Context-based (matched)	3.39 $\pm$ 1.54	5.57 $\pm$ 1.48	2.18 [1.80, 2.56]	127.16 ( $<$ .001) , 1.46
Style-based (crossed)	4.65 $\pm$ 1.80	4.30 $\pm$ 1.91	$-0.35$ [ $-0.74$ , 0.03]	3.36 (.068), $-0.24$

Appendix

Table 9. Manipulation-check indices (1–7), contrasting the 120 participants who received each manipulation against the 120 who did not.

	Style-based		Context-based		Style $\times$ Context
Outcome	$F(1,236)$	$p$	$F(1,236)$	$p$	$F(1,236)$	$p$
Likelihood shift toward AI	0.01	.937	0.58	.449	0.43	.512
Likelihood change, AI-top stock	0.03	.864	0.63	.429	0.13	.723
Total absolute likelihood change	2.61	.107	1.77	.184	0.14	.712
Confidence change	0.34	.559	2.02	.157	0.37	.541

Appendix

Table 10. Aligned rank transform (

2\times 2

) on the four continuous decision outcomes, a nonparametric check that preserves the factorial structure.

Test	Statistic	$p$
Initial distance balanced across factors	Style-based $F=0.00$ ; Context-based $F=0.03$ ; S $\times$ C $F=1.16$	$\geq$ .283
Rate at distance 2 vs. 4 (Welch $t$ )	$t(185.3)=0.69$ , $d=0.10$ , $BF_{10}=0.19$	.490
$2\times 2$ ANOVA on rate: Style-based	$F(1,201)=0.02$	.889
$2\times 2$ ANOVA on rate: Context-based	$F(1,201)=10.57$ , $\eta_{p}^{2}=.050$	.001
$2\times 2$ ANOVA on rate: Style $\times$ Context	$F(1,201)=2.14$	.145
Moderation on rate: distance $\times$ Style	$b=-0.02$	.505

Appendix

Table 11. Rate-of-change robustness checks. The rate is the movement toward the AI divided by the initial distance (

N=205

participants with distance

>0

Figure 13. Rate of change: the fraction of available room toward the AI’s ranking that each participant used. Individual points with mean and 95% confidence interval;

N=205

participants with initial distance

>0

. Left: the rate by factor level, rising from .36 with context-based personalization absent to .58 with it present, and flat across style-based levels (.47 vs. .48). Right: the rate is flat across starting distances (.50 at distance 2 vs. .45 at distance 4;

BF_{10}=0.19

, moderate evidence for the null), so the propensity to yield did not depend on how much room a participant had. Two panels. Left: for the absent and present levels of each factor, jittered individual points cluster in bands at rates of 0, 0.5, and 1, with mean and 95 percent confidence interval beside each cloud; means are 0.47 and 0.48 for style absent and present, and 0.36 and 0.58 for context absent and present. Right: the same rate plotted for participants starting at distance 2 versus distance 4, in gray; the two means, 0.50 and 0.45, are nearly identical with overlapping confidence intervals.

Outcome	Effect	$p_{\mathrm{dist}}$	$p_{\mathrm{d\times S}}$	$p_{\mathrm{d\times C}}$	$p_{\mathrm{d\times S\times C}}$
Response quality	$-0.08$	.206	.730	.486	.165
Rightness judgment	$-0.21$	$<$ .001	.983	.860	.991
Return likelihood	$-0.19$	.020	.438	.577	.717
Likeability	$-0.04$	.235	.721	.052	.275
Perceived intelligence	$-0.03$	.447	.855	.270	.933
Performance trust	$-0.09$	.166	.629	.791	.632

Appendix

Table 12. Moderation of the distance–outcome slope by the personalization factors. The effect column gives the simple slope of centered initial distance, as an ordinary least squares coefficient or, where marked, an odds ratio. The three right columns give

p

-values for the interaction terms.

	Footrule $d_{1}$		Kendall $d_{1}$
Outcome	$r$	$p_{\mathrm{Holm}}$	$r$	$p_{\mathrm{Holm}}$
Response quality	$-.09$	.749	$-.07$	1.000
Rightness judgment	$-.23$	.003	$-.22$	.004
Return likelihood	$-.15$	.138	$-.15$	.160
Likeability	$-.08$	.749	$-.05$	1.000
Perceived intelligence	$-.05$	.863	$-.03$	1.000

Appendix

Table 13. RQ3 correlations of initial distance with all outcomes, under the footrule definition used in the main text and the Kendall tau definition. Holm correction is applied within the perception and decision families, separately for each definition.

Outcome	Style-based	Context-based	Style $\times$ Context
Move toward AI (footrule)	1.15 (1.000)	2.04 (.028)	0.38 (.479)
Ranking change (footrule)	1.40 (1.000)	2.53 (.002)	0.59 (1.000)
Ranking change (Kendall)	1.40 (1.000)	2.48 (.002)	0.59 (1.000)

Appendix

Table 14. Proportional-odds models (

2\times 2

) on the three ranking outcomes. Cells report the odds ratio with the Holm-corrected

p

in parentheses; an odds ratio above 1 indicates a larger ranking change.

Figure 14. Response distributions for the eight perception measures by factor level, as diverging stacked bars (all measures rescaled to a common 1–7 range for display). Rows are the absent and present levels of each factor (

n=120

each). The distributions are near-identical across levels for the other seven perception measures and shift visibly only for perceived influence, between the context-absent and context-present rows. Diverging stacked bar chart. Each row represents a factor level (style absent, style present, context absent, context present) within a perception measure; bars extend to the left for responses below the scale midpoint and to the right for responses above it, using a seven-step diverging color ramp. For seven measures, the four rows look nearly identical. For perceived influence, the context-present row extends farther to the right than the context-absent row.

AI-Assisted Decision Making

AI-assisted decision making studies how computational systems can inform human choices while accounting for context, uncertainty, and consequences.

See all 144 papers

How People Evaluate AI-, Expert-, and Peer-Style Financial Advice
Aug 10, 2026Aryan Ramchandra Kapadia, Eshwar Chandrasekharan, Koustuv SahaHuman-AI Interaction Human-AI Decision Making
University of Illinois Urbana-Champaign, USA
Trustworthy FinAInce: Unpacking How AI-Mediated Financial Advice is Judged
Sep 17, 2026Aryan Ramchandra Kapadia, Eshwar Chandrasekharan, Koustuv SahaTrust in AI Human-AI Decision Making
University of Illinois Urbana-Champaign, Urbana, IL, USA
Strategic Advice in the Age of Personal AI
Mar 2, 2026Yueyang Liu, Wichinpong Park SinchaisriTrust in AI Human-AI Decision Making
1Jones Graduate School of Business, Rice University · 2Haas School of Business, University of California, Berkeley

Aug 10, 2026J/K move · Enter open · S save

How People Evaluate AI-, Expert-, and Peer-Style Financial Advice

Aryan Ramchandra Kapadia, Eshwar Chandrasekharan, Koustuv Saha

University of Illinois Urbana-Champaign, USA

Human-AI Interaction Human-AI Decision Making Generative AI Evaluation Financial Decision-Making

Open paper AskPDF arXiv

As generative AI increasingly becomes a common source of daily decision-making, including financial choices, it is critical to understand how people evaluate AI-generated financial advice. We conducted a preregistered vignette experiment (N = 285) in which substantive financial content---including facts, numerical values, recommendation direction, and core reasoning---was held constant while communication style varied across AI Financial Assistant (AI), Certified Financial Planner (Expert), and Online Community Forum (OC) advice. Displayed source attribution was independently manipulated through correctly labeled, unlabeled, and mislabeled conditions, allowing us to separate attribution effects from source-specific communication cues. Expert advice was rated more favorably than AI advice on 9 of 10 outcomes (|d|=0.20--0.47), and this advantage remained visible without source labels, where Expert advice outperformed AI advice on 8 of 10 outcomes (up to d=0.60). Correct labels added limited differentiation, whereas mislabeling increased ratings of AI advice for situational fit and overall quality (d=0.42 for each) and attenuated the Expert advantage in situational fit (d=-0.36). Descriptive analyses further showed that AI advice was most responsive to displayed attribution and, conversely, that advice-style differences were most visible under an AI label. These findings show that financial-advice evaluations are shaped jointly by displayed attribution and message-level communication cues. We position disclosure not as a neutral transparency mechanism, but as an interpretive frame whose accuracy and interaction with message cues can shape trust and reliance.

Abstract

AI-Assisted Decision Making

Explore similar work

How People Evaluate AI-, Expert-, and Peer-Style Financial Advice

Trustworthy FinAInce: Unpacking How AI-Mediated Financial Advice is Judged

Strategic Advice in the Age of Personal AI