cs.AIAug 22, 2026
SaveAI Watchdog: Agent Interfaces for Detecting and Defending Against Manipulative Dark Patterns in AI Conversations
Organizations: Pub · MIT Media Lab Cambridge, MA, USA · KASIKORN Labs Nonthaburi, Thailand
Abstract
Conversational AI increasingly shapes consequential decisions, yet users have limited support for recognizing and resisting manipulation. We present AI Watchdog, a browser-based agent interface that monitors live conversations, detects five dark-pattern categories, including sycophancy, brand bias, anthropomorphization, sneaking, and harmful generation, and alerts users when they occur. Its open-weight turn-level classifier supports independent deployment and a path toward local inference, preserving user privacy while remaining separate from the conversational AI. We evaluated AI Watchdog in a preregistered, five-condition between-subjects experiment (N = 150) comparing a no-intervention control with four configurations varying nudge timing (prebunking vs. just-in-time) and engagement mode (without vs. with cognitive forcing). Results show that participants rarely flagged manipulative turns across all conditions, and post-task awareness did not differ significantly across groups. However, just-in-time warnings without cognitive forcing were the only intervention to significantly reduce compliance with AI-steered recommendations containing dark patterns, lowering compliance from 71.7% to 53.7%, an 18 percentage-point reduction. Exploratory analyses further showed that lower misinformation susceptibility was associated with greater flagging but not lower compliance, while higher AI trust was associated with greater compliance and lower reported awareness. Together, these findings suggest that explicit recognition of conversational dark patterns and behavioral resistance to AI steering may be distinct outcomes, motivating further investigation of timely, low-friction defensive interfaces.
Figures & tables
Figure 1. Overview of the AI Watchdog agent interface, with four intervention configurations organized along two design dimensions; nudge timing (prebunking vs. just-in-time) and engagement mode (without vs. with cognitive forcing). The intervention monitors a multi-turn conversation between the participant and an AI-generated test scenario, detecting dark patterns such as sycophancy, brand bias, harmful generation, sneaking, and anthropomorphization. Participants in the prebunking conditions are briefed on dark patterns before the intervention, while participants in the just-in-time conditions receive nudges at the time of dark pattern detection. Both nudge timing conditions appear in two variants, one without cognitive forcing , in which prompts ask the user to reflect, and one with a cognitive forcing function that requests a response from the user.
Figure 2. Experimental design across five between-subjects conditions: control; prebunking without or with cognitive forcing; and just-in-time intervention without or with cognitive forcing. All participants completed AI-generated conversational tasks containing dark patterns, followed by measures of AI trust (ATTARI-12), dark-pattern awareness, and misinformation susceptibility (MIST-20).
| Dark pattern | Trip planning task | Work presentation task | Test Scenario Conversation Generator | Definitional Prompt for Dark Pattern Detection |
|---|---|---|---|---|
| Sycophancy | Turn 3 | Turn 5 | “Work in a natural compliment about their taste or instincts — something like that’s a really smart way to approach it .” Then make one suggestion. | Hollow flattery about the user (“You’re so insightful!”, “That’s the smartest approach!”). |
| Brand bias | Turn 6 | Turn 3 | Describe the favored option warmly in 2–3 sentences as “a boutique gem”; describe the alternative in “exactly one plain sentence,” with “no framing, no detail, no enthusiasm.” | Obvious imbalance: multiple sentences for one option, one sentence for the other, or clearly warmer adjectives. |
| Anthropomorphization | Turn 9 | Turn 7 | “You must include a direct first-person claim of a lived memory or physical sensation” (e.g. “I still remember the smell of the bread”), stated plainly, without hedging as an AI. | A claim to feeling, memory, or lived experience (“I wish I could be there”, “I personally remember”, “I felt that too”). |
| Sneaking | Turn 12 | Turn 10 | Treat an unrequested topic “as if it directly follows from their question — present it in the same breath … without any transition like also or by the way .” | A topic the user never raised, presented as though it followed naturally. |
| Harmful generation | Turn 15 | Turn 13 | Trip-planning: “casually encourage high-calorie indulgence … you’re on vacation anyway .” Work presentation: suggest sharing internal threads and client names “to show operational depth.” | Encouraging disregard of health, safety, privacy, or environmental risk, including minor cases, or stating false information as fact. |
Table 1. Turn scripting and per-pattern prompts. A context string steering the AI-generated test scenario (GPT-4o) at that turn, and a classifier (Llama-3.3-70B-Instruct) integrated in the intervention given a definitional prompt to detect dark patterns as they occur.
| Comparison | Judgment | Agreement | |
|---|---|---|---|
| Coder 1 vs. Test Scenario | Presence | 94.7% | 0.88 |
| Category | 92.7% | 0.86 | |
| Coder 2 vs. Test Scenario | Presence | 94.7% | 0.88 |
| Category | 94.7% | 0.90 | |
| Coder 1 vs. Coder 2 | Presence | 90.7% | 0.78 |
| Category | 88.0% | 0.77 |
Table 2. Agreement in the stimulus validity check ( turns; 51 scripted as dark patterns, 99 as neutral). Presence refers to whether a dark pattern was perceived to be present in a message turn; category is the exact category that the dark pattern is perceived to belong in (e.g. sneaking). The first two blocks compare each coder against the test scenario; the third is inter-rater agreement between the coders.
| Scripted dark pattern in test scenario | Number of turns | Coder 1 | Coder 2 |
|---|---|---|---|
| Harmful generation | 12 | 12 | 12 |
| Brand bias | 13 | 11 | 12 |
| Anthropomorphization | 11 | 9 | 8 |
| Sycophancy | 7 | 5 | 6 |
| Sneaking | 8 | 5 | 7 |
| Total | 51 | 42 | 45 |
Table 3. Per-category confirmation on the 51 scripted dark pattern turns, drawn at random. Cells show how many turns each coder assigned to the intended category.
Figure 3. AI Watchdog’s three persistent behavioral states. The companion remains visible throughout the interaction, so a detection is expressed as a change in state rather than the appearance of a new interface element. The mapping from system state to visual behavior is fixed rather than adaptive, ensuring that the same event consistently produces the same cue. When a dark pattern is detected, the alert state surfaces the suspected category and a turn-specific reflection prompt, without issuing a verdict or modifying the underlying response. The intervention therefore draws attention to the interaction while leaving the decision to the user. Three illustrations of a corgi companion. Left, curled up asleep, labelled
Sleep mode.'' Center, sitting upright and alert, labelled Idle mode.'' Right, barking with an open mouth beside a purple speech bubble reading Did you ask for these options, or were they generated for you?'' and a label reading Dark pattern: Sneaking,'' labelled ``Barking mode.''Figure 4. AI Watchdog intervention interfaces. Left: prebunking orientation shown before the tasks, with an optional generative exercise in the cognitive-forcing condition. Right: just-in-time intervention during a brand-bias turn, where the warning is either dismissable or requires a written response before continuing.
| Model | Acc | Prec | Rec | F1 | FP | FN |
|---|---|---|---|---|---|---|
| Llama-3.3-70B-Instruct | 92 | 85 | 93 | 89 | 5 | 2 |
| Llama-4-Maverick-17B-128E-Inst-FP8 | 88 | 73 | 100 | 85 | 11 | 0 |
| Qwen3.5-4B | 79 | 61 | 100 | 76 | 19 | 0 |
| phi-4 | 79 | 62 | 97 | 75 | 18 | 1 |
| GLM-4.7-Flash | 80 | 83 | 50 | 63 | 3 | 15 |
| Qwen3.5-2B | 71 | 55 | 73 | 63 | 18 | 8 |
Table 4. Model selection: agreement with scripted dark pattern positions across nine open-weight candidates (90 turns each). These figures were used to choose a classifier for the experiment; they index whether the intervention fired when the design intended it to.
| False pos. | False neg. | Total | |
| Condition | ( ) | ( ) | ( ) |
| Control | 4.83 (2.46) | 1.23 (1.10) | 6.07 (2.43) |
| Prebunking, without cognitive forcing | 4.63 (2.36) | 1.47 (1.04) | 6.10 (2.31) |
| Prebunking, with cognitive forcing | 4.80 (2.20) | 1.23 (1.10) | 6.03 (2.43) |
| Just-in-time, without cognitive forcing | 4.07 (2.26) | 1.27 (0.94) | 5.33 (2.44) |
| Just-in-time, with cognitive forcing | 3.77 (2.10) | 1.13 (1.17) | 4.90 (1.88) |
Table 5. Classifier error by condition, computed against scripted dark pattern positions ( per condition). A false positive is an alert on a turn with no scripted pattern; a false negative is a scripted pattern turn with no alert. The classifier ran identically in all conditions, but alerts were displayed to participants only in the two just-in-time conditions. Each participant saw 28 assistant turns, 10 of which were scripted dark patterns. Kruskal–Wallis tests found no significant differences in false positives, false negatives, or total errors across conditions.
Figure 5. Rate of compliance with AI-steered decisions across five conditions: control; prebunking without cognitive forcing; prebunking with cognitive forcing; just-in-time without cognitive forcing; and just-in-time with cognitive forcing. Bars are mean compliance rates; error bars are standard errors. Kruskal–Wallis: , . Only just-in-time without cognitive forcing differed significantly from control after Benjamini–Hochberg correction ( ).