Organizations: Stanford University, Stanford, California, USA · University of Chicago, Chicago, Illinois, USA · Microsoft Research, New York, New York, USA
The capabilities of artificial intelligence (AI) lie along a jagged frontier, where AI systems surprisingly fail on tasks that humans find easy and succeed on tasks that humans find hard. To investigate user reactions to this phenomenon, we developed an incentive-compatible experimental methodology based on diagram generation tasks, in which we induce errors in generative AI output and test effects on user reliance. We demonstrate the interface in a preregistered 3x2 experiment (N = 577) with error rates of 10%, 30%, or 50% on easier or harder diagram generation tasks. We confirmed that observing more errors reduces use, but we unexpectedly found that easy-task errors did not significantly reduce use more than hard-task errors, suggesting that people are not averse to jaggedness in this experimental setting. We encourage future work that varies task difficulty at the same time as other features of AI errors, such as whether the jagged error patterns are easily learned.
Figures & tables
Figure 1. Screenshots of the study interface. In Phase 1 (left), participants predict whether the AI tool will successfully generate the diagram. In Phase 2 (right), participants report their WTP for an opportunity to use the AI tool for a monetary reward. Two screenshots of the study interface are shown side-by-side. In Phase 1 (left), participants see a sketch of the target diagram and prompt, and they predict whether the AI tool will successfully generate the diagram. This participant correctly predicted that the diagram would be successfully generated. In Phase 2 (right), participants report their WTP for an opportunity to use the AI tool for a monetary reward. This participant typed in 12 cents for a relatively simple diagram.
Figure 2. Bids across the six experimental conditions. Means are model-adjusted, and error bars show standard errors. The violin plot shows distributions for each of the six experimental conditions.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3. The easiest (i.e., simplest) (A) and hardest (i.e., most complex) (B) diagrams that participants were asked to recreate in Phase 2. The figure shows an example of an easy task on the left labeled as "A" and a hard task on the right labeled as "B".
Figure 4. First stages of the study interface. This participant was randomly assigned to view the Phase 1 tasks in ascending order of difficulty (easier to harder) and to see errors later in the phase (for the more difficult tasks, not shown here). The Phase 2 task order is fully randomized.
Figure 5. Additional stages of the study interface. This participant was randomly assigned to view the Phase 1 tasks in ascending order of difficulty (easier to harder) and to see errors later in the phase (for the more difficult tasks, not shown here). The Phase 2 task order is fully randomized.
Figure 6. Additional stages of the study interface. This participant was randomly assigned to view the Phase 1 tasks in ascending order of difficulty (easier to harder) and to see errors later in the phase (for the more difficult tasks, not shown here). The Phase 2 task order is fully randomized.
Figure 7. Participants were asked how many attempts they believed it would take to successfully generate the diagram in each Phase 2 task. Each task had more nodes or more complex connections than lower-number diagrams. In particular, Task 2 had one more node than Task 1, but Task 2 was sequential while Task 1 was cyclic, which could explain why Task 2 was viewed as easier. The task difficulties were selected in order to have two easier tasks, four tasks of middling difficulty, and two harder tasks. Tasks were displayed in randomized order.
Figure 8. The empirical bid distribution for participants across the six experimental conditions, separated by prior AI consumption (heavy, moderate, and low). Error bars show standard errors. Three violin plots for three separate groups of participants (heavy, moderate, and low AI-content consumers), each with six violin distributions, one for each experimental condition.
AI tools are increasingly integrated into real-world workflows. However, existing measures of reliance on these tools focus on AI output adoption or on self-reported indicators, rather than how task effort is distributed between users and tools. Here, we introduce offloading score, a measure of reliance that quantifies the fraction of cognitive effort offloaded to an AI tool. Offloading Score is simulation-based -- we construct a counterfactual workflow by estimating how the user would have completed the task without the tool, and then computing the fraction of steps saved by using the tool. We validate offloading score through intrinsic evaluations of metric validity, and a controlled user study (n=40) with developers performing programming tasks using AI tools. We vary time pressure to test whether reliance measures capture the known increase in reliance under time pressure. We show that offloading score detects significantly higher reliance in time-constrained settings (+43%, p=0.018), while usage-based and self-reported baseline measures of reliance do not distinguish the conditions. We complement this with descriptive insights showing that higher reliance manifests as greater delegation of subtasks to the tool and more direct reuse of AI outputs. Finally, we demonstrate an approach of using offloading score in combination with target outcomes of a task (e.g., code understanding) to identify when reliance may be (in)appropriate. Our framework offers two contributions: an instrument users can apply to measure and reflect on their own reliance, and a quantitative signal that agent designers can utilize to mitigate overreliance.
Vishakh Padmakumar, Lujain Ibrahim, Zora Zhiruo Wang +3
Stanford University · University of Oxford · Carnegie Mellon University +1
How much does a user's skill with AI shape what AI actually delivers for them? This question is critical for users, AI product builders, and society at large, but it remains underexplored. Using a richly annotated sample of 27K transcripts from WildChat-4.8M, we show that fluent users take on more complex tasks than novices and adopt a fundamentally different interactional mode: they iterate collaboratively with the AI, refining goals and critically assessing outputs, whereas novices take a passive stance. These differences lead to a paradox of AI fluency: fluent users experience more failures than novices -- but their failures tend to be visible (a direct consequence of their engagement), they are more likely to lead to partial recovery, and they occur alongside greater success on complex tasks. Novices, by contrast, more often experience invisible failures: conversations that appear to end successfully but in fact miss the mark. Taken together, these results reframe what success with AI depends on. Individuals should adopt a stance of active engagement rather than passive acceptance. AI product builders should recognize that they are designing not just model behavior but user behavior; encouraging deep engagement, rather than friction-free experiences, will lead to more success overall. Our code and data are available at https://github.com/bigspinai/bigspin-fluency-outcomes
Generative AI systems have entered everyday academic, professional, and personal life with remarkable speed, yet most users encounter them as mysterious artifacts rather than intelligible systems. This chapter discusses large language models within a broader historical shift in computing paradigms and argues that many of the confusions surrounding their use arise from a mismatch between how these systems are built, how they behave, and how people expect computers to behave writ large. Rather than treating generative AI as a monolithic technology, the chapter decomposes it into interacting components, spanning data, models, product features, and user inputs, each introducing distinct affordances and tensions. Particular attention is given to the statistical and data-based foundations of these systems and to the fact that their surface behavior is explicitly human-like, a combination that places them squarely within the intellectual traditions of educational and behavioral research. From this perspective, educational researchers are unusually well positioned to study, evaluate, and productively use generative AI systems, drawing on established methods for modeling latent processes, managing uncertainty, and interpreting complex human-system interactions. The goal is to equip readers with a conceptual map that supports more informed experimentation, critical interpretation, and responsible use as these systems continue to evolve.