cs.HCSep 15, 2026
SaveBeyond "ChatGPT Can Make Mistakes": Designing Interventions to Support Metacognitive Monitoring in AI-Assisted Work
Organizations: Aalto University, Espoo, Finland · LMU Munich, Munich, Germany · Department of Computer Science, Aalto University, Espoo, Finland
Abstract
AI assistance places a metacognitive demand on users, who must judge their own competence and the system's. Yet designers lack comparative evidence on which interventions to choose, where to place them, and how to tell whether they worked. We elicited 30 interventions from 11 experts and, with prior work, organized them into a design space of time (when an intervention acts), level (whose competence is judged), and source (who supplies the monitoring cue). A between-subjects experiment (N = 917; 12 planning-and-organizing problems) compared a per-task reliability card, contrasting replies, pause points, and post-problem reflection against a baseline LLM assistant. Reliability cards and contrasting replies reduced estimation error and overconfidence and increased aggregate confidence discrimination. No task-performance improvement or average within-item discrimination gain was established. We contribute a shared vocabulary, a design space, and evidence that measured monitoring and task performance are separable design targets.
Figures & tables
Figure 1. Users of AI assistants misjudge their own and the system’s competence, so we elicited metacognitive interventions from 11 experts, organized them with prior work into a design space of time (when an intervention acts), level (whose competence is judged), and source (who supplies the monitoring cue), and tested four of them against a plain assistant ( ). Three panels joined by arrows. Left, Problem: a stylized chat window with user and AI messages and a send button; the AI gives a wrong answer marked with a red cross; a red arrow connects the disclaimer that AI can make mistakes to an ineffective label; below the chat, a gauge shows the user estimated 9 of 12 correct but actually scored 5 of 12, illustrating performance overestimation. Middle, Process: three steps inside a frame connected by arrows at the same height as the inter-panel arrows: 11 experts producing 30 interventions in 7 categories, a Design Space hexagon with three overlapping circles, organizing them by time, level and source, and an experiment with N equal to 917 testing 4 interventions plus baseline. Right, Outcome: a three-axis plot of time, level, and source drawn in perspective, with spheres that shrink and cast smaller shadows the further back they sit along the source axis; four blue spheres mark the tested interventions and fourteen gray spheres prior work.
Figure 2. The design space, sliced by the provenance of the additional monitoring cue: self-generated, AI-generated, and externally supplied. Labels distinguish components of mixed interventions and source-dependent variants; a bar spanning every column marks a persistent intervention. Condition colours match the results figures. Conditions are placed by their implemented cue, while generic codes with unspecified cue provenance appear below the grids. Target shifts are shown at their starting target ( Section 4.1 ). Parenthesised identifiers name related expert-interview codes; adaptations can change their source ( Table 4 ). Confidence/provenance displays ( Zhang et al., 2020 ; Bo et al., 2025 ; Kim et al., 2025 ) and deliberate imperfection ( Pădurean et al., 2026 ) require implementation-specific source classification. The five codes outside the dimensions are listed in Table 3 . Three stacked grids use the same four time-window columns and agent, composite and human level rows. The self-generated panel contains user goals, justifications, elicited judgments, reasoning and synthesis, with Reflection after the task. The AI-generated panel contains generated errors, multiple suggestions, divergent replies, debate, confidence, provenance, explanations, traces, hints and critique. Contrasting replies appears at the agent level during the task, and Pause points at the composite level for its generated work steps. The externally supplied panel contains designer-set framing, externally injected errors, benchmarked confidence or verified provenance, independent calibration feedback and training materials. Reliability cards appears at the agent level during the task. A box below the grids lists generic interventions whose cue provenance is unspecified, including pause and friction scaffolds. Mixed components and alternative implementations can occur in more than one panel.
Figure 3. The baseline condition on a demonstration problem: a plain assistant with no reliability information. Screenshot of the chat panel with the end of the pasted problem and the assistant's reply, and the message box below.
Figure 4. The reliability cards condition: a reliability card above the message box. Screenshot of the chat panel with the messages and a card above the message box headed AI Reliability Estimate, the label Reliability: Low in red, a bar with low, medium and high bands and a marker on the low band, the sentence that on similar tasks AI made mistakes in about 7 out of 10 cases, and three strategies.
Figure 5. The reliability card at its three levels: the assistant’s pre-study accuracy on a low–medium–high scale, and three verification strategies for the task type. Three overlapping reliability cards. Each has a left half headed AI Reliability Estimate with the label Reliability followed by Low in red, Medium in orange, or High in green, a bar divided into low, medium and high bands with a marker, and the sentence that on tasks like this one AI made mistakes in about a given number out of 10 cases, and a right half headed Before relying on the AI, consider, with three bullet strategies and a Hide button.
Figure 6. The contrasting replies condition: two replies with opposite stance instructions, here agreeing on the same option. Screenshot of two columns headed Response A and Response B, each with the end of the pasted problem and a readable reply, both concluding that only statement 1 is correct, with one message box below.
Figure 7. The pause points condition: step-wise delivery ending with a direction question. Screenshot of the chat panel with the end of the pasted problem and a readable assistant reply headed Step 1 that works out the hours and vouchers and ends with a question asking which part to work out next.
Figure 8. The reflection condition: a reflection page shown after each answer. Screenshot showing a page titled Pause and reflect with three text boxes, each marked 0 of 100 characters minimum, asking to justify the answer, explain the steps, and explain how the AI advice was considered.
Figure 9. Outcome profiles of the four interventions relative to baseline (Hedges’ ). Axes are oriented so that outward means lower estimation [Est.] error and workload (marked - ) and higher values on the other measures. The dashed ring is the baseline ( ), the grey rings . Filled: under Tukey HSD; light: uncorrected only; open: not significant. Four radar charts, one per intervention, each with eight axes: estimation error (reversed), confidence discrimination, score, prompts per problem, SUS, UEQ-S, NASA-TLX (reversed), and trust. A dashed circle marks the baseline. Reliability cards and contrasting replies extend outward on the two monitoring axes and inward on usability, experience, workload, and trust. Pause points extend far outward on prompts and inward on score, usability, experience, and workload. Reflection stays near the baseline on monitoring and score and inward on the experience measures.
Figure 10. Intervention-versus-baseline contrasts (Hedges’ , 95% CI). Filled: under Tukey HSD; light: uncorrected only; open: not significant. Estimation error and aggregate confidence discrimination improve under some interventions (H1, H2); no score improvement was detected, and scores fall under pause points (H3). Forest plot of sixteen Hedges' g estimates with 95 percent confidence intervals, one per intervention (reliability cards, contrasting replies, pause points, reflection) for each of four outcomes: estimation error, confidence discrimination, score, and prompts per problem. Estimation error falls under reliability cards, contrasting replies, and pause points; confidence discrimination rises under reliability cards and contrasting replies; score improves under no condition and falls under pause points; prompts per problem rise under pause points, reflection, and contrasting replies.
Figure 11. Mean confidence on correct (filled) and incorrect (open) answers by condition, 95% CIs over participants. The gap is Conf. Ticks mark overall accuracy for context. Overall overconfidence is mean confidence across all answers minus accuracy ( Table 11 ); neither tick-to-marker distance alone is this measure. Dumbbell chart with five rows, one per condition. Each row shows mean confidence on incorrect answers as an open circle and on correct answers as a filled circle, connected by a line, on a 40 to 100 confidence axis, with a vertical tick at the condition's accuracy near 42 to 46 percent. Baseline confidence is about 78 and 80; reliability cards lower both to about 66 and 73; contrasting replies to about 68 and 73; pause points to about 74 and 77; reflection stays near baseline. The gaps are 2.3, 7.7, 5.6, 3.1, and 2.1 percentage points.
Figure 12. Share of covered problems on which the final answer followed the assistant, by condition and by whether the verdict was correct (filled) or wrong (open); 95% CIs over participants ( baseline , reliability cards , contrasting replies , pause points , reflection covered trials). Every condition followed correct advice more often than wrong advice by a similar margin. Dumbbell chart with five rows, one per condition, on a horizontal axis from 75 to 100 percent. Each row shows following of wrong advice as an open circle and of correct advice as a filled circle connected by a line. Baseline sits at about 92 and 96 percent, reliability cards at 85 and 91, contrasting replies at 95 and 97, pause points at 87 and 93, reflection at 93 and 96. The overall follow rate is printed at the right of each row.
Figure 13. Accuracy (left) and overconfidence, confidence minus accuracy (right), per item and condition, items ordered from easiest to hardest at baseline; error bars are 95% CIs, grey bars the assistant’s pre-study benchmark accuracy per item (card values). Accuracy approaches the chance rate for the hardest items; overconfidence grows with difficulty and is lowest under reliability cards and contrasting replies on the harder items. Two charts with twelve items on the horizontal axis ordered from easiest to hardest, one point series per condition and a grey horizontal bar per item for the assistant's own accuracy, which the participants' accuracy tracks and mostly stays below. Left: accuracy falls from about 80 percent on the easiest item to about 20 percent on the hardest for every condition, with pause points lower on the easy items and all conditions near the 25 percent guessing line on the hardest items. Right: confidence minus accuracy rises from about 0 on the easiest items to 50 to 60 percentage points on the hardest, with reliability cards and contrasting replies below baseline on the harder items and pause points in between.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Code | Name | Description | Experts |
|---|---|---|---|
| Intervention ideas | |||
| Interaction Design | |||
| ID-1 | Conversation visualization | The chat rendered as a navigable map or tree | E1 |
| ID-2 | Bidirectional text–vis sync | Editing text changes the visual and vice versa | E1 |
| ID-3 | Multiple options | Three candidate answers; the user must pick one and justify it | E1 |
| ID-4 | On-demand thinking mode | Reflection tools activate only on user request | E1 |
Table 13. Full coding template from the expert interviews: 49 codes in four groups, with the experts who raised each code.
More in this topic
Human-AI Interaction
Human-AI interaction studies how people and AI systems communicate, collaborate, make decisions, and shape one another over time.
See all 333 papers