cs.HCFeb 18, 2026
SaveOverseeing Agents Without Constant Oversight: Challenges and Opportunities
Organizations: University of Washington, United States · Microsoft Research, United States
Abstract
To enable human oversight, agentic AI systems often provide a trace of reasoning and action steps. Designing traces to have an informative, but not overwhelming, level of detail remains a critical challenge. In three user studies on a Computer User Agent, we investigate the utility of basic action traces for verification, explore three alternatives via design probes, and test a novel interface's impact on error finding in question-answering tasks. As expected, we find that current practices are cumbersome, limiting their efficacy. Conversely, our proposed design reduced the time participants spent finding errors. However, although participants reported higher levels of confidence in their decisions, their final accuracy was not meaningfully improved. To this end, our study surfaces challenges for human verification of agentic systems, including managing built-in assumptions, users' subjective and changing correctness criteria, and the shortcomings, yet importance, of communicating the agent's process.
Figures & tables
Figure 1. Through three user studies, we investigate how the presentation of an agentic workflow’s actions affects human verification. We first find that participants miss small but impactful errors with the current verbose display listing agent steps with interleaved summaries. With three design probes, we next find that focusing on the steps provides useful context but can justify overreliance. Finally, with our novel interface that explicitly defines the task’s requirements and the agent’s assumptions, participants found errors faster but also reported increased confidence with no meaningful effect on accuracy. There are two rows of items. One is titled "Human oversight aims to improve accuracy and reduce cost". Under that heading, there are two boxes, one of which says
(HIL) Agent Workflow) that has a Task'' input and an Output'' output. An arrow connects this box to another titled Review Mechanism''. This arrow says actions reported to user''. There is an arrow going from the Review Mechanism'' box to the (HIL) Agent Workflow'' box that says user takeaways inform''. The second row of items is titled We inspect to improve the review mechanism.'' This section is split into three columns, each of which represent a user study and have an n=12 indication. First is titled Step summaries'' and has the takeaway Existing methods are verbose and hide errors.'' It shows a diagram of small action steps that are split into groups of actions that are summarized in text. The second column says Design probes'' with the takeaway A reasonable process justifies overreliance.'' It has three items along an arrow going from Process-Oriented'' to Outcome'' with three methods Flowchart'', Citation'' and Spec'' along that arrow. The third column is titled Control'' with the takeaway Find errors faster but more false confidence.'' It has two parts, the Baseline: Step summaries approach with a final summary'' and Treatment: Spec design with improved linking and detail acess''.Figure 2. Magentic-UI interface. Our formative study illuminates verification practices with the HIL CUA Magentic-UI ( Mozannar et al., 2025 ) . Users first input a task [A], then use various features to co-plan, co-task, and adapt to errors as the system runs. To verify, they can first review the plan summary that decomposes the task into proposed plan steps [B]. Each plan step is then executed, and each action is described in the trace for transparency [C]. Collapsing these plan steps [D] shows summaries of each group of actions. While the task runs, users may watch the live view as the agent interacts with the browser [E]. This visual trace is stored in a series of screenshots as the agent progresses [F]. Clicking the icon next to a particular action [C] links to the associated screenshot [F] to improve navigation. Users can open tabs between different tasks to multitask [G] and interact with chat [H]. A “Final Answer” summary appears upon completion (not visible). An interface display of Magentic UI. The top left contains a small icon that, when clicked, opens different tabs next to the Magentic-UI logo. The left half of the screen mostly contains first the task, then a given plan of steps in distinct boxes, then a series of bullet points that show certain actions, such as
I typed <url> into the browser address bar''. Below is a chat input box. On the right side, there are two tabs: Screenshots and Live View. The screenshots tab is selected and shows one screenshot. There is an indicator that it is showing 3/14 screenshots, with arrow buttons to the left and right. The bottom of the interface stats Magentic-UI can make mistakes. Please monitor its work and intervene if necessary.''Figure 3. Three design probes. Our design probes focus on the process (Flowchart), outcome (Specification), or both (Citation). This figure illustrates each design for the task “How much would I save by getting annual passes for my family (2 adults, 1 kid age 5, 1 kid age 2) for the Seattle Aquarium, compared to buying daily tickets, if we visit 4 times in a year?” Additional examples are in Supplementary Materials. Three columns, each of which has a method. The Flowchart column has three rows of nodes. The first node says
Go to Seattle Aquarium site'' which connects to two nodes in the same row Find daily prices'' and Find membership prices''. Each of those connects to a node in the final row of Calculate the difference''. The second Citation column says the following, with [brackets] indicating underlined portions Based on the official Seattle Aquarium ticket page (https://www.seattleaquarium.org/), the family plan membership costs [\$249] total. The daily ticket price is [\$47.95 per adult] and estimated [\$32 for children 4-12 and free for children under 4], for [\$127.90] total per visit. If visiting 4 times in a year, the membership would save you [\$262.60].'' The final column is the specification, which includes the requirements 1) Daily prices for two adults, one 5yo and one 2yo found'', 2) Membership plan for two adults, one 5yo and one 2yo found'', and 3) Difference between membership and daily prices calculated'' and the assumptions Estimated youth price of \$32 is a typical rate (not shown on site)'', Your family is not WA residents'', Peak price used (changes daily)'', No guests needed on membership pass'', and ``Aquarium website is a reliable source''.Figure 4. Design probe study interface. The interface displays the given task [A], task output [B], and one of the three design probes [C]. Clicking on the design probe surfaces the relevant screenshot(s) [D] that have highlighted annotations of the relevant information [E]. The original Magentic-UI trace is available for further review [F]. Participants then answer the questions “Is this result correct?” and “Why or why not?” below [G]. This figure shows an interface display. The top row says
Flowchart Page - aquarium'' with the task How much will I save by getting annual passes for my family (2 adults, 1 kid age 5, 1 kid age 2) for the Seattle Aquarium, compared to buying daily tickets, if we visit 4 times in a year?'' The left side shows the outcome By purchasing the annual pass instead of daily tickets for 4 visits, your family would save \$511.60 − \$249 = \$262.60 annually'' as well as the flowchart from the design probes figure. The right side shows a screenshot with a yellow highlighted annotation over the membership price used. There is an indication that this is Image 1 of 2'' as well as text saying ``Session: 70 Image: 11''| GT Ans. | Method | Accuracy (%) | Duration (s) |
|---|---|---|---|
| No | Flowchart | 39.13 | 427.38 |
| Citation | 52.17 | 293.97 | |
| Spec | 65.22 | 542.15 | |
| Yes | Flowchart | 91.67 | 319.85 |
| Citation | 83.33 | 573.87 | |
| Spec | 100.00 | 491.65 |
Table 1. Design probe accuracy and duration performance. The Spec method shows the highest accuracy but also the highest duration. GT Ans. indicates if the output is correct, with “All” including all trials. This exploratory investigation had two incorrect questions and one correct question per method.
| Method | Preference | Validation |
|---|---|---|
| Flowchart | 2.75 | 3.25 |
| Citation | 2.17 | 2.50 |
| Spec | 2.08 | 2.25 |
| Magentic-UI | 3.00 | 2.00 |
Table 2. Mean rank preference for design probes and Magentic-UI. Participants ranked methods by Preference (P1-P12) and by their perception of which best supported Validation (P5-P12). For both conditions, Spec was the best and Flowchart the worst ranked, although there was much variation between participants (lower is better). The full Magentic-UI interface was less preferred but considered more likely to support validation.
Figure 5. Controlled study interface. The interface provides the task [A] and the output to verify [B]. It integrates the Magentic-UI trace directly into the interface, adding highlights to both the text [C] and screenshots [D]. The screenshots can be paged through, with the ability to magnify for detail and to open a new tab with the associated website for further review [E]. For additional path context, there is a brief summary [F] provided in the Citation format. For guiding review, there are requirements and assumptions [G], with additional explanatory context and icons indicating if requirements are completed, unknown, or contradicted. Each requirement and assumption refers to annotation(s) in the text and image trace with underlying icons [H] and tooltips for easier scanning [I]. For the study, there is a 5-minute timer [J] and a series of questions [K]. This figure shows an interface display. The top bar has a timer countdown and a question
Is this result correct'' with a dropdown selection and a submit button. There is a pink box that contains the task text at the top left, then a white box inset that contains the trace of actions, then a final answer, then a small paragraph stating I first navigated to the [Seattle Aquarium website]. I then checked for [annual membership rates] and [daily rates]. Finally, I [calculated the difference]. Below this pink box, the interface is split into two columns titled Requirements'' and Assumptions. Requirements has two icons at the top, one is green with a checkmark, and the number 2, and the other is yellow with a question mark and the number 1. Next to assumptions is an orange number 5. Below each are boxes containing a requirement or assumption. Each box has the text (e.g., Estimated youth price of \$32 is a typical rate (not shown on site)'') and several icons below, formatted either as an image icon or a document icon. Hovered icons show a black tooltip, in this case 2yo''. Selected icons turn a magenta color. Below the icons is a brief explanatory description (e.g., Youth ages 4-12 receive reduced pricing, but the exact numbers are not available. \$32 is an estimate based on similar museums.'' The right side has a header titled Screenshot Review'' indicating image 14 of 14. This header has a magnification icon, a browser icon, and arrow keys. The image itself has yellow highlighting. The text in the trace also has yellow highlighting.Figure 6. Control study results. The chart shows the Hedges’ effect size and the 95% confidence interval. The Duration and Confidence charts show subsets such that “No-Yes” displays when the ground truth answer is “No” and the user answers “Yes”. Overall, we find no effect on accuracy (0.18), a small effect decreasing duration (-0.29), and a medium effect increasing confidence (0.56). The effect on duration is medium-sized when the user correctly finds an error (“No-No”: -0.65). The largest confidence effect occurs when the answer is incorrect, but the user thinks it is correct (“No-Yes”: 0.85). Exact values are in Table 3 . Three dotted vertical lines show the cutoffs for small, medium, and large effect sizes. Three graphs. Values are from the main results table. Each has Hedges' g on the x-axis. First graph titled
All'' has y-axis values Accuracy'' Duration'' and Confidence''. Second graph titled Duration'' and third graph titled Confidence'' have y axis values Yes-Yes'', No-No'', and ``No-Yes''. Each graph has three points indicating the effect size with horizontal bars spanning the 95% confidence interval. There are light grey bars for small, medium, and large effect sizes at 0.2, 0.5, and 0.8.| Accuracy (%) | Duration (s) | Confidence (1-7) | ||||||||||||||||
| GT | User | Cohen’s | Cond. | Mean | STD | Eff. | CI L | CI H | Mean | STD | Eff. | CI L | CI H | Mean | STD | Eff. | CI L | CI H |
| No | No | Base. | 100.00 | 0.00 | – | – | – | 270.79 | 62.93 | -0.65 | -1.54 | 0.12 | 3.54 | 1.85 | 0.37 | -0.41 | 1.23 | |
| Treat. | 100.00 | 0.00 | – | – | – | 229.63 | 60.49 | 4.23 | 1.74 | |||||||||
| No | Yes | Base. | 0.00 | 0.00 | – | – | – | 251.85 | 93.01 | -0.20 | -1.12 | 0.67 | 3.64 | 1.63 | 0.85 | 0.01 | 1.88 | |
| Treat. | 0.00 | 0.00 | – | – | – | 234.48 | 69.99 | 4.82 | 0.98 | |||||||||
| Yes | No | Base. | 0.00 | 0.00 | – | – | – | 151.31 | 98.17 | – | – | – | 1.50 | 0.71 | – | – | – | |
Table 3. Quantitative results between the Baseline and Treatment conditions. GT and User refer to the ground truth and user answer to the question “Is this output correct?” respectively. “All” includes both “Yes” and “No” responses. Effects compare the Baseline (Base.) and Treatment (Treat.) conditions. We report the type of Cohen’s calculated (Cohen’s) and all effect sizes (Eff.) have an unbiasing correction applied to become Hedges’ . We include the lower (CI L) and upper (CI U) confidence intervals. Dashes occur where the effect cannot be calculated. Confidence is measured on a 7-point Likert scale.
| Task 1 | Task 2 | Task 3 | |
|---|---|---|---|
| Correct | 8 | 1 | 2 |
| Acceptable | N/A | 4 | N/A |
| Incorrect | 1 | 5 | 10 |
| Incompete – timing | 2 | 4 | 0 |
| Incomplete – technical | 3 | 1 | 1 |
Table 4. Outcome instances from the formative study.
| Key | Task | Gound Truth Ans. | Presented Ans. | Insuff. |
| FLOWCHART | ||||
| yellowstone | What are hikes in Yellowstone that have been recommended by at least three different people with kids and are highly rated on TripAdvisor (an average from 4.5/5 from at least 50 reviews)? | Mystic Falls Trail Fountain Paint Pot Trout Lake Trail Lone Star Geyser | 1. Fairy Falls Trail 2. Trout Lake Trail | Yes |
| dog | The dog genome was first mapped in 2004 and has been updated Several times since. What is the link to the files that were most recent in May 2020? | https://ftp.ensembl.org/pub/ release-100/fasta/ canis_lupus_familiaris/dna/ | https://www.ncbi.nlm.nih.gov /datasets/genome/ GCF_000002285.5/ | No |
| andew | On which social media platform does Andrew Ng have the most followers? | Linked-In | Linked-In | Yes |
| CITATION | ||||
| karting | Which paintball places in Cologne, Germany are within a 10 minute walk from a karting track? | Adrenalinpark Köln | None available | Yes |
Table 5. Tasks from the design probe study. The Insuff. column delineates if the answer “Insufficient” counts as correct.
Figure 7. Flowchart tasks. Keywords relate to Table 5 . Three columns for three flowchart tasks. Each has nodes with action items, some of which are highlighted in blue. Outcome nodes have a light blue background.
Figure 8. Citation tasks. Keywords relate to Table 5 . Three columns, each of which shows the citation description for the three citation tasks.
Figure 9. Specification tasks. Keywords relate to Table 5 . Three columns, each of which shows the specification for each of the specification tasks.
| Participant | Rank 1 (high) | Rank 2 | Rank 3 | Rank 4 |
|---|---|---|---|---|
| P6 | Flowchart | Spec | Citation | Magentic-UI |
| P3 | Flowchart | Spec | Magentic-UI | Citation |
| P1 | Spec | Flowchart | Citation | Magentic-UI |
| P5 | Spec | Flowchart | Citation | Magentic-UI |
| P4 | Spec | Citation | Flowchart | Magentic-UI |
| P8 | Spec | Citation | Flowchart | Magentic-UI |
Table 6. Overall preference rankings from the design probes study. Manually ordered by similarity.
| Participant | Rank 1 (high) | Rank 2 | Rank 3 | Rank 4 |
|---|---|---|---|---|
| P7 | Citation | Magentic-UI | Spec | Flowchart |
| P10 | Citation | Magentic-UI | Flowchart | Spec |
| P12 | Magentic-UI | Citation | Flowchart | Spec |
| P5 | Magentic-UI | Spec | Flowchart | Citation |
| P6 | Magentic-UI | Spec | Flowchart | Citation |
| P8 | Spec | Flowchart | Citation | Magentic-UI |
Table 7. Expected validation support rankings from the design probes study. Manually ordered by similarity. Only asked to P5-P12
| Task | Ground Truth Ans. | Presented Ans. |
| SET 1 | ||
| Which Fidelity international emerging markets equity mutual fund with $0 transaction fees had the lowest percentage increase between May 2019 to May 2024? | FPADX | FEDDX |
| What is the highest rated (according to IMDB) Isabelle Adjani feature film that is less than 2 hours and is available on Vudu (now called Fandango at Home) to buy or rent? | Nosferatu the Vampyre | No film fit the criteria. |
| How much will I save by getting annual passes for my family (2 adults, 1 kid age 5, 1 kid age 2) for the Seattle Children Museum, compared to buying daily tickets, if we visit 4 times in a year? | $61 | $61 |
| Which members of Fubo’s Management Team joined the company during the same year Fubo’s IPO happened? | Gina DiGioa | Gina DiGoia |
| SET 2 |
Table 8. Tasks from the control study.
| Value | Descriptor |
|---|---|
| 1 | Not at all confident |
| 2 | Slightly confident |
| 3 | Somewhat confident |
| 4 | Moderately confident |
| 5 | Quite confident |
| 6 | Very confident |
Table 9. Confidence scale. The scale values participants selected from for the question “How confident are you in your previous answer?”
Figure 10. Control study baseline interface. The content is near-identical Magentic-UI’s, but more visibly similar to the treatment condition. The display shows the task [A] and final answer [B], with a text description of actions [C] and a summarized final answer [D]. The screenshots are available [E], with one addition of a magnification feature [F]. Participants have five minutes [G] to determine if the output is correct [H]. This figure shows an interface display. The top bar has a timer countdown and a question
Is this result correct'' with a dropdown selection and a submit button. There is a pink box that contains the task text at the top left, then a white box inset that contains the trace of actions, and then a final answer is briefly stated. Below this pink box is a white box containing a Final Answer'' summary with more detail. The right side has a Screenshots'' header indicating we are on screenshot 14 of 14''. There is an icon of a magnifying glass and arrow keys. Below this header is the screenshot itself.More in this topic
Human Oversight of AI
Human oversight of AI studies how people supervise, verify, and control AI decisions, especially as systems become more autonomous and consequential.
See all 184 papers