cs.HCNov 17, 2026

How Web Research Agents Interpret User Priorities

Authors: Robin Sharma, Parisa Suchdev

Organizations: Independent Researcher Burlington, VT, USA · University of Vermont Burlington, VT, USA

Abstract

Web research agents gather information online and write cited reports, including financial analysis. Users can state a priority to guide this research, but a priority may not say how to judge the options, leaving the agent to choose the criteria. We examine how two such agents, built on OpenAI and Anthropic models, interpret a stated priority. In 36 runs, the agents compared Microsoft and Alphabet as businesses. Prompts asked for special attention to growth or resilience, or gave no priority. We analyzed reports, searches, returned sources and cited sources. Of the 24 reports given a priority, 21 focused on it, but their criteria varied. Under the growth priority, OpenAI reports that ranked the companies accounted for risk or predictability and favored Microsoft, while the Anthropic ones judged growth by its pace and favored Alphabet. Under the resilience priority, reports used partly different criteria, and most named Microsoft as the more resilient company. Of the search and source measures tested, only the Anthropic agent's search wording differed detectably by condition. These findings suggest evaluations should examine the criteria behind a report's judgements, not only its focus or accuracy. Interfaces could let users see and adjust those criteria.

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. The Default Trap: Rethinking Plan Evaluation in Tool-Using LLM Agents

    Sep 29, 2026Xueqi Li, Jingjie Ning, Yibo KongLLM Agent EvaluationTool-Using Agents

  2. Does Rank Still Matter? Position Bias When AI Agents Shop on Our Behalf

    Aug 24, 2026Davood Wadi, Yu MaAgentic SearchAgentic Commerce

  3. Source Preference in the Wild: How LLM Agents Favor Items by Source, and How to Reduce It

    Oct 2, 2026Jonghyun Song, Haewon Park, Jeonghoon Shim +2LLM AgentsLLM Decision-Making