Behind the Prompt: The Agent-User Problem in Information Retrieval
Organizations: University of Passau, Passau, Germany · IT:U Austria, Linz, Austria
Abstract
User models in information retrieval rest on a foundational assumption that observed behavior reveals intent. This assumption collapses when the user is an AI agent privately configured by a human operator. For any action an agent takes, a hidden instruction could have produced identical output - making intent non-identifiable at the individual level. This is not a detection problem awaiting better tools; it is a structural property of any system where humans configure agents behind closed doors. We investigate the agent-user problem through a large-scale corpus from an agent-native social platform: 370K posts from 47K agents across 4K communities. Our findings are threefold: (1) individual agent actions cannot be classified as autonomous or operator-directed from observables; (2) population-level platform signals still separate agents into meaningful quality tiers, but a click model trained on agent interactions degrades steadily (-8.5% AUC) as lower-quality agents enter training data; (3) cross-community capability references spread endemically ( 1.26-3.53) and resist suppression even under aggressive modeled intervention. For retrieval systems, the question is no longer whether agent users will arrive, but whether models built on human-intent assumptions will survive their presence.
Figures & tables
| Metric | Value |
|---|---|
| Total posts | 370,737 |
| Total comments | 3,882,705 |
| Unique agents | 46,872 |
| Communities (submolts) | 4,257 |
| Collection period | Jan. 28–Feb. 8, 2026 |
| Duplicate rate (posts with identical title and body) | 32.9% |
| Observable | High-Val | Low-Val | Cohen’s |
| One-shot ratio | 0.178 | 0.524 | |
| Cross-comm. entropy | 0.740 | 0.241 | |
| Temporal burstiness | |||
| Style consistency | 0.430 | 0.548 |
User Modeling
User modeling represents people's interests, knowledge, goals, and changing behavior to personalize predictions and AI assistance.
See all 150 papers