cs.IRNov 3, 2026

ττ-Rec-Books: A Disclosure-Controlled Benchmark for Requirement Elicitation in Agentic Recommender Systems

Authors: Bharath Sivaram Narasimhan, Karthik R. Narasimhan

Organizations: Independent Researcher, Mountain View, CA, USA · Princeton University, Princeton, NJ, USA

Abstract

While conversational recommendation systems provide a more natural user interface, they face the challenge of progressively discovering a user's needs before they can recommend an item that satisfies them. To evaluate the ability of AI agents at this elicitation process, we introduce ττ-Rec-Books, a benchmark that holds each user request fixed while varying only when the user reveals their requirements: all at the start (volunteer), one at the start and the rest when asked (mixed), or some never (hidden). A hidden requirement can be found only by proposing books and inferring it from the user's accept or reject verdicts. We generate 450 tasks over a catalog of 20,000 books from Wikidata, develop a rule-based simulated user to answer the agent, and score recommendations against catalog metadata without an LLM judge. Across four agents and two search interfaces, success rates fall for every agent and interface, on average from 0.975 (volunteer) to 0.756 (mixed) and 0.283 (hidden). Mixed tasks fail mainly because agents do not ask, and when told to ask, every agent reaches at least 0.931. On hidden tasks, most failures come from agents recommending a book they never proposed to the user. When told to propose first, agents only match a simple baseline that proposes popular books, and stay well below one that uses rejection to rule out possible requirements. We release the catalog pipeline, task generator, simulated user, validators, reference agents, and all agent traces at https://github.com/nbharaths/tau-rec/tree/main/data/tracks/books-wikidata-v2.

Figures & tables

Explore similar work

CardsList
  1. ττ-Rec: A Verifiable Benchmark for Agentic Recommender Systems

    Jun 8, 2026Bharath Sivaram Narasimhan, Karthik R NarasimhanAI Agent ReliabilityAI Agent Benchmarks

  2. SimRec: Learning to Search, Ask, and Recommend from Simulated Interaction

    Nov 11, 2026Yaxin Zhang, Xingyu Wu, Yuchen Yan +6Recommender SystemsLarge Language Model-Based Recommendation