IR Evaluation

IR: Information Retrieval

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 36% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 122

All topics

Language Models for Page-Level Layout Decisions in E-commerce Search
Oct 7, 2026Varun Joshi, Eva C. Song, ChengXiang ZhaiE-Commerce Search IR Evaluation
Walmart Global Tech Hoboken, NJ, USA · University of Illinois at Urbana-Champaign Urbana, IL, USA
Towards Explaining Query Expansion Performance in Information Retrieval
Oct 7, 2026Sourav Saha, Aditya Dutta, Soumajit Pramanik +1Information Retrieval Query Expansion
Indian Statistical Institute, Kolkata, India · IIT Bombay, Bombay, India · IIT Bhilai, Chhattisgarh, India
A Systematic Study of Semantic ID Spaces for Generative Information Retrieval
Oct 6, 2026Alexia Allal, Hicham Randrianarivo, Sylvain LamprierResidual VQ Vector Quantization
Artefact Research Center Paris, France · LERIA, Angers University Angers, France
Leveraging a four-quadrant approach for evaluating Redpine Science
Oct 6, 2026Filip Dorm, Leonora VesterbackaScientific QA RAG Evaluation
Redpine, Sweden.
Beyond Semantic Similarity: Performance and Costs of Agentic Retrieval for Complex Tasks
Oct 5, 2026Reza Esfandiarpoor, Radek Osmulski, Yauhen Babakhin +8Agentic Retrieval Information Retrieval
NVIDIA · University of Edinburgh
Better Retrieval, Limited Clustering Gains: A Controlled Study of Multilingual Company Entity Resolution
Oct 4, 2026Yijiashun Qi, Yuxuan Li, Hanzhe GuoEntity Resolution Clustering
University of Michigan, Ann Arbor, USA · University of Pennsylvania, Philadelphia, PA, USA
ScholarCatalyst: A Benchmark for Retrieving Papers That Inspire New Research
Oct 1, 2026Sohyeon Kim, Yoonho Lee, Bo Liu +11AI Agents for Scientific Discovery Agentic Search
Stanford University · Seoul National University · University of Washington +3
What Should an Agent Remember? Disentangling Retention from Retrieval in Bounded-Memory Evaluation
Sep 30, 2026Juli HuangAI Agent Evaluation Memory Agent Benchmarks
Stanford University
Where Scientific Search Agents Fail: Decision-Checkpoint Auditing of Exposure and Inspection Attempts
Sep 29, 2026Hongmin Li, Wanli ZhaoAI Agent Auditing Scientific QA
School of Life Science and Technology, Institute of Science Tokyo Tokyo, Japan · Department of Computational Biology and Medical Sciences Graduate School of Frontier Sciences, The University of Tokyo Kashiwa, Japan · Department of Information Science Graduate School of Advanced Science and Engineering, Hiroshima University Higashihiroshima, Japan
Generated Query Expansion Still Helps Strong Sparse Retrieval: A Controlled Study with SPLADE-v3
Sep 29, 2026Ryan C. Barron, Cade W. Trotter, Maksim E. Eren +3Learned Sparse Retrieval Lexical Retrieval
Computational Intelligence & Modeling, Los Alamos National Laboratory, Los Alamos, New Mexico, USA. · Modeling and Observations of Earth Systems, Los Alamos National Laboratory, Los Alamos, New Mexico, USA. · Fluid Dynamics and Solid Mechanics, Los Alamos National Laboratory, Los Alamos, New Mexico, USA. +2
Beyond Readability: Evaluating Task Information Recoverability
Sep 29, 2026Yiwei LiuIR Evaluation Optical Character Recognition
School of Science and Engineering The Chinese University of Hong Kong, Shenzhen
Evaluating Name-Only Directory Routing for One-Shot Code Search
Sep 28, 2026Manoj BajajInformation Retrieval LLM Routing
READ-Bench: Benchmarking Historical Instance Retrieval for Time-Series Diagnosis
Sep 26, 2026Gerardo Pastrana, Haojun Li, Dhruv Mehta +4Temporal IR IR Evaluation
C3 AI · The Ohio State University
SEEK: Skill-Routed Evaluation with Evolvable Knowledge for Industrial Search
Sep 24, 2026Zhongxin Huang, Songyang Li, Renzhe Zhou +5LLM Evaluation Automated Evaluation
Peking University Beijing, China · Kuaishou Technology Beijing, China · Kuaishou Technology Hangzhou, China +2
FinFIRST: Benchmarking Search Agents for Financial Information Retrieval, Sourcing and Traceability
Sep 21, 2026Wenqing Wang, Haitao Xiang, Xinyi Zhao +8Financial QA AI Agent Benchmarks
Ling Team, Inclusion AI · Peking University · China International Capital Corporation Limited
Re:CAP - Auditing Retrieval Coverage in Production RAG Pipelines
Sep 21, 2026Aviral Joshi, Hanoz Bhathena, Max Nelson +1RAG Evaluation IR Evaluation
Machine Learning Center of Excellence, JPMorgan Chase & Co.
TRACE: Accountable Agentic Retrieval for Source Discovery in Digital Archives
Sep 17, 2026Donghan Bian, Marie Puren, Florian CafieroData Provenance Agentic Retrieval
ENC, LRE · École nationale des chartes – PSL Centre Jean-Mabillon Paris, France · EPITA EPITA Research Laboratory Le Kremlin-Bicêtre, France +2
Diagnosing the Fact-Grounding Gap in Multi-Hop Question Answering
Sep 15, 2026Kevin Mo, Nathan Mo, Richard ZhuMulti-Hop QA Multi-Hop Reasoning
Independent · Northwestern University
Q2D-Web: A Large-Scale Benchmark for Retrieval in Agentic RAG Systems
Sep 8, 2026Maximilian Schall, Sedigheh Eslami, Markus Krimmel +4Agentic Retrieval Multilingual IR
Perplexity AI
OntologyBench: Can Dense Retrieval Satisfy Structured Biomedical Constraints?
Sep 8, 2026Xiao Yu Cindy Zhang, Wyeth Wasserman, Jian ZhuInformation Retrieval Biomedical IR
Graduate Program in Bioinformatics, The University of British Columbia · Centre for Molecular Medicine and Therapeutics, BC Children’s Hospital Research Institute · Department of Medical Genetics, The University of British Columbia +1
Does the Selected Object Reach the Reader? Auditing Identity Handoffs in Grounded Language-Model Pipelines
Sep 4, 2026Siddharth Vohra, Runmin Jiang, Xiaomo Li +1Question Answering RAG Evaluation
Carnegie Mellon University, Pittsburgh, PA, USA · Amazon Web Services AI Native, Pittsburgh, PA, USA
Comparing Retrieval Methods for Academic Advisor Discovery: A Six-Method Study of 768 CS Faculty Profiles Across 9 US Universities
Sep 3, 2026Biraj SubediLearning to Rank Hybrid Retrieval
Independent Researcher
SHELF: A Synthetic Harness for Multi-Task Bibliographic Benchmarking
Sep 2, 2026Michael J. BommaritoLLM Evaluation Library and Information Science
Portions of this work were prepared with assistance from large language models. The author is solely responsible for all content, including any errors or omissions.
Incremental Pooled LLM Evaluation for Cost-Effective Retrieval Model Selection
Sep 2, 2026Max Nelson, Hanoz Bhathena, Aviral Joshi +1IR Evaluation LLM-as-a-Judge
JPMorganChase
Retrieved but not ranked: surface-form bias in structural retrieval, from mathematics to agent trajectories
Sep 1, 2026Nabira Rashid, Manolis KellisLLM Reranking Information Retrieval
Independent; work conducted while contributing to Mantis at MIT CSAIL Kellis Lab · Independent · MIT CSAIL
RATIO: A Benchmark for Retrieval Across Typed Ideation Operations in Scientific Literature
Aug 27, 2026Maayan Sharon, Tom HopeAI-Assisted Scientific Research IR Evaluation
The Hebrew University of Jerusalem · The Allen Institute for AI (AI2)
The "Curse of Knowledge" in LLM Query Simulation: Concept Provenance for Tracing Answer-Side Intrusion
Aug 26, 2026Chenglong Ma, Xinye Wanyan, Danula Hettiachchi +2LLM Auditing IR Evaluation
RMIT University Melbourne, Victoria, Australia
GreekBarRetrieval: A Benchmark for Greek Statutory Retrieval
Aug 19, 2026Ernest Beta, Odysseas S. Chlapanis, Dimitrios Galanis +1Legal IR Query Reformulation
Department of Informatics, Athens University of Economics and Business, Greece · Archimedes, Athena Research Center, Greece · Institute for Language and Speech Processing, Athena Research Center, Greece
Post-Calibration Reliability Reranking of Relevance Decisions via Label-wise Monotone Projection
Aug 11, 2026Inwoo Tae, Yongjae LeePost-Hoc Calibration IR Evaluation
UNIST Ulsan, Republic of Korea · LinqAlpha New York, NY, USA
Persona Conditioning as an Assessor-Sensitivity Probe for LLM-Based IR Evaluation
Aug 11, 2026Samaneh Mohtadi, Pietro Bernardelle, Joel Mackenzie +1LLM Evaluation LLM-as-a-Judge
The University of Queensland · Brisbane, Australia

Oct 7, 2026, cs.LGJ/K move · Enter open · S save

Language Models for Page-Level Layout Decisions in E-commerce Search

Varun Joshi, Eva C. Song, ChengXiang Zhai

Walmart Global Tech Hoboken, NJ, USA · University of Illinois at Urbana-Champaign Urbana, IL, USA

E-Commerce Search IR Evaluation LLM Evaluation

Open paper AskPDF arXiv

E-commerce search pages are critical touchpoints for millions of online shoppers. While traditional search engines return a ranked list of results, modern E-commerce search pages increasingly incorporate recommender system modules -- for example, secondary stacks that surface alternative product groupings at specific positions. When introduced appropriately, secondary stacks can improve user engagement; however, suboptimal placement may disrupt browsing flow and degrade the primary results. Unlike traditional search ranking, where evaluation techniques such as interleaving are well established, evaluating page-level layout changes e.g., when and where to insert a secondary stack remains challenging without costly online A/B testing. To address this, we study offline methods for evaluating whether a given layout decision -- specifically, the inclusion of a secondary stack at a particular position -- is beneficial to users. We investigate language models as scalable evaluators by comparing direct prompt-based, prompt-derived feature, and representation-based methods. Our results show that representation-based approaches consistently outperform prompt-based judging in predicting user engagement, suggesting they provide a reliable foundation for offline layout evaluation in E-commerce search.

Momentum

Related topics

Latest papers 122