cs.IROct 30, 2026

ProSearch: Benchmarking Multi-Constraint Protocol Retrieval in Experimental Science

Authors: Wenliang Liang, Zhiyuan Ning, Haolin Chen, Yuanchun Zhou, Wenjuan Cui, Yi Du

Organizations: Computer Network Information Center, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Westlake University · Institute of Zoology, Chinese Academy of Sciences

Abstract

In AI for Science scenarios, experimental design and method reproduction require retrieving protocols that satisfy multiple interdependent constraints, yet existing benchmarks lack systematic evaluation of this capability. To address this gap, we introduce ProSearch, a benchmark for multi-constraint protocol retrieval in experimental science. ProSearch has three key features: (1) realism, with queries grounded in real experimental records and expert-defined templates; (2) complexity, with 60 experimental fields and their dependencies; and (3) diagnosticity, with 3-5 explicit constraints per query, including numerical and negation constraints. We construct ProSearch through LLM-based extraction, data-driven constraint selection, controlled query generation, and evidence-based relevance validation. The resulting benchmark contains 1,515 queries across six experimental science domains. We conduct extensive experiments on ProSearch with 19 representative retrievers and complementary analyses. The best-performing retriever achieves an average nDCG@10 of only 0.290, which improves to 0.474 with reranking. Structured diagnostic analysis shows that explicit constraint matching provides a strong ranking signal, while manual error analysis identifies joint constraint binding as a key bottleneck. ProSearch provides a testbed for systematically evaluating scientific retrieval systems.

Figures & tables

Appendix figures & tables38 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. ScholarCatalyst: A Benchmark for Retrieving Papers That Inspire New Research

    Oct 1, 2026Sohyeon Kim, Yoonho Lee, Bo Liu +11AI Agents for Scientific DiscoveryAgentic Search

  2. InfiniteScienceGym: An Unbounded, Procedurally-Generated Benchmark for Scientific Analysis

    Apr 14, 2026Oliver Bentham, Vivek SrikumarLLM EvaluationScientific Agent Benchmarks

  3. SciExplore: Evaluating Autonomous Agents from Scientific Navigation to Information Integration

    Jul 23, 2026Yinhao Tang, Youqing Fang, Yanan Sun +6AI Agent BenchmarksInformation Retrieval