ProSearch: Benchmarking Multi-Constraint Protocol Retrieval in Experimental Science
Organizations: Computer Network Information Center, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Westlake University · Institute of Zoology, Chinese Academy of Sciences
Abstract
In AI for Science scenarios, experimental design and method reproduction require retrieving protocols that satisfy multiple interdependent constraints, yet existing benchmarks lack systematic evaluation of this capability. To address this gap, we introduce ProSearch, a benchmark for multi-constraint protocol retrieval in experimental science. ProSearch has three key features: (1) realism, with queries grounded in real experimental records and expert-defined templates; (2) complexity, with 60 experimental fields and their dependencies; and (3) diagnosticity, with 3-5 explicit constraints per query, including numerical and negation constraints. We construct ProSearch through LLM-based extraction, data-driven constraint selection, controlled query generation, and evidence-based relevance validation. The resulting benchmark contains 1,515 queries across six experimental science domains. We conduct extensive experiments on ProSearch with 19 representative retrievers and complementary analyses. The best-performing retriever achieves an average nDCG@10 of only 0.290, which improves to 0.474 with reranking. Structured diagnostic analysis shows that explicit constraint matching provides a strong ranking signal, while manual error analysis identifies joint constraint binding as a key bottleneck. ProSearch provides a testbed for systematically evaluating scientific retrieval systems.
Figures & tables
| Benchmark | # Queries | Sci. | Evidence Unit | Query Construction | Relevance Requirement | ||||
| Hum. | LLM | Constr. | Hard | Bind. | Proc. | ||||
| MultiConIR ( Lu et al., 2025 ) | 23,070 | ✗ | Structured Item | ✗ | ✓ | ✓ | ✓ | ✓ | ✗ |
| BRIGHT ( Su et al., 2025 ) | 1,384 | ✗ | Document | ✓ | ✗ | ✗ | ✗ | ✗ | ✓ |
| FollowIR ( Weller et al., 2025a ) | 102 | ✗ | Document | ✓ | ✗ | ✗ | ✓ | ✗ | ✗ |
| TREC-COVID ( Roberts et al., 2020 ) | 50 | ✓ | Document | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ |
| SciFact ( Wadden et al., 2020 ) | 1,409 | ✓ | Abstract | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ |
| Total Number | Avg. Length | Query Diversity | Examples | |||||
| Domain | #Field Templates | Top-10 Coverage | ||||||
| Cell Culture | 284 | 46,004 | 7.1 | 22.0 | 218.7 | 14 | 97.2% | Tab. 41 |
| Immunotechnology | 264 | 93,315 | 9.0 | 26.9 | 228.1 | 26 | 85.6% | Tab. 42 |
| Electrochemistry | 201 | 91,375 | 8.7 | 28.0 | 231.4 | 44 | 58.7% | Tab. 43 |
| Organic Synthesis | 252 | 30,371 | 9.8 | 26.1 | 262.2 | 57 | 49.2% | Tab. 44 |
| Inorganic Materials | 247 | 46,136 | 4.0 | 27.2 | 217.7 | 104 | 34.8% | Tab. 45 |
| Criterion | Pass Rate |
| Faithfulness | 100% |
| Clarity | 100% |
| Retrieval Usefulness | 91.7% |
| Retriever | Cell. | Immuno. | Electro. | Organic | Inorg. | Polymer | Avg. |
| Sparse lexical retrieval | |||||||
| BM25 | 0.136 | 0.011 | 0.079 | 0.132 | 0.217 | 0.297 | 0.145 |
| General-purpose dense retrieval | |||||||
| Sentence-BERT | 0.082 | 0.012 | 0.064 | 0.122 | 0.263 | 0.065 | 0.101 |
| GTE-large | 0.222 | 0.041 | 0.127 | 0.184 | 0.441 | 0.182 | 0.199 |
| BGE-M3 | 0.254 | 0.025 | 0.106 | 0.216 | 0.369 | 0.326 | 0.216 |
| Query Formulation | Cell. | Immuno. | Electro. | Organic | Inorg. | Polymer | Avg. | Avg. |
| Original | 0.386 | 0.092 | 0.175 | 0.282 | 0.495 | 0.309 | 0.290 | 0.000 |
| Query Decomposition | 0.317 | 0.068 | 0.091 | 0.205 | 0.454 | 0.254 | 0.232 | |
| Structured Query | 0.324 | 0.080 | 0.138 | 0.250 | 0.494 | 0.189 | 0.246 | |
| INF-Query-Aligner 5 5 footnotemark: 5 | 0.382 | 0.075 | 0.141 | 0.233 | 0.487 | 0.196 | 0.252 | |
| HyDE ( Gao et al., 2023 ) | 0.359 | 0.098 | 0.154 | 0.262 | 0.501 | 0.255 | 0.272 | |
| TongSearch-QR ( Qin et al., 2025 ) | 0.385 | 0.092 | 0.172 | 0.265 | 0.505 | 0.288 | 0.285 |
| Model | Cell. | Immuno. | Electro. | Organic | Inorg. | Polymer | Avg. |
| First-stage retrieval | |||||||
| Qwen3-Embedding-8B | 0.386 | 0.092 | 0.175 | 0.282 | 0.495 | 0.309 | 0.290 |
| Reranking on top-100 candidates | |||||||
| BGE-Reranker-v2-m3 ( Chen et al., 2024 ) | 0.481 | 0.192 | 0.247 | 0.336 | 0.705 | 0.416 | 0.396 |
| Rank-R1 ( Zhuang et al., 2025 ) | 0.521 | 0.227 | 0.269 | 0.408 | 0.622 | 0.478 | 0.421 |
| Rank1 ( Weller et al., 2025b ) | 0.592 | 0.266 | 0.241 | 0.416 | 0.598 | 0.457 | 0.428 |
| Method | Cell | Electro | Immuno | Inorg | Organic | Polymer | Avg. |
| Qwen3-Embedding-8B | 0.386 | 0.175 | 0.092 | 0.495 | 0.282 | 0.309 | 0.290 |
| Structured Diagnostic | 0.789 | 0.698 | 0.682 | 0.875 | 0.773 | 0.829 | 0.774 |
| Hybrid Diagnostic | 0.858 | 0.741 | 0.723 | 0.910 | 0.795 | 0.859 | 0.814 |
Appendix figures & tables38 assets
Supplementary material from the paper’s appendix.
Appendix
| Field | Role | Value Type | Neg. | Exp. | Description |
| cultured_cells. cell_type | Goal | Categorical | F | F | Cell line or cell type, such as A549, HeLa, or primary fibroblasts. |
| culture_medium. basal_medium | Input | Categorical | T | T | Basal culture medium, such as DMEM, RPMI-1640, or IMDM. |
| culture_medium.serum. identity | Input | Categorical | T | T | Serum identity or serum-free condition, such as fetal bovine serum, horse serum, or serum-free medium. |
| culture_substrate_or_coating | Input | Categorical | T | T | Culture substrate or coating, such as collagen-coated plates, Matrigel, or poly-D-lysine. |
| culture_medium.serum. concentration | Condition | Numeric / Range | F | T | Serum concentration, usually expressed as a percentage. |
| seeding_density | Condition | Numeric / Range | F | T | Cell seeding density, usually expressed as cells per well, cells per dish, or cells per area. |
| Field | Role | Value Type | Neg. | Exp. | Description |
| protein_targets. target | Goal | Categorical | F | F | Protein target measured in the assay, such as AKT, ERK, or GAPDH. |
| normalization_control | Input | Categorical | T | T | Normalization or loading control used for comparison, such as -actin, GAPDH, or tubulin. |
| sample_type | Input | Categorical | F | F | Type of biological sample, such as cell lysate, tissue lysate, serum, or purified protein. |
| blocking_solution. identity | Input | Categorical | T | T | Identity of the blocking solution, such as milk, BSA, or skim milk. |
| protein_targets. target_form | Condition | Categorical | F | F | Target form or modification state, such as phosphorylated or total protein; retained in extraction but not used for query construction. |
| blocking_solution. concentration | Condition | Numeric / Range | F | T | Concentration of the blocking solution, typically expressed as a percentage. |
| Field | Role | Value Type | Neg. | Exp. | Description |
| working_electrode | Method | Categorical | F | F | Working electrode material or type, such as glassy carbon, platinum, or nickel foam. |
| electrolytes. chemical | Input | Categorical | T | T | Electrolyte chemical species, such as KOH, H 2 SO 4 , Na 2 SO 4 , or KCl. |
| reference_electrode | Method | Categorical | F | T | Reference electrode used in the electrochemical setup, such as Ag/AgCl, SCE, or RHE. |
| target_species | Goal | Categorical | F | F | Electrochemical target species or reaction subject, such as oxygen, hydrogen, nitrate, or glucose. |
| electrolytes. concentration | Condition | Numeric / Range | F | T | Electrolyte concentration, typically expressed in units such as M, mM, or wt%. |
| temperature | Condition | Numeric / Range | F | T | Experimental temperature, usually expressed in ∘ C. |
| Field | Role | Value Type | Neg. | Exp. | Description |
| transformation_type | Goal | Categorical | F | F | Type of chemical transformation, such as cross-coupling, arylation, amination, borylation, or C–H activation. |
| substrates | Input | Categorical | F | T | Reactant or substrate class used in the reaction, such as aryl halides, boronic acids, amines, or alkynes. |
| catalysts. name | Input | Categorical | F | T | Catalyst identity or catalytic species, such as Pd, Ni, Cu catalysts, or named catalyst complexes. |
| ligands | Input | Categorical | F | T | Ligand used with the catalyst system, such as phosphine, bipyridine, or N-heterocyclic carbene ligands. |
| catalysts. loading | Condition | Numeric | F | T | Catalyst loading, typically expressed as mol%; retained as an exploration field but not used as a fixed query constraint. |
| bases | Condition | Categorical | T | T | Base used in the reaction, such as K 2 CO 3 , Cs 2 CO 3 , NaOtBu, or triethylamine. |
| Field | Role | Value Type | Neg. | Exp. | Description |
| target_material | Goal | Categorical | F | F | Target inorganic material or composition to be synthesized, such as an oxide, perovskite, spinel, or ceramic material. |
| precursors | Input | Categorical | F | T | Starting precursor materials used for synthesis, such as metal oxides, carbonates, nitrates, or salts. |
| dopant_or_additive. identity | Input | Categorical | T | T | Identity of the dopant or additive introduced into the material system. |
| dopant_or_additive. amount | Condition | Numeric / Range | F | T | Amount or concentration of the dopant or additive, such as molar ratio, atomic percentage, or weight percentage. |
| atmosphere | Condition | Categorical | T | T | Processing atmosphere, such as air, oxygen, nitrogen, argon, vacuum, or reducing atmosphere. |
| heating_rate | Condition | Numeric / Range | F | T | Heating rate used during thermal processing, usually expressed in ∘ C/min. |
| Field | Role | Value Type | Neg. | Exp. | Description |
| polymer_system | Goal | Categorical | F | F | Polymer or resin system under study, such as PEGDA, GelMA, polyurethane acrylate, or epoxy acrylate. |
| polymer_concentration | Condition | Numeric | F | T | Polymer concentration in the formulation, typically expressed as wt%, vol%, or concentration per unit volume. |
| photoinitiators. name | Input | Categorical | T | T | Identity of the photoinitiator, such as Irgacure 2959, LAP, or TPO. |
| crosslinking_agent. name | Input | Categorical | T | T | Identity of the crosslinking agent, such as MBAA, PEGDA, or other crosslinking reagents. |
| photoinitiators. loading | Condition | Numeric | F | T | Amount or loading of the photoinitiator, typically expressed as wt%, mol%, or phr. |
| crosslinking_agent. loading | Condition | Numeric | F | T | Amount or loading of the crosslinking agent, typically expressed as wt%, mol%, or phr. |
| Domain | Explore Role | # Constraints | Template Properties | Total | |||||||
| No Exp. | Input | Cond. | Method | 3 | 4 | 5 | Numeric | Neg. | Neither | ||
| Cell Culture | 9 | 13 | 10 | 5 | 9 | 15 | 13 | 28 | 31 | 0 | 37 |
| Electrochemistry | 17 | 5 | 25 | 8 | 16 | 24 | 15 | 50 | 44 | 0 | 55 |
| Immunotechnology | 7 | 6 | 9 | 7 | 10 | 13 | 6 | 23 | 12 | 4 | 29 |
| Inorganic Materials | 103 | 65 | 198 | 42 | 55 | 136 | 217 | 361 | 251 | 10 | 408 |
| Organic Synthesis | 23 | 26 | 47 | 15 | 31 | 50 | 30 | 8 | 60 | 47 | 111 |
| Trigger Field | Required Fields |
| Cell Culture | |
| culture_medium.serum.concentration | culture_medium.serum.identity , culture_medium.basal_medium |
| culture_medium.serum.identity | culture_medium.basal_medium |
| seeding_density | cultured_cells.cell_type , culture_mode |
| differentiation_state | cultured_cells.cell_type |
| confluence | cultured_cells.cell_type , culture_mode |
| Role | Field | Operator | Value | Explanation | Dependent Fields |
| Goal | Cell type | HUVEC | Uses HUVEC cells | – | |
| Input | Basal medium | DMEM | Does not use DMEM | – | |
| Condition | Seeding density | cells/well | Seeded at cells/well | Cell type | |
| Condition | Seeding density | between | – cells/well | Seeded within this density range | Cell type |
| Method | Culture mode | 3D | Uses 3D culture | – |
| Template ID | Fixed Role Template | Explore Role | # Constraints |
| T1 | Goal + Input + Condition | – | 3 |
| T2 | Goal + Input + Method | – | 3 |
| T3 | Goal + Input + Input + Condition | – | 4 |
| T4 | Goal + Input + Condition + Method | – | 4 |
| T5 | Goal + Input + Condition + Condition + Method | – | 5 |
| T6 | Goal + Input | Input | 3 |
| Template ID | Level | Fixed Template | Explore Role/Field |
| T1 | Role-level | Goal + Input + Condition | – |
| Field-level | target_material + dopant_or_additive.identity + atmosphere | – | |
| T5 | Role-level | Goal + Input + Condition + Condition + Method | – |
| Field-level | target_material + precursors + sintering.temperature + sintering.time + synthesis_route | – | |
| T6 | Role-level | Goal + Input | Input |
| Field-level | target_material + dopant_or_additive.identity | precursors |
| Domain | Search Query |
| Cell Culture | (cell culture OR cell line OR in vitro) AND (DMEM OR RPMI OR IMDM OR DMEM/F12 OR DMEM-F12 OR alpha-MEM OR McCoy’s 5A OR FBS OR fetal bovine serum OR serum-free OR incubat* OR seeded OR seeding density OR confluence OR culture medium) NOT (review OR meta-analysis OR survey OR perspective) |
| Immunotechnology | (western blot OR immunoblot OR immunoblotting OR western blotting) AND (primary antibody OR secondary antibody OR membrane OR SDS-PAGE OR 1: OR ECL OR TBST) NOT (review OR meta-analysis OR survey OR perspective) |
| Electrochemistry | (electrochem* OR electrochemical OR electrocatal* OR cyclic voltammetry OR linear sweep voltammetry OR voltammetry OR potentiostat OR galvanostat) AND (electrolyte OR electrode OR three-electrode OR reference electrode OR Ag/AgCl OR RHE) NOT (review OR meta-analysis OR survey OR perspective OR benchmarking OR standardization OR protocol) |
| Organic Synthesis | (cross-coupling OR cross coupling OR coupling OR arylation OR amination OR borylation OR Suzuki OR Heck OR Sonogashira OR Buchwald-Hartwig OR C-H activation OR photoredox OR electrochemical) AND (metal-catalyzed OR transition-metal-catalyzed OR catalyzed OR catalytic OR catalyst OR Pd OR Ni OR Cu) AND (ligand OR base OR solvent OR reaction conditions OR atmosphere) NOT (review OR meta-analysis OR survey OR perspective OR editorial) |
| Inorganic Materials | (solid-state synthesis OR solid state synthesis OR solid-state reaction OR solid state reaction OR conventional solid-state OR ceramic method OR mixed-oxide OR mixed oxide OR calcination OR sintering OR anneal* OR heat treatment OR heat-treat* OR firing OR fired) AND (ceramic* OR oxide* OR metal oxide* OR perovskite* OR spinel* OR ferrite* OR titanate*) NOT (review OR meta-analysis OR survey OR perspective) |
| Polymer Materials | (photopolymer* OR photopolymerization OR UV cur* OR UV-cur* OR UV curable OR photo cur* OR light cur* OR UV irradiation OR light irradiation OR photoinitiator OR photo-initiator OR UV lamp OR UV LED OR UV-LED) NOT (review OR meta-analysis OR survey OR perspective) |
| Category | Heading Patterns |
| Generic method headings | experimental section, experimental part, experimental details, experimental methods, experimental procedures, experimental setup, experimental design, experimental conditions, experiments, materials and methods, methods and materials, methods, methodology, materials preparation, sample preparation, preparation of samples, preparation of materials, preparation of compounds, preparation of polymers, synthesis and characterization, characterization, characterization methods, measurements, instrumentation, analytical methods |
| Cell culture | cell culture, cell culture methods, cell culture conditions, culture conditions, cell lines, in vitro experiments, cell treatment, cell treatments, incubation conditions |
| Immunotechnology | western blot, western blotting, immunoblot, immunoblotting, ELISA, flow cytometry, immunofluorescence, immunostaining, immunohistochemistry, antibodies, SDS-PAGE, protein extraction, sample processing |
| Electrochemistry | electrochemical methods, electrochemical measurements, electrochemical characterization, electrochemical testing, electrochemical evaluation, electrode preparation, electrolyte preparation, cell assembly, electrochemical setup, cyclic voltammetry, linear sweep voltammetry, chronoamperometry, chronopotentiometry, impedance spectroscopy, EIS |
| Organic synthesis | general procedure, typical procedure, synthetic procedure, synthetic details, synthesis of, preparation of, reaction conditions, catalytic conditions, substrate scope, optimization of reaction, optimization of conditions |
| Inorganic materials | solid-state synthesis, solid state synthesis, solid-state reaction, solid state reaction, calcination, sintering, sintered, annealing, annealed, firing, pellet preparation, powder preparation, ceramic processing, fabrication, materials fabrication |
| Domain | Retrieved Articles | Downloaded Articles | Method Pool Size | Full Pool Size |
| Cell Culture | 15,600 | 9,633 | 46,004 | 311,042 |
| Immunotechnology | 33,400 | 19,909 | 93,315 | 644,077 |
| Electrochemistry | 42,040 | 13,879 | 91,375 | 409,133 |
| Organic Synthesis | 14,420 | 7,669 | 30,371 | 197,506 |
| Inorganic Materials | 26,910 | 8,794 | 46,136 | 240,380 |
| Polymer Materials | 18,045 | 9,360 | 50,977 | 283,907 |
| Rejected constraint set | Rejection reason |
| Cell culture culture_medium.basal_medium RPMI-1640 culture_medium.serum.concentration 10% culture_medium.serum.identity fetal bovine serum cultured_cells.cell_type prostate cancer | The cell type value is too broad to define a stable cell culture target. |
| Electrochemistry electrolytes.chemical PBS measurement_method EIS scan_rate 50 mV/s target_species K 3 [Fe(CN) 6 ]/K 4 [Fe(CN) 6 ] | The setup is methodologically inconsistent because scan rate is not a stable defining parameter for EIS. |
| Immunotechnology blocking_solution.concentration 5% blocking_solution.identity skim milk protein_targets.target LIMK1 | The constraint set is dominated by a minor blocking condition and lacks stronger assay anchors such as sample type or normalization context. |
| Inorganic materials precursors alumina sintering.temperature 1600 ∘ C target_material Al 2 O 3 | The precursor and target material describe the same alumina system, making the constraint set redundant and weak as a retrieval request. |
| Organic synthesis solvents DME substrates N-(prop-2-yn-1-yl)benzamide transformation_type Suzuki coupling | The substrate and transformation type do not form a coherent reaction setup, so the solvent query is not well anchored. |
| Polymer materials photoinitiators.name dye polymer_system PEGDA wavelength 405 nm | The photoinitiator value is too generic to define a reliable PEGDA photocuring setup. |
| Rejected query and structured constraints | Scores and rejection reason |
| Cell culture Query: What seeding density is used for A549 cells cultured in 2D? Structured constraints: culture_mode 2D cultured_cells.cell_type A549 seeding_density null | Scores: F=1, C=1, U=0. Reason: The fixed context is too sparse. A549 cells in 2D culture define a common baseline setup, so the query is too generic to retrieve specific experimental evidence. |
| Immunotechnology Query: What normalization control other than AKT is used for PTEN Western blotting in tissue lysate with 30 g loading? Structured constraints: loading_amount 30 g normalization_control AKT protein_targets.target PTEN sample_type tissue lysate | Scores: F=1, C=1, U=0. Reason: AKT is a signaling protein rather than a standard normalization control, so the negated normalization-control constraint is scientifically unstable. |
| Organic synthesis Query: What solvent is used for cross-coupling of aryl halides? Structured constraints: substrates aryl halide transformation_type cross-coupling solvents null | Scores: F=1, C=1, U=0. Reason: The fixed conditions are too broad. Aryl-halide cross-coupling covers many reaction classes, catalysts, and bases, so the solvent target is not meaningfully constrained. |
| Polymer materials Query: Find photocuring experiments using TMPTA with DQ1 as the photoinitiator at 1.35 10 -3 –1.80 10 -3 M. Structured constraints: photoinitiators.loading between 1.35--1.80 10 -3 M photoinitiators.name DQ1 polymer_system TMPTA | Scores: F=1, C=1, U=0. Reason: The molarity-based loading range is an unstable retrieval anchor for this polymer curing setup, where photoinitiator loading is more commonly specified by mass or molar fraction. |
| Statistic | Cell. | Electro. | Immuno. | Inorg. | Organic | Polymer |
| Field-level templates | 37 | 55 | 29 | 408 | 111 | 82 |
| Constructed sets | 16,181 | 143,381 | 434,912 | 648,472 | 171,753 | 23,177 |
| After automatic filtering | 723 | 10,018 | 38,232 | 1,906 | 10,271 | 876 |
| Eligible for sampling | 723 | 7,083 | 38,232 | 1,906 | 10,271 | 876 |
| Sampled sets | 300 | 300 | 300 | 300 | 300 | 300 |
| Final evaluation queries | 284 | 201 | 264 | 247 | 252 | 267 |
| Model | Size | Architecture | Max | Max | Instruction | Version / Checkpoint | License |
| Sparse lexical retrieval | |||||||
| BM25 | N/A | Sparse | No | rank_bm25.BM25Okapi | Apache-2.0 | ||
| General-purpose dense retrieval | |||||||
| Sentence-BERT | 109M | Encoder | 384 | 384 | No | sentence-transformers/all-mpnet-base-v2 | Apache-2.0 |
| Harrier-OSS-v1 | 0.6B | Decoder | 32768 | 32768 | No | microsoft/harrier-oss-v1-0.6b | MIT |
| BGE-M3 | 568M | Encoder | 8192 | 8192 | No | BAAI/bge-m3 | MIT |
| Models | Query input |
| NV-Embed-v2, BGE-Reasoner-Embed, INF-X-Retriever, ReasonEmbed | Instruct: Given a web search query, retrieve paragraphs that answer the question Query: {query} |
| Qwen3-Embedding-8B, Harrier-OSS-v1, DIVER-Retriever | Instruct: Given a question, retrieve paragraphs that answer the question Query:{query} |
| Query Decomposition |
| Given a retrieval query, decompose it into a small set of simpler retrieval sub-queries. First, identify internally the query intent and the constraints in the query. Then generate the smallest non-redundant set of sub-queries that together cover the original constraints. Requirements: (1) Each sub-query must remain a natural question. (2) Each sub-query must preserve the original query intent. For an exploration query, it must still ask about the same field as the original query. For a reproduction query, it must still ask for a setup/protocol/paragraph satisfying constraints. (3) Each sub-query should keep only part of the original constraints, so that it is simpler than the original query. (4) Different sub-queries should reflect different retained constraints, not paraphrases of the same reduced query. (5) Together, the sub-queries should cover all original constraints. (6) Use as few sub-queries as possible. (7) Do not introduce new constraints or change the meaning of the original query. Examples: Query: What applied potential is used (vs SCE) at 25 ∘ C with an Au working electrode? Decomposition: [ “What applied potential is used (vs SCE) with an Au working electrode?”, “What applied potential is used (vs SCE) at 25 ∘ C?” ] Query: Find an electrochemistry experimental protocol describing cyclic voltammetry (CV) performed in acetate buffer using an AGCE working electrode. Decomposition: [ “Find an electrochemistry experimental protocol describing cyclic voltammetry (CV) performed in acetate buffer.”, “Find an electrochemistry experimental protocol describing cyclic voltammetry (CV) using an AGCE working electrode.” ] Return only a JSON-style list of plain question strings. Query: {query} |
| Structured Query |
| No generation prompt is used. We directly use the structured constraint representation as the retrieval input. |
| HyDE |
| Given the query, write a short scientific paragraph that would likely answer it. Preserve the key experimental constraints. Query: {query} |
| Reasoning-augmented Query |
| Original Query |
| Find 3D culture experiments in which C2C12 cells are grown in alginate. |
| Query Decomposition |
| [ “Find 3D culture experiments in which C2C12 cells are grown.”, “Find 3D culture experiments in which cells are grown in alginate.” ] |
| Structured Query |
| task=reproduction; culture_mode=3D; culture_substrate_or_coating=alginate; cultured_cells.cell_type=C2C12 |
| HyDE |
| Model | Size | Checkpoint | Ranking Format |
| BGE-Reranker-v2-m3 | 0.6B | BAAI/bge-reranker-v2-m3 | Pointwise |
| Rank-R1 | 7B | ielabgroup/Rank-R1-7B-v0.1 | Setwise |
| Rank1 | 7B | jhu-clsp/rank1-7b | Pointwise reasoning |
| DIVER-GroupRank-7B | 7B | AQ-MedAI/Diver-GroupRank-7B | Groupwise |
| ReasonRank-8B | 8B | lwhlwh/reasonrank-8B | Listwise reasoning |
| Qwen3-Reranker | 8B | Qwen/Qwen3-Reranker-8B | Pointwise |
| Retriever | Cell. | Immuno. | Electro. | Organic | Inorg. | Polymer | Avg. |
| BM25 | +0.002 | +0.000 | +0.001 | -0.010 | +0.005 | +0.003 | 0.000 |
| GTE-large | +0.031 | +0.001 | +0.005 | -0.008 | +0.024 | +0.010 | +0.011 |
| ChEmbed-prog | +0.019 | +0.004 | -0.002 | -0.015 | -0.009 | +0.002 | 0.000 |
| BGE-Reasoner-Embed | +0.034 | +0.003 | -0.003 | -0.001 | -0.012 | +0.003 | +0.004 |
| NV-Embed-v2 | +0.053 | +0.003 | -0.005 | +0.004 | -0.009 | +0.019 | +0.011 |
| Qwen3-Embedding-8B | +0.063 | +0.004 | -0.002 | -0.010 | -0.033 | +0.045 | +0.011 |
| Domain | Added Pos. (%) | nDCG@10 | |
| Cell Culture | 2.74 | +0.042 | 0.993 |
| Immunotechnology | 1.89 | +0.029 | 0.954 |
| Electrochemistry | 3.97 | +0.059 | 0.977 |
| Organic Synthesis | 1.60 | +0.012 | 0.993 |
| Inorganic Materials | 0.74 | +0.009 | 0.988 |
| Polymer Materials | 0.21 | -0.001 | 0.996 |
| Validator | Agreement | Avg. | ||
| Qwen3-235B | 66.48% | 0.2592 | 0.970 | |
| Claude Sonnet 4.6 | 90.20% | 0.6353 | 0.982 |
| Majority (2/3) | Unanimous (3/3) | ||||
| Domain | GPT-5.4 | ||||
| Cell Culture | 0.226 | 0.995 | 0.991 | ||
| Immunotechnology | 0.026 | 0.951 | 0.993 | ||
| Electrochemistry | 0.130 | 0.995 | 0.997 | ||
| Organic Synthesis | 0.171 | 0.998 | 0.997 | ||
| Inorganic Materials | 0.341 | 0.979 | 0.982 | ||
| Retriever | Cell. | Immuno. | Electro. | Organic | Inorg. | Polymer | Avg. |
| BM25 | -0.048 | -0.005 | -0.029 | -0.071 | -0.078 | -0.108 | -0.057 |
| BGE-Reasoner-Embed | -0.091 | -0.008 | -0.045 | -0.107 | -0.067 | -0.096 | -0.069 |
| NV-Embed-v2 | -0.095 | -0.010 | -0.056 | -0.109 | -0.070 | -0.140 | -0.080 |
| Qwen3-Embedding-8B | -0.080 | -0.026 | -0.047 | -0.102 | -0.121 | -0.128 | -0.084 |
| Condition | Method | |||
| Retriever | Absent | Present | Absent | Present |
| BM25 | 0.181 | 0.136 | 0.132 | 0.159 |
| Qwen3 | 0.356 | 0.285 | 0.288 | 0.293 |
| Query | Retrieved paragraph excerpt | Error category | Reason |
| Find 3D culture conditions for HCT116 cells in DMEM. | The HCT116 colon cancer cell line was collected for cell culture. RPMI1640 supplemented with 10% fetal bovine serum was prepared at 37 ∘ C. Tumor spheres of HCT116 cells were formed in suspension. Sphere Formation Media was composed of RPMI1640, epidermal growth factor, fibroblast growth factor-basic, B27, and bovine serum albumin. The cells were diluted to 2 10 4 /ml and cultured in 24-well plates. | Categorical | The paragraph describes HCT116 sphere culture, but the medium is RPMI/SFM rather than DMEM. |
| Find 3D culture conditions for HCT116 cells in DMEM. | HCT 116 cell lines were maintained in DMEM supplemented with 10% FBS, L-glutamine, penicillin, and streptomycin. HCT 116 cells were detached by trypsin-EDTA. Cells were cultivated at 37 ∘ C, 5% CO 2 , and 95% humidity. | Categorical | The paragraph gives DMEM culture conditions for HCT116 cells, but does not describe a 3D culture setup. |
| What basal medium other than RPMI is used for K-562 cells cultured in suspension? | The continuous cell line K562 was established by Lozzio and Lozzio. The suspension cell lines were cultivated using Roswell Park Memorial Institute (RPMI) 1640. The medium was supplemented with 10% fetal bovine serum and 1% Penicillin-Streptomycin. The cells were incubated at 37 ∘ C and 5% CO 2 . | Negation | The query asks for a basal medium other than RPMI, but the paragraph explicitly uses RPMI 1640. |
| What solvent other than DCM is used for Heck cross-coupling with Pd@ASP-EDTA-CS? | Reaction conditions: aryl halide, alkene, K 2 CO 3 , Pd@ASP-EDTA-CS, and solvent unless otherwise stated. Isolated yield. | Topic-only | The paragraph is about the relevant Heck reaction system, but it does not name a concrete solvent value. |
| Query |
| Find 3D culture experiments in which C2C12 cells are grown in alginate. |
| Structured Constraints |
| culture_mode 3D culture_substrate_or_coating alginate cultured_cells.cell_type C2C12 |
| Example Positive Paragraph |
| The voltage of the electrostatic droplet generator was optimized and two commercial cell attachment factors (gelatin and ECL) were tested to establish robust production of alginate microcapsules. A voltage of 5–5.25 kV for droplet generation was applied, resulting in spherical microcapsules and a homogenous size distribution ( 2–3%) (Supplementary Table S1). Furthermore, we explored the incorporation of gelatin and ECL into the alginate as a cell adhesion matrix in both serum (DMEM+FBS) and serum-free media (AIM-V), since alginate lacks peptide sequence that supports cell attachment. Overall, encapsulated C2C12 cells in alginate microcapsule cultured in serum-free media showed a decreasing trend in viability, but no significant difference, compared to that cultured in serum-supplemented media, demonstrating the potential of serum-free media for the alginate microcapsule cultivation. A significant difference in cell viability between the addition of two extracellular matrices (ECL vs. Gelatin) into alginate was observed. Alginate beads with ECL showed significantly higher cell viability than alginate alone and alginate with gelatin in both serum-supplemented and serum-free media cultivation (Supplementary Figure S1). Finally, the addition of ECL was chosen to enhance cell survival in alginate microcapsule culture systems. |
| Example Negative Paragraph |
| Query |
| What loading amount is used for Western blot detection of BECN1 in tissue protein extract? |
| Structured Constraints |
| protein_targets.target BECN1 sample_type tissue protein extract loading_amount null |
| Example Positive Paragraph |
| Tibial tissues were lysed with radioimmunoprecipitation assay buffer (Cell Signaling Technology, Danvers, MA) containing protease inhibitor cocktail. The concentration of proteins (10 g per lane) was determined using a Pierce™ BCA Protein Assay Kit. Proteins were separated by sodium dodecyl sulfate-polyacrylamide gel electrophoresis and were then transferred to polyvinylidene difluoride (PVDF) membranes at 15 V for 1 h at low temperature. Membranes were blocked with 5% skim milk for 2 h. Proteins were detected using the following primary antibodies at a 1:1000 dilution overnight at 4 ∘ C: anti-Beclin-1 (ab210498; Abcam, UK), anti-light chain 3B (LC3B; ab48394; Abcam), anti-GNB1 (ab137635; Abcam), anti-autophagy-related 5 (ATG5; ab108327; Abcam), anti-p62 (ab56416; Abcam), anti-Bcl-2 (ab194583; Abcam), anti-Bax (ab32503; Abcam), anti-cleaved caspase 3 (PA5-114687, Thermo Fisher Scientific) and anti-GAPDH (ab9485; Abcam) antibodies. Then, membranes were incubated with horseradish peroxidase-conjugated antirabbit IgG (Zsbio) for 2 h at room temperature. Finally, immunoreactions were evaluated with an enhanced chemiluminescence detection system. The protein levels were normalized to those of the corresponding GAPDH loading control. An Alpha Innotech Gel Imager (Alpha Innotech Corporation, San Leandro, CA) was used to visualize the specific bands, and the optical density of each band was measured using Image Pro Plus software (version X; Media Cybernetics, Silver Springs, MD). |
| Example Negative Paragraph |
| Query |
| Find cyclic voltammetry experiments targeting DCF in phosphoric acid electrolyte. |
| Structured Constraints |
| electrolytes.chemical phosphoric acid measurement_method CV target_species DCF |
| Example Positive Paragraph |
| The Cyclic Voltammetry (CV) measurements were carried out by GCE and aGCE and by SPCE and aSPCE to study the DCF electrochemical behavior. With the aGCEs, the following measurement parameters were used: time for equilibration: 15 s; potential step: 5.0 mV; potential range: 0.0–0.9 V; scan rate: 0.02–0.175 V s -1 . The CV signals were recorded on solution mol L -1 DCF at different pH and medium: pH 2.0, with mol L -1 hydrochloric acid; pH 2.0, with mol L -1 phosphoric acid; pH 4.7, with mol L -1 acetic buffer; pH 7.0, with mol L -1 phosphate buffer. |
| Example Negative Paragraph |
| Query |
| What additive is used in Suzuki-Miyaura coupling with Pd(OAc) 2 ? |
| Structured Constraints |
| catalysts.name Pd(OAc)2 transformation_type Suzuki-Miyaura coupling additives null |
| Example Positive Paragraph |
| General method for the synthesis of 1,1-diarylvinyl sulfones CA4 analogues (4a–u) [48][49][50]. To a stirred mixture of (E)-5-(1-iodo-2-tosylvinyl)-1,2,3-trimethoxybenzene 4A (100 mg, 0.211 mmol, 1 eq), different aryl boronic acids (30.64 mg, 0.253 mmol, 1.2 eq) and Na 2 CO 3 (44.73 mg, 0.422 mmol, 2 eq) in water and dimethylformalmide (1:2), with palladium acetate (Pd(OAc) 2 ) catalyst (0.0006 mmol, 0.005 eq) was added and reaction was stirred at room temperature under nitrogen for 3 h. The residue was purified by chromatography (EtOAc/hexanes) to afford the corresponding coupled product 4A-V (Fig. 4d). Antiproliferative assay. The cytotoxicity of synthesized compounds 4A-V were determined in triplicates by 3-[4,5-dimethylthiazol-2-yl]-2,5-diphenyltetrazolium bromide (MTT) assay [51,52]. The pale yellow coloured tetrazolium salt (MTT) reduced to a dark blue water-insoluble formazan by metabolically active cells and the product was measured quantitatively after solubility in DMSO. The absorbance of the soluble formazan is directly proportional to the number of viable cells. A panel of four cancer cell lines were used for testing the in-vitro cytotoxicity. These cell lines were grown in Dulbecco’s modified Eagle’s medium (DMEM) containing non-essential amino acids and 10% FBS. All the cells were maintained under humidified conditions of 5% CO 2 atmosphere at 37 ∘ C in a CO 2 incubator (Model Galaxy 170S, Eppendorf, USA). The 96-well micro titre plates were incubated for 24 h prior to addition of the experimental compounds. Cells were treated with vehicle alone (DMSO) or compounds (drugs (1 g/mL) were dissolved in 100 L DMSO previously) at different concentrations (0.1, 1, 10 and 25 M) of test compounds for 48 h. The assay was completed with the addition of MTT (5%, 10 L) and incubated for 60 min at 37 ∘ C. The supernatant was aspirated and plates were air dried and the MTT-formazan crystals were dissolved in 100 L of DMSO. |
| Example Negative Paragraph |
| Query |
| What sintering time is used for MgO-doped mullite? |
| Structured Constraints |
| dopant_or_additive.identity MgO target_material mullite sintering.time null |
| Example Positive Paragraph |
| Different amounts of MgO were added to the mullite stoichiometric composition (71.8 wt% of Al 2 O 3 and 28.2 wt% of SiO 2 ), as follows: 0, 0.8, 1.6, 2.4, 3.1, and 4.0 wt%. These compositions were identified as M0, M1, M2, M3, M4, and M5, respectively. The compositions were mixed in a ball mill for 6 h. Next, they were dried with hot air flow and deagglomerated in an 80 mesh nylon sieve. To evaluate the densification of the compositions with different MgO amounts, dilatometry tests were carried out. For the tests, a sample of each composition was formed using uniaxial pressing, in a cylindrical mold. From the densification behavior, the determined sintering temperatures were used. The dilatometry was performed up to 1400 ∘ C at a 10 ∘ C/min heating rate, in an air atmosphere, with a dilatometer (Setsys Evolution, Setaram). To study the material sinterability, the samples were formed similarly to those used in the dilatometry. Then, the green samples were sintered at a 10 ∘ C/min constant heating rate up to the sintering temperature (1350, 1450, and 1550 ∘ C), with a 4 h holding time. |
| Example Negative Paragraph |
| Query |
| What PEGDA loading is used with alginate as the polymer system? |
| Structured Constraints |
| crosslinking_agent.name PEGDA polymer_system alginate crosslinking_agent.loading null |
| Example Positive Paragraph |
| Effects of precursor compositions: To examine the effect of glycerol ratios, mixed solvents were prepared with glycerol mass ratios ( ) of 25%, 40%, 50%, and 60%, dissolving alginate at a concentration of 1.128% (w/w) in each. Acrylamide monomers (19.33% w/w) and PEGDA crosslinkers (0.67% w/w, ) were added, and the mixtures were stirred overnight to obtain a clear solution. The solutions were divided into degassed and non-degassed groups. For modulus measurements, 3 g of each solution was sonicated for 2, 3, 4, 6, and 8 min at 53 W cm -2 . For fracture energy measurements, 3 g of solution was sonicated for 6 min at the same intensity, followed by post-crosslinking as described above. |
| Example Negative Paragraph |
| Model | Prompt |
| BGE-Reranker-v2-m3 | No natural-language prompt is added. We use the official cross-encoder scoring interface with paired inputs: Query: {query} Passage: {paragraph} The model returns a relevance score for each query–paragraph pair. |
| Rank-R1 | System: A conversation between User and Assistant. The user asks a question, and the Assistant solves it. The assistant first thinks about the reasoning process and then provides the answer. User: Given the query: “{query}”, which of the following documents is most relevant? {passages} After completing the reasoning process, provide only the label of the most relevant document to the query, enclosed in square brackets. For example, if the third document is the most relevant, the answer should be: [3]. The official setwise implementation compares a set of candidates and uses the selected label during sorting. |
| Rank1 | Determine if the following passage is relevant to the query. Answer only with “true” or “false”. Query: {query} Passage: {paragraph} The model first generates a reasoning trace and then makes a binary relevance judgment. We use the probability of the “true” token as the reranking score. |
| DIVER-GroupRank-7B | System: Your task is to evaluate and rank documents based on how well they help answer the given query. Usefulness and helpfulness are the primary criteria, followed by topical relevance. For each document, assess how directly it addresses the user’s intent and how much actionable information it provides. User: I will provide you {TOPK} documents, each indicated by a numerical identifier. Score these documents based on their usefulness and relevance to the query. Query: {query} Documents: {passages} The output should contain a brief reasoning process followed by JSON scores, with document identifiers as keys and integer scores from 0 to 10 as values. |
| ReasonRank-8B | System: You are RankLLM, an intelligent assistant that can rank passages based on their relevance to the query. Given a query and a passage list, first think about the reasoning process and then provide the reranked passage list. User: I will provide you with {N} passages, each indicated by a numerical identifier. Rank the passages based on their relevance to the search query: {query}. {passages} Search Query: {query}. Rank the {N} passages above based on their relevance to the search query. All passages should be included and listed using identifiers in descending order of relevance. The format of the answer should be [] [], e.g., [2] [1]. |
| Qwen3-Reranker | System: Judge whether the document meets the requirements based on the query and the instruction provided. The answer can only be “yes” or “no”. User input: Instruct: Given a web search query, retrieve relevant passages that answer the query Query: {query} Document: {paragraph} We use the probability of the “yes” token as the reranking score. |