IR Evaluation

Information Retrieval evaluation studies whether search systems find useful documents and evidence, and how reliably their performance can be measured. Common approaches use judged benchmark collections and metrics such as precision, recall, mean reciprocal rank, and nDCG to compare lexical methods like BM25, dense and hybrid retrievers, and rerankers. Benchmarks increasingly test specialized domains, multilingual and multimodal search, temporal change, and retrieval within RAG and agent workflows, alongside efficiency, robustness, and coverage. A central challenge is that aggregate relevance scores and automated judgments can miss incomplete evidence, biased or outdated benchmarks, and whether retrieved material actually helps downstream decisions.