cs.AINov 13, 2026

Evaluating Large Language Models for Scientific Paper Search: Retrieval Success and Inference Latency on Mac Studio and NVIDIA A100

Authors: Frederic Andriantseheno Rakoto, Yannis Bendi-Ouis, Xavier Hinaut

Organizations: Mnemosyne · Mnemosyne Inria Bordeaux, France

Abstract

Scientific literature search can benefit from locally deployed LLM assistants, which reduce reliance on hosted model providers while giving research teams greater control over their data and infrastructure. This work investigates whether open-weight models can support web-assisted paper retrieval under practical computing constraints. We first characterize inference latency as the input context grows from 512 to 128k tokens on a Mac Studio and a server equipped with two NVIDIA A100 GPUs. We then evaluate eight models on the retrieval of 70 papers from seven scientific repositories, using queries that represent three levels of domain knowledge. The results show that retrieval success depends strongly on the selected model and the specificity of the query. And that A100 GPUs are generally faster than the Mac Studio, but its unified memory allows larger models that could not be loaded in the tested GPU configuration.

Explore similar work

CardsList
  1. Do We Need Bigger Models for Science? Task-Aware Retrieval with Small Language Models

    Apr 2, 2026Florian Kelber, Matthias Jobst, Yuni Susanti +1AI-Assisted Scientific ResearchRetrieval-Augmented Generation

  2. Can Large Language Models Review the Scientific Literature? A Test Against a 102-Article Benchmark

    Nov 3, 2026Abdullah Al Kalbani, David M. Schultz, Alexandra K. Anderson-Frey +2AI-Assisted Scientific ResearchLLM Evaluation

  3. Evaluating Open-Weight LLMs for Turkish Domain Documents Under Retrieval and Hardware Constraints

    Sep 23, 2026Imtiaz Ul Hassan, Öykü Akbulut, Onur Kaya +4Multilingual Language Model EvaluationLLM Evaluation