CiteLLM: An Agentic Platform for Trustworthy Scientific Reference Discovery
Organizations: Hong Kong Polytechnic University, Hong Kong, China · Beijing University of Posts and Telecommunications, Beijing, China · Independent Researcher, Hong Kong, China · Swiss Federal Technology Institute of Lausanne (EPFL), Lausanne, Switzerland ·
Abstract
Large language models (LLMs) have created new opportunities to enhance the efficiency of scholarly activities; however, challenges persist in the ethical deployment of AI assistance, including (1) the trustworthiness of AI-generated content, (2) preservation of academic integrity and intellectual property, and (3) protection of information privacy. In this work, we present CiteLLM, a specialized agentic platform designed to enable trustworthy reference discovery for grounding author-drafted claims and statements. The system introduces a novel interaction paradigm by embedding LLM utilities directly within the LaTeX editor environment, ensuring a seamless user experience and no data transmission outside the local system. To guarantee hallucination-free references, we employ dynamic discipline-aware routing to retrieve candidates exclusively from trusted web-based academic repositories, while leveraging LLMs solely for generating context-aware search queries, ranking candidates by relevance, and validating and explaining support through paragraph-level semantic matching and an integrated chatbot. Evaluation results demonstrate the superior performance of the proposed system in returning valid and highly usable references.
Figures & tables
| Method | Validity (%) | Precision (%) | Usability (%) | ||
|---|---|---|---|---|---|
| Human | GPT-5 | Human | GPT-5 | ||
| CiteLLM (ours) | 100 | 84.0 | 87.5 | 87.5 | 92.5 |
| Google Scholar | 100 | 64.0 | 73.5 | 80.0 | 85.0 |
| ChatGPT | 74 | 84.5 | 91.9 | 30.0 | 37.5 |
| *Note: Precision for ChatGPT is evaluated among the valid references. | |||||
AI-Assisted Scientific Research
AI-assisted scientific research uses machine learning and AI agents to support discovery, analysis, experimentation, and scientific communication.
See all 253 papers