ARISMA: Guidelines for AI- and LLM-Assisted Systematic Reviews, Scoping Reviews, and Mapping Studies
Organizations: University of Southern Denmark, Odense 5230, DK
Abstract
Systematic reviews, scoping reviews, mapping studies, and related evidence syntheses are increasingly difficult to conduct with fully manual workflows as search volumes, update cycles, and synthesis requirements continue to expand. At the same time, artificial intelligence, machine learning, and large language models are rapidly entering review practice across query formulation, screening, extraction, categorization, appraisal support, and reporting. Yet the empirical evidence remains uneven, task-dependent, and insufficient to justify unconstrained automation. Existing standards such as PRISMA 2020, PRISMA-S, PRISMA-ScR, PRISMA-P, PRESS, and SWiM remain essential, but none provides an end-to-end operational standard for when AI use is methodologically appropriate, how it should be validated, which review decisions must remain human-led, and how AI involvement should be reported so that readers can audit it. This paper proposes ARISMA, an AI Reporting and Integration standard for Systematic Methods and Analysis. ARISMA treats AI as an inspected, benchmarked, logged, and reversible assistant rather than an autonomous reviewer. It is built around one governing principle: every consequential scientific decision must remain human-interpretable, human-auditable, and human-accountable. The paper contributes a lifecycle taxonomy, process guidance, stepwise recommendations across the review pipeline, a governance and provenance model, a tool-support framework, an AI-integrated reporting checklist, and a validation matrix. It also addresses legal, privacy, infrastructure, and sustainability considerations. The framework was iteratively refined through structured expert consultation. The result is a practical and auditable guideline for responsible AI-assisted evidence synthesis.
Figures & tables
| Item | What should be reported | Additional AI and LLM reporting required |
|---|---|---|
| Title | Identify the review type clearly | State if the review was AI-assisted |
| Abstract | Structured summary of rationale, methods, results, and conclusions | Name the AI-supported steps and the level of human oversight |
| Rationale | Why the review was needed | Why AI was considered necessary or useful for this workflow |
| Objectives | Explicit review question and scope | Which steps AI was allowed to influence and which were human-only |
| Review design | Systematic, scoping, mapping, EGM, rapid, or other | Why the chosen AI role is appropriate for that review family |
| Eligibility criteria | Full inclusion and exclusion criteria | How criteria were translated into prompts, rules, or labels |
| Step | What must be validated | Minimum ARISMA evidence |
|---|---|---|
| Search-term generation | Recall of known or sentinel studies | Pilot benchmark set, documented failures, revised search after peer review |
| Deduplication | False positives and false negatives | Tool log plus human audit of uncertain duplicate clusters |
| Title and abstract screening | Sensitivity for relevant records | Human-coded pilot set, error review, explicit rule for handling AI exclusions |
| Full-text screening | Justification quality and false exclusions | Human consensus sign-off on all final exclusions |
| Extraction of descriptive fields | Agreement with source papers | Random or full audit, depending on field criticality |
| Extraction of numeric fields | Exactness of synthesis-critical values | Full human verification of all values used in aggregation or meta-analysis |