Conversational AI-Enhanced Exploration System to Query Large-Scale Digitised Collections of Natural History Museums

Yiyuan Wang; Andrew Johnston; Zoë Sadokierski; Rhiannon Stephens; Shane T. Ahyong

cs.HCMar 11, 2026

Conversational AI-Enhanced Exploration System to Query Large-Scale Digitised Collections of Natural History Museums

Authors: Yiyuan Wang, Andrew Johnston, Zoë Sadokierski, Rhiannon Stephens, Shane T. Ahyong

Organizations: School of Computer Science, University of Technology Sydney, Australia · School of Design, University of Technology Sydney, Australia · Australian Museum Research Institute, Australian Museum, Australia

Abstract

Recent digitisation efforts in natural history museums have produced large volumes of collection data, yet their scale and scientific complexity often hinder public access and understanding. Conventional data management tools, such as databases, restrict exploration through keyword-based search or require specialised schema knowledge. This paper presents a system design that uses conversational AI to query nearly 1.7 million digitised specimen records from the life-science collections of the Australian Museum. Designed and developed through a human-centred design process, the system contains an interactive map for visual-spatial exploration and a natural-language conversational agent that retrieves detailed specimen data and answers collection-specific questions. The system leverages function-calling capabilities of contemporary large language models to dynamically retrieve structured data from external APIs, enabling fast, real-time interaction with extensive yet frequently updated datasets. Our work provides a new approach of connecting large museum collections with natural language-based queries and informs future designs of scientific AI agents for natural history museums.

Figures & tables

Figure 1. Development process of the Australian Museum Collection Explorer.

Figure 2. Photos taken at the workshops (top) and interview questions in focus-group activities (bottom).

Expressed Need	Current Challenge	Design Goal
Communicate the vast scale and diversity of back-of-house collections	Collections are invisible to the public; the volume and richness of data is difficult to convey	Comprehensibility of scale : Make the scale of collections immediately perceptible and understood
Reduce barriers to accessing scientific and research information	Non-specialised audiences lack both technical database skills and natural science domain knowledge	Query accessibility : Enable information retrieval without technical expertise
Automate audience–expert message flow (i.e. “Ask an Expert” service)	Manual workflows to answer web-submitted questions are not scalable and require multi-step staff routing	Response reliability : Deliver accurate, collection-specific information efficiently
Reach diverse audiences including remote and underserved communities	Physical museum visits and staff consultations have geographic and temporal constraints	Ubiquitous access : Enable exploration of collections from anywhere, at any time

Table 1. Mapping stakeholder requirements to design goals. A three-column table mapping expressed needs from focus group attendees to current challenges and resulting design goals. Four rows cover: communicating collection scale leading to comprehensibility of scale; reducing access barriers leading to query accessibility; automating expert services leading to response reliability; and reaching diverse audiences leading to ubiquitous access.

Figure 3. The first interactive prototype we developed (left); ALA Lens (middle); Museum Animal Factsheets (right).

Figure 4. The second interactive prototype we developed: a bird collection explorer consisting of an initial version of the interactive map (left) and the conversational agent (right).

Item	Summary
Age ( $M\pm SD$ )	$51.6\pm 24.9$ years
Gender	7 female, 5 male
Prior AI experience	4 regular, 5 limited, 3 none
In-person museum engagement	All weekly (volunteer work)
Front-of-house exhibition visits	2 monthly, 3 bimonthly, 7 few times per year
Remote museum engagement	3 weekly, 2 monthly, 5 few times per year, 2 never

Table 2. Summary of participant profile in Design Iteration II (

N=12

Figure 5. An overview of the system architecture for the Australian Museum Collection Explorer.

Figure 7. Examples of collection data queries: searching for sugar gliders collected from 2000–2010 (left); requesting a count of Christmas beetles from NSW (right). Both responses include direct links to ALA (information source) for verification and further exploration.

Figure 8. Image output example 1: spotting a Macleay’s Swallowtail record on the map and requesting additional images of this species.

Figure 9. Image output example 2 (left): requesting fish specimens from Queensland. Image input example (right): identifying birds from a photograph taken in the wild as crested pigeons, with descriptive information and a link to related museum records.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Function	Purpose	Key Parameters
search_specimens	Query specimen records with filters	scientific_name , common_name , state_province , locality , year_range , has_image
get_specimen_statistics	Return aggregated counts and faceted distributions	scientific_name , common_name , include_facets
get_specimen_by_id	Retrieve detailed specimen information	specimen_id (required)

Appendix

Table 3. Summary of tool definitions

Cultural Heritage

Cultural heritage uses AI to document, analyze, preserve, and interpret artifacts, sites, archives, and living traditions.

See all 57 papers

CultureBot: A Framework for Retrieval-Augmented Exploration of Cultural Heritage Data
Oct 12, 2026Vasileios Karampinis, Angelos Vlachos, Giorgos Filandrianos +4Digital Cultural Heritage Cultural Heritage
School of Electrical and Computer Engineering, AILS Laboratory National Technical University of Athens
The Living Library: Transforming Archival Collections into Conversational Knowledge Systems -- Lessons from the Theodore Roosevelt Presidential Library
Sep 8, 2026Pengce Wang, Lucia Ronchi Darre, Matt Briney +8Library and Information Science Conversational Agents
Microsoft · TRPL
Multi-Agent Discovery and Resource-Aware Autonomous Exploration of Scientific Datasets
Aug 22, 2026Aashish Panta, Hugo Lee, Giorgio Scorzelli +2AI Agents for Scientific Discovery Multi-Agent Systems
University of Utah Salt Lake City, Utah, USA · Jet Propulsion Laboratory, Caltech Pasadena, CA, USA

Oct 12, 2026J/K move · Enter open · S save

CultureBot: A Framework for Retrieval-Augmented Exploration of Cultural Heritage Data

Vasileios Karampinis, Angelos Vlachos, Giorgos Filandrianos, Maria Lymperaiou+3

School of Electrical and Computer Engineering, AILS Laboratory National Technical University of Athens

Digital Cultural Heritage Cultural Heritage Graph Retrieval-Augmented Generation Evidence Retrieval Hybrid IR Knowledge Graph Question Answering

Open paper AskPDF arXiv

Digital cultural heritage collections encode rich relations among objects, places, periods, providers, images, and source institutions, yet users typically access them through keyword search and faceted browsing. We present CULTUREBOT, an interactive framework for natural-language exploration of cultural heritage collections that combines knowledge-graph-guided retrieval, lexical-semantic retrieval, and source-linked generation. CULTUREBOT maps user queries to structured catalogue relations, enriches retrieved records with textual evidence, and generates answers that link back to the underlying repository records. We instantiate the framework on 81,852 SearchCulture records and release an online demonstration. In an expert evaluation involving six cultural-heritage specialists, 55 persona-grounded queries, and 250 evaluations, the graph-guided system was rated significantly higher than two general-purpose assistants and a non-graph ablation, with the largest gains in evidence grounding, diversity, reasoning, and completeness. These results support graph-guided retrieval as a practical way to expose the relational structure of cultural collections through conversational access while preserving links to catalogue evidence.

Abstract

Cultural Heritage

Explore similar work

CultureBot: A Framework for Retrieval-Augmented Exploration of Cultural Heritage Data

The Living Library: Transforming Archival Collections into Conversational Knowledge Systems -- Lessons from the Theodore Roosevelt Presidential Library

Multi-Agent Discovery and Resource-Aware Autonomous Exploration of Scientific Datasets