SACRED: A Faithful Annotated Multimedia Multimodal Multilingual Dataset for Classifying Connectedness Types in Online Spirituality

Qinghao Guan; Yuchen Pan; Donghao Li; Zishi Zhang; Yiyang Chen; Lu Li; Flaminia Canu; Emilia Volkart

cs.CLMar 28, 2026

SACRED: A Faithful Annotated Multimedia Multimodal Multilingual Dataset for Classifying Connectedness Types in Online Spirituality

Authors: Qinghao Guan, Yuchen Pan, Donghao Li, Zishi Zhang, Yiyang Chen, Lu Li, Flaminia Canu, Emilia Volkart,

Organizations: Department of Communication and Media Research, University of Zurich, Zurich, Switzerland · Department of Computational Linguistics, University of Zurich, Zurich, Switzerland · Department of Informatics, University of Zurich, Zurich, Switzerland · Ping An Technology, Shanghai, China

Abstract

In religion and theology studies, spirituality has garnered significant research attention for the reason that it not only transcends culture but offers unique experience to each individual. However, social scientists often rely on limited datasets, which are basically unavailable online. In this study, we collaborated with social scientists to develop a high-quality multimedia multi-modal datasets, SACRED, in which the faithfulness of classification is guaranteed. Using SACRED, we evaluated the performance of 13 popular LLMs as well as traditional rule-based and fine-tuned approaches. The result suggests DeepSeek-V3 model performs well in classifying such abstract concepts (i.e., 79.19% accuracy in the Quora test set), and the GPT-4o-mini model surpassed the other models in the vision tasks (63.99% F1 score). Purportedly, this is the first annotated multi-modal dataset from online spirituality communication. Our study also found a new type of connectedness which is valuable for communication science studies.

Figures & tables

Figure 1: Workflow Diagram of SACRED Dataset Annotation and LLMs Evaluation

Concept	Description	Example
Connectedness to Self	authenticity, inner harmony and inner peace, consciousness, self-knowledge, and experiencing and searching for meaning in life	Just spent the morning in deep meditation and self-reflection. Amazing how connecting with my inner self brings clarity and peace to my spirit. #InnerPeace #SelfAwareness
Connectedness to Others	compassion, caring, gratitude, and wonder	Just returned from volunteering at the local shelter, and my spirit feels so enriched! Connecting with others through acts of kindness truly uplifts the soul. #CommunityLove #MentalWellbeing
Connectedness to Nature	the deep sense of relationship that individuals feel with the natural world and understanding of humanity’s place within the broader ecological system	Hiking through the forest today, I felt an incredible connection to the earth. The serenity of nature rejuvenates my spirit and calms my mind. #NatureHealing #SoulConnection #EarthSpirit
Connectedness to Transcendence	something or someone beyond the human level, such as the universe, transcendent reality, a higher power, or God	Attended a beautiful church service this morning. As we sang together and lifted our voices in prayer, I felt an incredible connection to God that filled my spirit with joy and hope. Truly a transcendent experience. #Faith #DivineConnection #SpiritualUpliftment

Table 1: Concepts of Connectedness and Examples

Figure 2: Instagram Image Example: The Left One Only Contains Visuality and The Right One Only Contains Textual Information

Source	Spirituality-related (%)
subreddit	17.76%
Instagram captions	21.27%
Instagram images	28.87%
Quora topics	53.94%

Table 2: Percentage of Spirituality-related Content by Platform

Figure 3: Language Distribution of Instagram Caption Excluding English

Figure 4: Language Distribution of Instagram Images Excluding English

Model	Reddit		Instagram Caption		Instagram Image	Quora
	Zero-	Definition-	Zero-	Definition-	Definition-	Zero-	Definition-
	shot	injected	shot	injected	injected	shot	injected
Rule-based	22.67		20.27	-	-	10.74	-
Fine-tuning BERT	53.33	-	46.62	-	-	48.32	-
LLaMA3.1	58.00	66.00	45.27	54.05	-	53.02	52.35
Reka Core	52.00	55.33	43.00	45.33	-	31.00	28.33

Table 3: Accuracy Performance Comparison Across Platforms and Methods

Figure 5: Heatmap of Annotation Agreement Among Annotators

Type	Reddit Posts	Instagram Captions	Instagram Images	Quora Answers
Self	542	766	1166	436
Others	28	146	89	109
Nature	22	80	83	12
Transcendence	282	444	873	1503

Table 4: Connectedness Types Across Different Platforms

Figure 6: Unique Words in Four Types of Connectedness

Figure 7: Validation Accuracy Curve over 30 Epochs

Model	Recall (%)	Precision (%)	F1 (%)
GPT-4o-mini	63.42	65.66	63.99
GPT-4o	60.46	62.87	61.34
GPT-4.1	48.42	59.68	49.83
Gemini-2.5	55.07	63.79	56.99
Claude-Sonnet	56.03	60.60	57.18
LLaVA-1.5	34.83	47.44	38.96

Table 12

Multimodal Dataset

See all 49 papers

A large-scale corpus of religious radio broadcast transcripts from webstream recordings in the United States
Jul 28, 2026Samuel Bestvater, Athena Chapekis, Skyler Seets +3Speech Corpus Construction
1Pew Research Center, Washington DC, USA
When AI Takes Sides on Questions of Faith: Persistent Asymmetries in AI-Mediated Faith Guidance
May 21, 2026Brett Israelsen, Sheryl Carty, Josh Coates +3Social Bias in Language Models Language Model Bias Evaluation
Brigham Young University · B. H. Roberts Foundation
Measuring Faithfulness Depends on How You Measure: Classifier Sensitivity in LLM Chain-of-Thought Evaluation
Mar 20, 2026Richard J. YoungCoT Faithfulness LLM Evaluation
University of Nevada, Las Vegas, Department of Management, Entrepreneurship and Technology, Lee Business School, Las Vegas, NV, USA · DeepNeuro AI, Las Vegas, NV, USA

Jul 28, 2026J/K move · Enter open · S save

A large-scale corpus of religious radio broadcast transcripts from webstream recordings in the United States

Samuel Bestvater, Athena Chapekis, Skyler Seets, Anna Lieb+2

1Pew Research Center, Washington DC, USA

Speech Corpus Construction

Open paper AskPDF arXiv

Religious radio is a widespread but understudied form of mass communication in the United States, and content-level analysis of it has been constrained by the absence of large-scale transcript data. This Data Descriptor presents a corpus of transcribed English-language religious radio broadcasts captured from live webstreams over a one-month period in July 2025. Fifteen-minute segments were recorded on a rolling schedule from 785 distinct streams, which together rebroadcast the signals of more than two thousand AM and FM stations, yielding over 700,000 recordings and more than 60 million diarized lines of speech. Each recording was transcribed and speaker-diarized with an automated pipeline, and segmented and labeled by programming format and topic using a large language model. The corpus is organized as linked tables of stream metadata, recording metadata, and transcript lines. It supports descriptive study of religious broadcasting across regions and traditions, analysis of how social and political issues are discussed in religious media, and speech-processing research in an underrepresented domain.

Abstract

Multimodal Dataset

Explore similar work

A large-scale corpus of religious radio broadcast transcripts from webstream recordings in the United States

When AI Takes Sides on Questions of Faith: Persistent Asymmetries in AI-Mediated Faith Guidance

Measuring Faithfulness Depends on How You Measure: Classifier Sensitivity in LLM Chain-of-Thought Evaluation