TaigiSpeech: A Low-Resource Real-World Speech Intent Dataset and Preliminary Results with Scalable Data Mining In-the-Wild

Kai-Wei Chang; Yi-Cheng Lin; Huang-Cheng Chou; Wenze Ren; Yu-Han Huang; Yun-Shao Tsai; Chien-Cheng Chen; Yu Tsao

cs.CLMar 23, 2026

TaigiSpeech: A Low-Resource Real-World Speech Intent Dataset and Preliminary Results with Scalable Data Mining In-the-Wild

Authors: Kai-Wei Chang, Yi-Cheng Lin, Huang-Cheng Chou, Wenze Ren, Yu-Han Huang, Yun-Shao Tsai, Chien-Cheng Chen, Yu Tsao,

Organizations: Massachusetts Institute of Technology, USA · National Taiwan University, Taipei, Taiwan · Signal Analysis and Interpretation Laboratory (SAIL), University of Southern California, USA · Academia Sinica, Taiwan ·

Abstract

Speech technologies have advanced rapidly and serve diverse populations worldwide. However, many languages remain underrepresented due to limited resources. In this paper, we introduce TaigiSpeech, a real-world speech intent dataset in Taiwanese Taigi (aka Taiwanese Hokkien/Southern Min), which is a low-resource and primarily spoken language. The dataset is collected from older adults, comprising 21 speakers with a total of 3k utterances. It is designed for practical intent detection scenarios, including healthcare and home assistant applications. To address the scarcity of labeled data, we explore two data mining strategies with two levels of supervision: keyword match data mining with LLM pseudo labeling via an intermediate language and an audio-visual framework that leverages multimodal cues with minimal textual supervision. This design enables scalable dataset construction for low-resource and unwritten spoken languages. TaigiSpeech will be released under the CC BY 4.0 license to facilitate broad adoption and research on low-resource and unwritten languages. The project website and the dataset can be found on https://kwchang.org/taigispeech.

Figures & tables

Category	Details
Speakers	21 (Male: 8, Female: 13)
Age Range	54–78 (Mean: 67.9)
Emergency Intents
1. SOS_CALL	Calls for emergency help.
2. BREATH_EMERG.	Reports breathing difficulty.
3. FALL_HELP	Indicates a fall and requests help.

Table 1: Statistics and intent description of TaigiSpeech dataset.

Figure 1: Primary language use among residents in Taiwan. Elders (aged 65+) have 64.9% reporting Taiwanese as their primary language. Data source: 2020 Population and Housing Census, DGBAS, Taiwan [ 5 ] .

Corpus	Year	Intent/ Command	# Spk. (Gender)	Age	#Utt.	Hours	Language	Smart Home	Emergency
CEMO (Corpus 2) [ 6 ]	2005	Emergency	410 (256F; 154M)	–	–	20	French	–	✓
HIS [ 7 ]	2010	Emergency	15 (6F; 9M)	24-43	0.67k	0.14	French	✓	–
CARES [ 8 ]	2013	Emergency	40 (19M; 21F)	23–91	–	53.3	Canadian English	–	✓
HIS-Distress Call [ 9 ]	2013	20 Distress	10 (7M; 3F)	$\sim$ 37.2	450	–	French	–	✓
SWEET-HOME [ 10 ]	2014	Command+Emergency	71 (Unknown)	19–91	9.2k	33	French	✓	✓
ITAAL [ 11 ]	2015	Commands+Distress	20 (10F; 10M)	–	2.4k	1.1	Italian	✓	✓

Table 2: Spoken intent and command corpora across languages and application settings. (# = number of; Spk. = speakers; F = female; M = male; Utt. = utterances; TTS = text-to-speech synthesized audio).

Corpus	Recording Condition	Speech Style	Target Task	Hours	Speaker Type
SuiSiann [ 37 ]	Studio	Reading speech	TTS	20	1 female
TAT [ 38 ]	Studio	Reading speech	ASR	100	92 males, 108 females
TAT-TTS [ 38 ]	Studio	Reading speech	TTS	40	2 males, 2 females
CommonVoice [ 39 ] *	Crowdsourced	Reading speech	ASR	23.67	Diverse speakers
ML-SUPERB [ 40 ] **	TV production	Scripted dialogue	ASR	1.5	Professional Actors
TG-ASR [ 41 ]	TV production	Scripted dialogue	ASR	30	Professional Actors

Table 3: Overview of Taiwanese Hokkien speech corpora across recording conditions and speech styles.

Figure 2: Recording app interface. The English translations are shown for illustrative purposes in this paper. All recordings were conducted under the researcher’s supervision to assist elderly participants.

Intent	Count	Duration (s)	Std. (s)	Total (min)
1. SOS_CALL	387	7.78	3.28	50.16
2. FALL_HELP	385	7.77	3.73	49.88
3. BREATH_EMERG	385	8.22	3.30	52.71
4. PAIN_GENERAL	385	7.91	3.73	50.79
5. CALL_CONTACT	385	7.04	5.30	45.16
6. LIGHT_ON	384	5.95	2.81	38.08

Table 4: Intent-level statistics of the TaigiSpeech dataset.

Figure 3: Age distribution of speakers in the TaigiSpeech corpus. Our study focuses on elderly participants.

Figure 4: The distribution of ambient noise level (dB).

Intent	Scenario Prompt (Imagined Context)	Example Spoken Utterance (English Translation)
Emergency Intents
1.SOS_CALL	You are at home and notice smoke and flames spreading.	“Ahh! Fire! It’s on fire! Help!”
2.FALL_HELP	You slipped in the bathroom and cannot stand up.	“Ah! I fell down… I can’t get up. Please help me!”
3.BREATH_EMERG	You suddenly feel chest tightness and difficulty breathing.	“Ah… my chest feels tight. I can’t breathe well…”
4.PAIN_GENERAL	You suddenly feel severe stomach pain and cannot move.	“Ahh… my stomach… it hurts so much. I can’t move…”
Non-emergency Functional Intents

Table 5: Examples of scenario prompts and corresponding elicited spoken utterances in the TaigiSpeech recording protocol. The utterances shown here are English translations of the original Taiwanese Hokkien speech.

Figure 5: Overview of the explored data mining methods. Left: Keyword Match Data Mining. Right: Audio-Visual Data Mining.

Intent	Total (#)	True (#)	False (#)
1.SOS_CALL	4114	2801	1313
2.FALL_HELP	3017	165	2852
3.BREATH_EMERG	614	121	493
4.PAIN_GENERAL	5729	3054	2675
5.CALL_CONTACT	5228	2203	3025
6.LIGHT_ON	157	33	124

Table 6: Statistics (number of sentences) of LLM pseudo labeling in the Keyword Match Mining method.

	Keyword Mining	AV Mining
# Target Classes	5	2
Train	11.3k	3.6k
Val	1.4k	400
Test (Drama)	1.4k
Test (Taigi)	960
Taigi-adapt (Independent-speaker Split)

Table 7: Data statistics under different mining strategies and Taigi adaptation setting.

Scenario		Keyword Match Mining			Audio-Visual Mining			Scratch
Model	# Params	Drama (C=5)	Taigi (C=5)	Taigi-adapt (C=8)	Drama (C=2)	Taigi (C=2)	Taigi-adapt (C=8)	Taigi (C=8)
MatchboxNet-S	69k	60.11 $\pm$ 2.51	45.83 $\pm$ 7.97	28.02 $\pm$ 5.68	71.99 $\pm$ 2.30	51.88 $\pm$ 6.67	27.29 $\pm$ 5.73	23.65 $\pm$ 5.99
MatchboxNet-M	85k	64.99 $\pm$ 2.44	48.02 $\pm$ 7.55	33.33 $\pm$ 6.20	74.12 $\pm$ 2.37	52.71 $\pm$ 7.24	25.94 $\pm$ 5.62	19.17 $\pm$ 4.64
MatchboxNet-L	119k	63.86 $\pm$ 2.51	43.44 $\pm$ 8.54	29.38 $\pm$ 6.82	72.70 $\pm$ 2.33	53.75 $\pm$ 7.66	24.79 $\pm$ 5.68	23.65 $\pm$ 5.47
HuBERT-base	94M	89.60 $\pm$ 1.56	67.92 $\pm$ 7.03	90.10 $\pm$ 3.85	79.49 $\pm$ 2.05	54.58 $\pm$ 8.44	86.77 $\pm$ 4.43	84.69 $\pm$ 5.21
HuBERT-large	315M	90.59 $\pm$ 1.49	69.48 $\pm$ 6.82	84.06 $\pm$ 4.79	78.93 $\pm$ 2.05	53.75 $\pm$ 7.97	77.60 $\pm$ 5.05	78.65 $\pm$ 5.73

Table 8: Accuracy (%) comparison on the Drama and TaigiSpeech datasets (Taigi) under two data mining strategies. C=5: a 5-class setting (4 emergency intents plus a non-emergency category). C=2: binary classification ( Emergency vs. non-emergency ). C=8: the original 8-intent setting in TaigiSpeech. Taigi-adapt means models trained on the mined data are further fine-tuned on the training set of TaigiSpeech. Scratch : Models trained from scratch without being pre-trained on mined data. For the Drama test set, the ground-truth labels are obtained via LLM pseudo-labeling, thus evaluating the consistency with the LLM-generated labels.

Figure 6: Confusion matrix of SSL speech models under the Taigi-adapt setting in the Keyword Match Mining scenario.

Model	# Params	Accuracy (%)
Cascade System (ASR + Qwen3-8B)
Whisper base	74M + 8B	15.00
Whisper small	244M + 8B	15.83
Whisper medium	769M + 8B	42.92
Whisper large-v3	1.55B + 8B	34.38
Qwen3-ASR 0.6B	0.6B + 8B	63.65

Table 9: Comparison between Cascade ASR+LLM and End-to-End Speech Classification Systems on Taiwanese.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 7 — Figure 7: Illustration of the recording environment.

Intent & Component	English Prompt	Chinese Translation (中文翻譯)
1. SOS_CALL (求救)
Video Description (8 seconds)	An elderly Taiwanese man living alone in social housing in Taiwan sits on the living room sofa. Suddenly, his face turns pale, he clutches his chest tightly, and curls up his body. He opens his mouth to gasp rapidly, his eyes full of extreme panic. He struggles to reach out, trying to grab the phone on the table, but lacks strength throughout his body. Finally, his hand drops weakly, and he looks desperately towards the door.	一位在台灣社會住宅的獨居台灣阿公坐在客廳沙發上。突然他臉色慘白，手緊緊揪住胸口，身體蜷縮起來。他張開嘴急促地喘息，眼神中充滿了極度的慌張。他掙扎著伸出手，試圖去拿桌上的手機，但全身無力。最終虛弱地垂下手，絕望地望向門口的方向。
Visual Style	Realistic, with a strong sense of tension and urgency. The camera can shake slightly to simulate the viewer’s nervous heartbeat, with cold color tones.	寫實、帶有強烈緊張與急迫感，鏡頭可以微微顫抖，模擬觀看者的緊張心跳，色調偏冷。
Audio	The background only has a faint television sound, quickly covered by rapid, heavy heartbeats and high-frequency tense string music. The sound fluctuates, creating a suffocating sense of life hanging by a thread.	背景只有微弱的電視聲，很快被一陣急促、沉重的心跳聲和高頻的緊張弦樂覆蓋。聲音時強時弱，營造出命懸一線的窒息感。
2. FALL_HELP (跌倒／起不來)
Video Description (8 seconds)	An elderly Taiwanese person living alone in social housing in Taiwan, wearing loungewear, tries to sit down but accidentally slips from the chair and falls heavily onto the floor. She shows a pained expression, struggles to prop herself up with both hands, and tries twice to stand up. However, her legs are visibly weak due to lack of exercise, and she eventually sits paralyzed and helpless on the spot, her eyes full of frustration and a longing for help.	一位住在台灣社會住宅穿著居家服的獨居台灣長者，想要坐下，不小心從椅子上滑下並重摔在地上。她臉上露出痛苦的表情，掙扎著用雙手撐地，嘗試了兩次想要站起來。但雙腿因缺乏運動明顯無力，最終只能無助地癱坐在原地，眼神充滿了挫敗與求助的渴望。

Appendix

Table 10: Scenario Video Generation Prompts (Part 1/3).

Intent & Component	English Prompt	Chinese Translation (中文翻譯)
4. PAIN_GENERAL (一般疼痛)
Video Description (8 seconds)	An elderly Taiwanese man living alone in social housing in Taiwan is sitting in the living room watching television. He suddenly stops his actions, closes his eyes tightly, and presses one hand forcefully against his temple. His facial muscles are tense, showing that he is enduring a severe headache. He slowly puts down his book, supports his forehead with his other hand as well, and leans slightly forward.	一位居在台灣社會住宅的獨居阿公正坐在客廳看電視機。他突然停下動作，緊閉雙眼，一隻手用力地按壓著自己的太陽穴。他的臉部肌肉緊繃，顯現出正在忍受劇烈頭痛的樣子。他緩慢地放下書，另一隻手也扶住額頭，身體微微前傾。
Visual Style	Realistic, everyday life. Soft lighting, capturing the tense facial details of the protagonist caused by pain through a close-up shot.	寫實、生活化。光線柔和，透過特寫鏡頭捕捉主角因疼痛而緊繃的臉部細節。
Audio	The background consists of quiet room environmental sounds (e.g., a ticking clock). When the protagonist feels pain, a very faint but piercing high-frequency tinnitus sound effect is added to simulate the feeling of a headache, which then slowly fades.	背景是安靜的房間環境音（如時鐘滴答聲）。當主角感到疼痛時，加入一陣非常細微但刺耳的高頻耳鳴音效，模擬頭痛時的感受，然後慢慢減弱。
5. CALL_CONTACT (打電話)
Video Description (8 seconds)	An elderly Taiwanese woman living alone in social housing in Taiwan, wearing reading glasses, smiles as she flips through an old photo album. She stops at a family portrait, gently stroking the child’s face in the photo with her finger, her eyes filled with longing. Then, she looks up, turns her gaze to the smart speaker in the room, moves her lips slightly, and wears an expectant expression on her face.	一位在台灣社會住宅獨居台灣阿嬤戴著老花眼鏡，微笑著翻閱一本舊相簿。她停在一張全家福照片上，用手指溫柔地撫摸著照片中孩子的臉龐，眼神充滿思念。接著，她抬起頭，將目光轉向房間裡的智慧音箱，嘴唇微動，臉上帶著期待的表情。

Appendix

Table 11: Scenario Video Generation Prompts (Part 2/3).

Intent & Component	English Prompt	Chinese Translation (中文翻譯)
7. LIGHT_OFF (關燈)
Video Description (8 seconds)	At night, an elderly Taiwanese person living alone in social housing in Taiwan is already lying comfortably in bed, covered with a blanket. He lets out a big yawn, rubs his eyes, and then, with tired eyes, looks up at the ceiling light that is still on, showing obvious sleepiness in his eyes.	夜晚，一位在台灣社會住宅的台灣獨居長者已經舒適地躺在床上，蓋好了棉被。他打了個大大的哈欠，揉了揉眼睛，然後睜著疲憊的雙眼，看著天花板上仍然亮著的燈，眼中流露出明顯的睡意。
Visual Style	Realistic and tranquil. The room has warm lighting, focusing on capturing the protagonist’s natural, sleepy actions like yawning and rubbing eyes.	寫實、寧靜。房間內是溫暖的燈光，重點捕捉主角打哈欠、揉眼睛等充滿睡意的自然動作。
Audio	Only faint environmental sounds or distant sounds of insects chirping. The sounds of yawning and blanket friction can be heard clearly, creating a quiet pre-sleep atmosphere.	只有微弱的環境音，或遠處傳來的蟲鳴聲。可以清晰地聽到哈欠聲和棉被摩擦的聲音，營造安靜的睡前氛圍。
8. CANCEL_ALERT (解除警報)
Video Description (8 seconds)	In the kitchen, the fire detector on the wall is flashing red and emitting a continuous ”beep-beep” alarm sound. An elderly Taiwanese woman living alone in social housing in Taiwan calmly takes a towel and fans the detector, wearing a slightly helpless ”not again” smile. She casually stands on her tiptoes, presses the button on the detector, and the alarm stops immediately.	廚房裡，牆上的防火災偵測器正閃爍著紅光，並發出持續的「嗶—嗶—」警報聲。一位在台灣社會住宅的台灣獨居阿嬤從容地拿著一條毛巾，對著偵測器搧風，臉上帶著一點「又來了」的無奈笑容。她輕鬆地踮起腳尖，按下了偵測器上的按鈕，警報隨即停止。

Appendix

Table 12: Scenario Video Generation Prompts (Part 3/3).

Low-Resource Language Processing

Low-resource language processing builds and evaluates language technologies for languages with limited data, tools, or digital representation.

See all 565 papers

ELARSpeech: A 30K-Hour Dataset Across 675 Underrepresented Languages for Long-Tail Multilingual Speech Technologies
Oct 14, 2026Sangmin Lee, Yejin Jeon, David Ifeoluwa AdelaniLow-Resource Language Processing Multilingual ASR
Yonsei University · McGill University · Mila - Quebec AI Institute +1
Breeze Taigi: Benchmarks and Models for Taiwanese Hokkien Speech Recognition and Synthesis
Feb 26, 2026Yu-Siang Lan, Chia-Sheng Liu, Yi-Chang Chen +5Speech Processing ASR Evaluation
National Yang Ming Chiao Tung University · Work done at MediaTek Research · MediaTek Research
TG-ASR: Translation-Guided Learning with Parallel Gated Cross Attention for Low-Resource Automatic Speech Recognition
Feb 25, 2026Cheng-Yeh Yang, Chien-Chun Wang, Li-Wei Chen +3Automatic Speech Recognition Speech Processing
Dept. Computer Science and Information Engineering, National Taiwan Normal University, Taiwan · United Link Co., Ltd., Taiwan · Institute of Computer Science, Academia Sinica, Taiwan

Oct 14, 2026J/K move · Enter open · S save

ELARSpeech: A 30K-Hour Dataset Across 675 Underrepresented Languages for Long-Tail Multilingual Speech Technologies

Sangmin Lee, Yejin Jeon, David Ifeoluwa Adelani

Yonsei University · McGill University · Mila - Quebec AI Institute+1

Low-Resource Language Processing Multilingual ASR Low-Resource Speech Recognition Low-Resource TTS Synthesis

Open paper AskPDF arXiv

Massively multilingual speech modeling has made remarkable progress in recent years, yet existing training resources and benchmarks still underrepresent much of the linguistic long tail. To address this gap, we introduce ELARSpeech, a large-scale multilingual speech resource comprising 30,202 hours of speech across 675 underrepresented languages, curated from the Endangered Languages Archive. Combining human-annotated and unlabeled recordings, the corpus spans diverse recording conditions, domains, and language families. Beyond corpus construction, our experiments show how ELARSpeech supports extending ASR and TTS capabilities to diverse underrepresented languages, with further benefits from pseudo-labeling unlabeled data. At the same time, our speech representation analysis reveals remaining limitations in transferring existing multilingual models to the linguistic long tail. Together, ELARSpeech provides both a resource and a testbed for extending multilingual speech technology further into the linguistic long tail. We publicly release the dataset, all trained ASR and TTS models, and the complete data construction pipeline to facilitate research on long-tail multilingual speech technologies.

Abstract

Low-Resource Language Processing

Explore similar work

ELARSpeech: A 30K-Hour Dataset Across 675 Underrepresented Languages for Long-Tail Multilingual Speech Technologies

Breeze Taigi: Benchmarks and Models for Taiwanese Hokkien Speech Recognition and Synthesis

TG-ASR: Translation-Guided Learning with Parallel Gated Cross Attention for Low-Resource Automatic Speech Recognition