We present the Spoken Wikipedia Presentation Corpus, an extension of the Spoken Wikipedia Corpora featuring LLM-generated slide decks for multimodal ASR. Slides are created from LLM-segmented sections using a hybrid pipeline that combines LLM-based content planning with rule-based design decisions. For each section, an LLM generates a slide title, bullet points, a takeaway message, and a visual description that is used to create an illustration. Rule-based matching then selects layouts, themes, and styles to produce the final slides. A vision LLM extracts slide text as Markdown. We evaluate multiple ASR and spoken language models (SLMs). The best model achieves an average micro-WER of 10.23% and an average micro-CER of 6.48% on audio-only inputs. English yields the lowest error rates, followed by German and Dutch, while performance declines across lower-resource languages. Although audio-only baselines are strong, multimodal zero-shot prompting of omni models remains challenging. The aligned slide, text, and audio data show a strong potential to improve recognition through cross-modal context.
Figures & tables
Fig. 1: Example of a rendered SWPC slide with title, bullet points, takeaway message, and generated illustrative image on the right.
Fig. 2: High-level overview of the SWPC data creation workflow. Spoken Wikipedia audio and article text are processed into aligned utterance–section mappings, which are used to generate slide images, extract slide text, apply quality control, and assemble the final multimodal dataset.
Category
Theme
Layout
Literary analysis
Literature
Editorial cards
Historical context
History
Timeline visual
Person biography
Biography
Portrait focus
Place geography
Geography
Map focus
Science technical
Science
Concept system
Arts culture
Arts
Gallery focus
TABLE I: Rule-based mapping from category to slide theme and layout.
Split
Lang.
Utterances
Articles
Text Seg.
Slides
Speakers
Duration
Train
EN
69,672
4,950
13,636
14,138
363
149:09:36
Train
DE
86,354
6,067
14,823
15,684
281
213:38:15
Train
NL
36,842
3,890
7,094
7,322
57
76:12:42
Train
Total
192,868
14,907
35,553
37,144
701
439:00:33
Dev
EN
7,531
597
1,460
1,526
46
19:01:12
Dev
DE
9,959
712
1,730
1,823
35
27:15:15
TABLE II: SWPC dataset statistics for train, development, test, and total splits. We report utterance, article, text-segment, unique slide, speaker counts, and aligned audio duration.
Model
Lang.
WER μ
WER M
CER μ
CER M
whisper-large-v3
EN
8.14
9.78
5.90
7.04
DE
10.60
12.12
6.25
7.58
NL
17.21
23.36
10.27
13.57
Avg.
10.77
13.81
6.81
8.73
canary-1b-v2
EN
9.85
12.82
6.83
8.86
DE
11.09
13.61
6.38
8.26
TABLE III: SWPC results for baseline ASR and multimodal speech models, sorted by parameter count. Micro/macro WER and CER (%) are reported for English, German, Dutch, and averaged.