SPARC: Spoken Profile Authorship Recognition Corpus

Efthymios Tsaprazlis; Bowen Yi; Tiantian Feng; Junghwan Kim; David Jurgens; Shrikanth Narayanan

cs.CLOct 18, 2026

SPARC: Spoken Profile Authorship Recognition Corpus

Authors: Efthymios Tsaprazlis, Bowen Yi, Tiantian Feng, Junghwan Kim, David Jurgens, Shrikanth Narayanan

Organizations: University of Southern California · University of Michigan

Abstract

Speech anonymization has focused on the acoustic signal, leaving open how much speaker information the transcribed words themselves retain. In this work, we study pairwise speaker and demographic-attribute verification from transcribed speech. We introduce SPARC (Spoken Profile Authorship Recognition Corpus), a benchmark built from two podcast corpora comprising over 12,500 speakers, 1.5M quality-filtered utterances, and speaker-level inferred or source-provided labels for sex, age, and accent, and evaluate five attribution models and three LLMs on identity- and attribute-verification trials with episode- and metadata-based topic proxies. LLM adversaries distinguish same-speaker pairs from text alone. Attribute-label agreement is harder to verify, but LLMs perform sex- and accent-label verification above chance. Finally, releasing only short probe texts (50 words) degrades the strongest adversaries the most, offering a partial reduction in observed linkage.

Figures & tables

Figure 1: Overview of the SPARC construction pipeline. Eligible single-speaker speech from NaturalVoices and SPoRC is filtered for SNR and background music, retained when two ASR models agree by at least 90%, and annotated with age, sex, and accent labels from source metadata or model predictions.

	NaturalVoices	SPoRC
Number of Speakers	2,344	10,165
Number of Episodes	3,500	15,555
Speaker–episode pairs	6,766	15,555
Number of Utterances	779,494	795,591
Avg. utt. / speaker	332.6	78.3
Avg. utt. / speaker–episode	115.2	51.2

Table 1: Corpus statistics after filtering and attribute inference. Sex and age are taken from dataset metadata for NaturalVoices and inferred via Vox-Profile for SPoRC subset. Accent is binarized into North American vs. Other due to the sparsity of remaining accent groups.

Figure 2: ROC curves for identity verification with 200-word probes and $L{=}20$ enrollment texts, for all eight models. NaturalVoices-C/S/D denote the combined, same-topic proxy (within-episode), and different-topic proxy (cross-episode) conditions and SPoRC-D is the different-topic condition of SPoRC.

		Combined		Same Topic		Diff. Topic
Attribute	Model	EER $\downarrow$	AUC $\uparrow$	EER $\downarrow$	AUC $\uparrow$	EER $\downarrow$	AUC $\uparrow$
Biological Sex	SentenceBERT	0.468	0.543	0.508	0.490	0.484	0.515
	GPT-5-mini	0.379	0.664	0.437	0.608	0.377	0.669
	Llama-3.3-70B	0.403	0.632	0.422	0.611	0.420	0.612
Age Group	SentenceBERT	0.447	0.576	0.461	0.545	0.469	0.548
	GPT-5-mini	0.442	0.572	0.496	0.510	0.459	0.557

Table 2: Attribute verification results on NaturalVoices with 200-word probes and

L{=}20

enrollment texts. Same- and different-topic labels denote within- and cross-episode proxies, respectively. Best result per condition within each attribute in bold.

Model	EER $(\downarrow)$	AUC $(\uparrow)$	EER $(\downarrow)$	AUC $(\uparrow)$	EER $(\downarrow)$	AUC $(\uparrow)$	EER $(\downarrow)$	AUC $(\uparrow)$
	Main - 200w / $L$ =20		$\Delta$ 200w / $L$ =5		$\Delta$ 50w / $L$ =20		$\Delta$ 50w / $L$ =5
SentenceBERT	0.351	0.708	$+0.024$	$-0.025$	$+0.038$	$-0.047$	$+0.046$	$-0.059$
GPT-5-mini	0.285	0.790	$+0.041$	$-0.049$	$+0.065$	$-0.077$	$+0.076$	$-0.089$
Llama-3.3-70B	0.281	0.789	$+0.044$	$-0.045$	$+0.047$	$-0.053$	$+0.064$	$-0.075$

Table 3: Identity verification ablations on NaturalVoices (different-topic). Main results use probe

\approx

200 words and

L{=}20

;

\Delta

columns show the change when reducing probe length and/or enrollment size

L

. Best EER and AUC in the main setting in bold .

Model	EER $\downarrow$	AUC $\uparrow$	TPR@1%	[email protected]%
SentenceBERT	0.351	0.708	0.143	0.056
GPT-5-mini	0.285	0.790	0.277	0.121
Llama-3.3-70B	0.281	0.789	0.133	0.011

Table 4: Low-FPR identity verification on NaturalVoices (different-topic, 200-word probes,

L{=}20

). TPR at FPR

{=}1\%

and

0.1\%

; bootstrap 95% CIs are tight for TPR@1% (e.g. GPT-5-mini

[0.265,0.293]

) and wider at

0.1\%

. Best per column in bold.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Probe / enrollment	Task	EP	HP	EN	HN	Total
$P=50,\ L=5$	Speaker	3,748	3,046	4,015	2,789	13,598
	Sex	1,748	1,965	2,050	1,041	6,804
	Age	2,222	2,956	1,059	567	6,804
	Accent	2,445	3,055	960	344	6,804
$P=50,\ L=20$	Speaker	3,748	3,046	4,015	2,789	13,598
	Sex	1,748	1,965	2,050	1,041	6,804

Appendix

Table 5: NaturalVoices trial counts by task, input-length setting, and trial condition. Easy positives (EP) pair the same speaker or attribute within a podcast, whereas hard positives (HP) pair the same speaker or attribute across podcasts. Easy negatives (EN) pair different speakers or attributes across podcasts, whereas hard negatives (HN) pair them within a podcast. All demographic trials are cross-speaker. Here,

P

is the target probe length in words and

L

is the maximum number of enrollment utterances. The four reported settings contain 124,356 task-specific trials.

Probe / enrollment	Task	EP	HP	EN	HN	Total
$P=50,\ L=5$	Speaker	–	2,336	2,336	–	4,672
	Sex	–	2,000	2,000	–	4,000
	Age	–	2,000	2,000	–	4,000
	Accent	–	2,000	2,000	–	4,000
$P=50,\ L=20$	Speaker	–	2,327	2,327	–	4,654
	Sex	–	2,000	2,000	–	4,000

Appendix

Table 6: SPoRC trial counts by task, input-length setting, and trial condition. The metadata-filtered SPoRC protocol retains hard-positive (HP) and easy-negative (EN) trials. Easy-positive (EP) and hard-negative (HN) conditions were not constructed since the SPoRC podcast has only a single speaker and are marked with dashes. Positive and negative counts are balanced within every task and setting. Here,

P

is the target probe length in words and

L

is the maximum number of enrollment utterances. The protocol contains 65,174 task-specific trials.

Panel of Figure 3 — Figure 3: ROC curves for speaker identity verification on SPoRC (different-topic condition) for all eight models, across the four probe-length ( $P$ , in words) and enrollment-size ( $L$ , in utterances) settings.

Panel of Figure 4 — Figure 4: ROC curves for sex-label verification on SPoRC (different-topic condition) for all eight models, across the four probe-length ( $P$ ) and enrollment-size ( $L$ ) settings.

Panel of Figure 5 — Figure 5: ROC curves for age-group verification on SPoRC (different-topic condition) for all eight models, across the four probe-length ( $P$ ) and enrollment-size ( $L$ ) settings.

Panel of Figure 6 — Figure 6: ROC curves for accent verification on SPoRC (different-topic condition) for all eight models, across the four probe-length ( $P$ ) and enrollment-size ( $L$ ) settings.

Probe/enrollment	GPT-5-mini	Llama-3.3-70B
$P=50,\ L=5$	0.410	0.390
$P=50,\ L=20$	0.406	0.376
$P=200,\ L=5$	0.399	0.390
$P=200,\ L=20$	0.402	0.391

Appendix

Table 7: EER for SPoRC speaker verification when positive and negative pairs both cross podcast series (lower is better).

Metric	Age group	Sex	Accent
Human agr.	60–76%	94–98%	80–82%
Model–majority agr.	64%	98%	88%
Human $\kappa$	0.489	0.920	0.613
Model–majority $\kappa$	0.531	0.960	0.760

Appendix

Table 8: Perceptual audit of model-inferred attribute labels on 50 category-balanced recordings per attribute. Human agreement is shown as the reported range.

Model	Condition	EER $(\downarrow)$	AUC $(\uparrow)$	EER $(\downarrow)$	AUC $(\uparrow)$	EER $(\downarrow)$	AUC $(\uparrow)$	EER $(\downarrow)$	AUC $(\uparrow)$
		Main - 200w / $L$ =20		$\Delta$ 200w / $L$ =5		$\Delta$ 50w / $L$ =20		$\Delta$ 50w / $L$ =5
SentenceBERT	Combined	0.350	0.683	$+0.048$	$-0.049$	$+0.043$	$-0.046$	$+0.062$	$-0.062$
	Easy	0.168	0.913	$+0.071$	$-0.072$	$+0.070$	$-0.071$	$+0.110$	$-0.118$
	Hard	0.618	0.334	$-0.005$	$+0.017$	$-0.009$	$+0.020$	$-0.032$	$+0.050$
	Same Topic	0.460	0.567	$+0.040$	$-0.064$	$+0.004$	$-0.025$	$+0.020$	$-0.034$
	Diff Topic	0.351	0.708	$+0.024$	$-0.025$	$+0.038$	$-0.047$	$+0.046$	$-0.059$

Appendix

Table 9: Identity verification results on NaturalVoices for all models across trial conditions and ablation settings. Main results use probe

\approx

200 words and

L=20

;

\Delta

columns show the change from the main setting. Best EER and AUC per condition (main setting) in bold .

Model	Condition	EER $(\downarrow)$	AUC $(\uparrow)$	EER $(\downarrow)$	AUC $(\uparrow)$	EER $(\downarrow)$	AUC $(\uparrow)$	EER $(\downarrow)$	AUC $(\uparrow)$
		Main - 200w / $L$ =20		$\Delta$ 200w / $L$ =5		$\Delta$ 50w / $L$ =20		$\Delta$ 50w / $L$ =5
SentenceBERT	Combined	0.468	0.543	$-0.001$	$-0.001$	$+0.003$	$-0.006$	$-0.003$	$+0.007$
	Easy	0.222	0.855	$+0.038$	$-0.033$	$+0.051$	$-0.059$	$+0.056$	$-0.066$
	Hard	0.776	0.146	$-0.025$	$+0.025$	$-0.052$	$+0.043$	$-0.081$	$+0.089$
	Same Topic	0.508	0.490	$-0.010$	$+0.010$	$-0.005$	$\pm 0.000$	$-0.031$	$+0.041$
	Diff Topic	0.484	0.515	$-0.001$	$-0.008$	$+0.018$	$-0.011$	$+0.015$	$-0.011$

Appendix

Table 10: Biological sex verification results on NaturalVoices for all models across trial conditions and ablation settings. Main results use probe

\approx

200 words and

L=20

;

\Delta

columns show the change from the main setting. Best EER and AUC per condition (main setting) in bold .

Model	Condition	EER $(\downarrow)$	AUC $(\uparrow)$	EER $(\downarrow)$	AUC $(\uparrow)$	EER $(\downarrow)$	AUC $(\uparrow)$	EER $(\downarrow)$	AUC $(\uparrow)$
		Main - 200w / $L$ =20		$\Delta$ 200w / $L$ =5		$\Delta$ 50w / $L$ =20		$\Delta$ 50w / $L$ =5
SentenceBERT	Combined	0.447	0.576	$+0.037$	$-0.052$	$+0.023$	$-0.039$	$+0.026$	$-0.043$
	Easy	0.198	0.873	$+0.051$	$-0.045$	$+0.063$	$-0.060$	$+0.091$	$-0.092$
	Hard	0.746	0.173	$+0.002$	$\pm 0.000$	$-0.033$	$+0.037$	$-0.053$	$+0.065$
	Same Topic	0.461	0.545	$+0.030$	$-0.049$	$+0.033$	$-0.038$	$+0.052$	$-0.051$
	Diff Topic	0.469	0.548	$+0.018$	$-0.037$	$+0.022$	$-0.033$	$+0.014$	$-0.028$

Appendix

Table 11: Age-group verification results on NaturalVoices for all models across trial conditions and ablation settings. Main results use probe

\approx

200 words and

L=20

;

\Delta

columns show the change from the main setting. Best EER and AUC per condition (main setting) in bold .

Model	Condition	EER $(\downarrow)$	AUC $(\uparrow)$	EER $(\downarrow)$	AUC $(\uparrow)$	EER $(\downarrow)$	AUC $(\uparrow)$	EER $(\downarrow)$	AUC $(\uparrow)$
		Main - 200w / $L$ =20		$\Delta$ 200w / $L$ =5		$\Delta$ 50w / $L$ =20		$\Delta$ 50w / $L$ =5
SentenceBERT	Combined	0.441	0.578	$-0.002$	$+0.010$	$+0.004$	$-0.001$	$+0.010$	$-0.014$
	Easy	0.212	0.869	$+0.033$	$-0.027$	$+0.045$	$-0.053$	$+0.074$	$-0.086$
	Hard	0.752	0.171	$-0.039$	$+0.045$	$-0.063$	$+0.060$	$-0.076$	$+0.083$
	Same Topic	0.483	0.534	$-0.023$	$+0.018$	$-0.010$	$+0.003$	$-0.004$	$-0.012$
	Diff Topic	0.494	0.510	$-0.021$	$+0.021$	$-0.009$	$+0.011$	$-0.005$	$+0.003$

Appendix

Table 12: Accent verification results on NaturalVoices for all models across trial conditions and ablation settings. Main results use probe

\approx

200 words and

L=20

;

\Delta

columns show the change from the main setting. Best EER and AUC per condition (main setting) in bold .

Speaker Verification

Speaker verification determines whether two speech recordings come from the same person, enabling voice authentication and identity analysis.

See all 61 papers

VoxAnon: A balanced dataset for a fair evaluation of speech anonymization
Oct 9, 2026Orane Dufour, Paul Magron, Mickael RouvierPrivacy-Preserving Speech Processing Speaker Anonymization
Université de Lorraine, CNRS Inria, LORIA, F-54000 Nancy, France · LIA, Avignon University F-84911 Avignon, France
SpeakerLLM: A Speaker-Specialized Audio-LLM for Speaker Understanding and Verification Reasoning
May 14, 2026KiHyun Nam, Jungwoo Heo, Siu Bae +2Audio Representation Learning Speaker Verification
Korea Advanced Institute of Science and Technology (KAIST) · University of Seoul
Speaker Verification with Speech-Aware LLMs: Evaluation and Augmentation
Mar 11, 2026Thomas Thebaud, Yuzhe Wang, Laureano Moro-Velazquez +2Speaker Embedding Speaker Verification
Electrical and Computer Engineering Department, Johns Hopkins University, Baltimore, MD, USA · Human Language Technology Center of Excellence, Johns Hopkins University, Baltimore, MD, USA

Oct 9, 2026J/K move · Enter open · S save

VoxAnon: A balanced dataset for a fair evaluation of speech anonymization

Orane Dufour, Paul Magron, Mickael Rouvier

Université de Lorraine, CNRS Inria, LORIA, F-54000 Nancy, France · LIA, Avignon University F-84911 Avignon, France

Privacy-Preserving Speech Processing Speaker Anonymization

Open paper AskPDF arXiv

Speech anonymization is evaluated with automatic speaker verification attackers whose performance depends on the attribute distribution of both their training and evaluation data, which may bias privacy estimates for some speaker groups. To alleviate this issue, we introduce VoxAnon, a dataset built from VoxCeleb1 and 2 with training and evaluation splits balanced across gender, age, accent, and pitch. We extract and cross-check attributes using two complementary methods, balance trial lists per subgroup, and draw test/enrollment utterances from distinct video sources in order to limit linguistic and session bias. Comparing VoxAnon with unbalanced training and test data across four attackers and three anonymizers, we show that attribute imbalance changes which subgroups appear hardest to re-identify, in directions that depend on the attribute. In the balanced setting, residual disparities persist, mostly tied to accent and pitch, and depend on the anonymizer/attacker pair. Code and trial lists are publicly available.

Abstract

Speaker Verification

Explore similar work

VoxAnon: A balanced dataset for a fair evaluation of speech anonymization

SpeakerLLM: A Speaker-Specialized Audio-LLM for Speaker Understanding and Verification Reasoning

Speaker Verification with Speech-Aware LLMs: Evaluation and Augmentation