Speech Generation Evaluation

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, up 133% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 35

All topics

NSV-Shift: A Contrastive Benchmark for Non-Speech Vocalization Understanding and Response Adaptation in Speech-to-Speech Models
Sep 27, 2026Ziwei ChenSpeech Generation Evaluation Audio Understanding
University of California San Diego
XSQ-AST: An Explainable Audio Spectrogram Transformer Framework for Localising Synthetic Speech Artifacts
Sep 21, 2026Ben Heritage, Luca Resti, Mónica Villanueva Aylagas +3Explainable Artificial Intelligence Speech Generation Evaluation
AudioLab, School of Physics, Engineering and Technology, University of York, United Kingdom · Department of Computer Science, University of York, United Kingdom · SEED – Electronic Arts (EA), Sweden +1
HaikuS2S: A Cascaded System For Responding In Verse
Sep 20, 2026Devangi Sharma, Sophia Judicke, Glenda Tan +2TTS Synthesis Speech Generation Evaluation
Carnegie Mellon University · Unaffiliated
Multi-Dimensional Prosody Judgment For Live Streaming Speech Synthesis
Sep 17, 2026Zifan Guan, Longyu Lu, Junan Zhang +3Speech Generation Evaluation TTS Evaluation
The Chinese University of Hong Kong, Shenzhen, China · TaoLive-AIGC Team Taobao & Tmall Group of Alibaba
RoleBreak: Benchmarking Long-Horizon Role-Playing Robustness in Spoken Dialogue
Sep 15, 2026Yuqi Wang, Fengyuan Liu, Haochen Luo +2Large Language Model-Based Role-Play Simulation Speech Generation Evaluation
The University of Hong Kong · Kami AI
Quantifying the Generation Modality Gap in Speech-Text Language Models
Sep 13, 2026Ju-Chieh Chou, Jiawei Zhou, Karen LivescuSpeech Generation Evaluation Speech Language Models
TTI-Chicago · Stony Brook University
Voice or Stereotype? Disentangling Acoustic and Content-Based Gender in Speech-to-Speech Models
Sep 10, 2026Xiaoqun Liu, Tanu Mitra, Harshit Rajgarhia +1Speech Generation Evaluation Speech Processing
Centific Research · University of Washington
VoxReason: Auditing Source-Grounded Speech Plans Before Synthesis
Sep 2, 2026Mengzhe GengSpeech Generation Evaluation Controllable Speech Generation
National Research Council Canada
When Does Predictor-Based RL Align with Human Perception? A Study of Subjective Rewards in Codec-Based Speech Language Models
Aug 31, 2026Joonyong Park, Jerry LiReward Modeling RL for Language Models
Spellbrush
Beyond Naturalness: Probing Automated Text-To-Speech Evaluators on Linguistically Grounded Dimensions
Aug 10, 2026Oluwanifemi Bamgbose, Simon Rosen, Jash Shah +6Audio-Language Model Evaluation Human Preference Evaluation
ServiceNow
MMAG: A Multi-Control Mixed Audio Generation Benchmark
Aug 7, 2026Zihao Zheng, Xuenan Xu, Jiahao Mei +5Speech Generation Evaluation Audio Generation
MoE Key Lab of Artificial Intelligence, X-LANCE Lab, Shanghai Jiao Tong University, China · Shanghai AI Lab, China
Domain-Specific Evaluation of Text-to-Speech Systems: A Multi-Metric Benchmarking Study
Aug 3, 2026Ali Jafar, Amal Sarmad, Shifa Yousaf +1Speech Generation Evaluation TTS Evaluation
FAST School of Computing, National University of Computer and Emerging Sciences (FAST-NUCES), Lahore, Punjab, Pakistan
Beyond Prompt Adherence: Auditing Attribute-Level Voice Control in Speech Generation
Aug 1, 2026Xianhao Zhou, Jianghao WuSpeech Generation Evaluation TTS Evaluation
School of Mechanical and Electrical Engineering, UESTC, Chengdu, China · Faculty of Information Technology, Monash University, Melbourne, Australia
A Production-Oriented Framework for Evaluation of SFX Generation
Jul 10, 2026Mélodie Desbos, Yara Bahram, Eric Granger +1Speech Generation Evaluation Audio Editing
LIVIA, Dept. of Systems Engineering, ETS Montréal, Canada
Best-of- $N$ TTS Evaluation is Confounded by ASR Family Alignment
Jul 9, 2026Taehyung Yu, Seongjae KangSpeech Generation Evaluation TTS Evaluation
KAIST, Daejeon, South Korea.
SPEARBench: A Benchmark for Naturalness Evaluation in Streaming Speech-to-Speech Language Models
Jul 6, 2026Thomas Thebaud, Yuzhe Wang, Hao Zhang +5Audio-Language Model Evaluation Language Model Generation Evaluation
∗Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA · †Amazon Inc., Seattle, WA, USA
Towards Digital Preservation of Efik: TTS for a Low-Resource African Language
Jul 5, 2026Offiong Bassey Edet, Emmanuel Oyo-Ita, Archibong Okon Archibong +2TTS Synthesis Speech Generation Evaluation
University of Cross River State, Nigeria · ML Collective · University of Calabar, Nigeria
Towards a Phonology-Informed Evaluation of Multilingual TTS
Jul 2, 2026Sneha Ray Barman, Neeraj Kumar Sharma, Shakuntala MahantaTTS Synthesis Speech Generation Evaluation
Centre for Linguistic Science & Technology, IIT Guwahati · Mehta Family School for Data Science & Artificial Intelligence, IIT Guwahati · Department of Humanities & Social Sciences, IIT Guwahati
Is Natural Always Appropriate? Investigating Naturalness and Appropriateness Across Different Domains for TTS Evaluation
Jun 30, 2026Dominika Woszczyk, Andreas Triantafyllopoulos, Jura Miniota +2TTS Synthesis Speech Generation Evaluation
Iconic, United Kingdom · Technische Universität München, Germany · KTH Royal Institute of Technology, Sweden +1
Reference-Based Prosody and Rhythm Evaluation for Spoken Dialogue Systems
Jun 30, 2026Ashish Hallur, Thomas Thebaud, Georgi Tinchev +2Voice Agent Evaluation Speech Generation Evaluation
Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA · Amazon Inc., Seattle, WA, USA
CN-NewsTTS Bench: a target-level automatic benchmark for raw-input Chinese news TTS pronunciation
Jun 23, 2026Shijun LuoSpeech Generation Evaluation TTS Evaluation
NetEase Cloud Music, Hangzhou, China
ParaPairAudioBench: Paralinguistic Pairwise Audio Benchmark for LALM-as-a-Judge
Jun 23, 2026Jisu Jeon, Seungyeon Jwa, Joosung Lee +6Audio-Language Model Evaluation Pairwise Comparison
Hongik University, South Korea · Seoul National University, South Korea · NAVER Cloud, South Korea +1
An Evaluation Framework for Text-to-Speech Voice Reconstruction
Jun 19, 2026Ariadna Sanchez, Christoph Minixhofer, Korin Richmond +3Speech Generation Evaluation TTS Evaluation
The Centre for Speech Technology Research, University of Edinburgh, UK
Investigating Human-Model Discrepancies in Speech Quality Assessment via Acoustic and Prosodic Perturbations
Jun 18, 2026Masato Takagi, Masaya Kawamura, Reo Shimizu +1Human Preference Evaluation Speech Generation Evaluation
Nagoya Institute of Technology, Japan · LY Corporation, Japan
AnyAudio-Judge: A Dynamic Rubric-Based Benchmark and Evaluator for Audio Instruction Following
Jun 2, 2026Haitao Li, Tian Tan, Yuguang Yang +2Audio-Language Model Evaluation LLM-as-a-Judge
Zhejiang University · Shanghai Innovation Institute · Shanghai Jiao Tong University +1
PashtoTTS-Bench: automated screening for low-resource non-Latin-script text-to-speech
May 26, 2026Hanif RahmanSpeech Generation Evaluation TTS Evaluation
aIndependent Researcher
UniSRM: A Unified Speech Reward Model for Reasoning-Based Fine-grained Assessment
May 22, 2026Yuanyuan Wang, Dongchao Yang, Yayue Deng +4Reward Modeling Speech Generation Evaluation
The Chinese University of Hong Kong · Tsinghua University · Independent Researcher
Optimal Transport Audio Distance with Learned Riemannian Ground Metrics
May 7, 2026Wonwoo JeongSpeech Generation Evaluation Metric Learning
Sogang University
The False Resonance: A Critical Examination of Emotion Embedding Similarity for Speech Generation Evaluation
Apr 29, 2026Yun-Shao Tsai, Yi-Cheng Lin, Huang-Cheng Chou +5TTS Synthesis Speech Generation Evaluation
Graduate Institute of Communication Engineering, National Taiwan University, Taiwan · Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California, USA · Gilbert AI Lab, USA +2
PSP: An Interpretable Per-Dimension Accent Benchmark for Indic Text-to-Speech
Apr 28, 2026Venkata Pushpak Teja MentaTTS Synthesis Speech Generation Evaluation
Praxel Ventures

Sep 27, 2026, cs.CLJ/K move · Enter open · S save

NSV-Shift: A Contrastive Benchmark for Non-Speech Vocalization Understanding and Response Adaptation in Speech-to-Speech Models

Ziwei Chen

University of California San Diego

Speech Generation Evaluation Audio Understanding Emotion Recognition in Conversations

Open paper AskPDF arXiv

We introduce NSV-Shift, a contrastive benchmark for evaluating whether speech-to-speech models can understand non-speech vocalizations (NSVs) and adapt their responses accordingly. Each pair contains two conversations with identical lexical content that differ only in the NSV embedded in the final turn. Our pilot contains 22 human-verified pairs (44 audio conditions) and evaluates five models on NSV perception, emotion understanding, and response adaptation. Results show that models generally perform better at detecting NSVs than at interpreting their fine-grained emotional meaning or producing appropriately differentiated responses. The data construction pipeline, dataset, and evaluation pipeline are publicly available at https://github.com/ChenzwNina/nsv-construction.

Momentum

Related topics

Latest papers 35