cs.CLNov 11, 2026

Human-Likeness and Conversational Preference Diverge for Synthesized Agent Speech in Customer-Service Calls

Authors: Jeffrey M. Girard, Antony D'Avirro, Jason Z. Zheng, Jacqueline R. Vertino, Benjamin Peloquin

Organizations: Fluid Concepts Research, San Francisco, USA

Abstract

Listeners can judge an agent's speech on how human it sounds (human-likeness) and on how much they would want to talk to it (conversational preference). Listening tests rarely ask about conversational preference, so the relation between the two judgments is unclear. We compared them in paired-choice tests on 120 turns that actors improvised as customer-service representatives. On each trial, two agents spoke the same turn. In Study 1, both agents came from six commercial text-to-speech systems, each with eight voices. In Study 2, one was the actor's original recording and the other a system. Some systems ranked higher on human-likeness, and others on conversational preference. Listeners chose systems over actors in 19% of human-likeness trials but 36% of conversational preference trials (and 49% for the most preferred system). On both judgments, variation among a system's voices was about three-quarters of that between systems. Of three automatic naturalness scores, two predicted lower human-likeness, and none predicted conversational preference. In this setting, listeners tended to favor faster speech and fewer pauses between clauses on both judgments, and more pitch variation for conversational preference. Thus, measuring only human-likeness or testing one voice per system can misjudge which agents listeners would rather talk to.

Figures & tables

Explore similar work

CardsList
  1. Human or Machine? A Preliminary Turing Test for Speech-to-Speech Interaction

    Feb 27, 2026Xiang Li, Jiabao Gao, Sipei Lin +5Voice Agent EvaluationSpoken Dialogue Systems

  2. Talk2Agent: Benchmarking Voice Interfaces for Text Agents

    Sep 30, 2026Terumi Chiba, Guangzhi Sun, Zheqi Yuan +1ASR EvaluationVoice Agent Evaluation

  3. Louder, Longer, Livelier: Acoustic Shortcuts and Underspecified Rationales in Speech LLM Judges

    Sep 29, 2026Mingyue Huo, Shivam Mehta, Bhavin Jawade +2LLM-as-a-JudgeSpeech Quality Assessment