Speech Translation

Follow this topic Ask about this topic

Latest papers 57

10 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

All topics

Dialect-Robust Speech Language Models with Synthetic Pseudo-Dialect Augmentation
Oct 7, 2026Shunsuke Mitsumori, Tomoya Mizumoto, Yusuke FujitaSpeech Language Models Language Model Robustness
SB Intuitions · Waseda University, Tokyo, Japan
Quality-Aware Self-Correcting Speech Translation on an Edge Device
Oct 6, 2026Zubair Ajmal Farooq, Diptesh KanojiaSpeech Translation Machine Translation
School of CS and EE University of Surrey United Kingdom · Institute for People-Centred AI University of Surrey United Kingdom
Learning When to Commit from Partial Speech for End-to-End Simultaneous Speech Translation
Oct 2, 2026Hieu Hoang, Amittai AxelrodSpeech Translation Simultaneous Speech Translation
Microsoft
Index-Translate: A Multilingual Translation Model Family -- Text, Speech, Controlled Dubbing, and Long-Document Translation
Sep 30, 2026Tianjiao Li, Mengran Yu, Chenyu Shi +8Machine Translation Speech Translation
Index LLM Team
MGhana-ST: A Low-Resource Speech Translation Dataset for Ghanaian Languages and an Analysis of Multilingual Training Trade-offs
Sep 30, 2026Frank Lawrence Nii Adoquaye Acquaye, Eric George Parakal, Jesse Johnson +2Speech Translation Low-Resource Language Processing
Ashesi University, Berekuso, Ghana · AdwumaTech AI, Accra, Ghana · HSE University, Moscow, Russia +1
DuraS2ST: Chain-of-Thought and Reinforcement Learning for Duration-Aligned Speech-to-Speech Translation
Sep 27, 2026Yayue Deng, Dingdong Wang, Yuxuan Hu +7Speech Translation Reinforcement Learning
The Chinese University of Hong Kong, China · Microsoft Corporation · Work done during an internship at Microsoft
All In Good Time: Causality-Aware Framework for LLM-Based Simultaneous Speech-to-Speech Translation
Sep 24, 2026Amir Hussein, Enas Albasiri, Travis M. Bartley +7Speech Translation Simultaneous Speech Translation
Johns Hopkins University · NVIDIA
Transcribe, Translate, and Optimize: Joint Reward Learning for Speech Translation
Sep 22, 2026Yanghe Dong, Wanting Huang, Weiran WangRL for Language Models Speech Translation
Independent Researcher · Department of Computer Science, University of Iowa, USA
Align, Integrate, and Fire: Efficient Token-Level Alignment for Zero-Shot SpeechLLMs
Sep 16, 2026Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes +1Speech Translation Automatic Speech Recognition
Department of Advanced Computing Sciences Maastricht University
CVSS-X: A Multilingual Speech-to-Speech Translation Corpus for 28 Languages
Sep 11, 2026Lucas Rafael Stefanel Gris, Alef Iury Siqueira Ferreira, Frederico Santos de Oliveira +4Speech Translation
Federal University of Goiás, Goiânia, Brazil · Federal University of Mato Grosso, Cuiabá, Brazil · São Paulo State University, São Paulo, Brazil
Kraken: LLM-based Speech-to-Speech Translation via Low-bitrate VQ and Dual-path Source Conditioning
Sep 11, 2026Hayato Futami, Hassan Shahmohammadi, Tushar Dhyani +5Speech Translation Speech Language Models
Sony Group Corporation · Sony Europe Limited
NOPE-HYPE: A Structured Simulation Workflow for Robust Speech-to-Text Across Diverse Acoustic Environments
Sep 9, 2026Niramay M. Patel, Bibek Behera, Raksha SharmaSpeech Translation Automatic Speech Recognition
IISER Bhopal, India · IIT Bombay, India · IIT Roorkee, India
Is Prosody Lost in Translation? Fine-Grained Cross-Lingual Prosody Similarity Across Languages
Aug 28, 2026Haopeng Xie, Ismail Rasim Ulgen, Sofia Son +2Speech Prosody Cross-Lingual Transfer
Center for Language and Speech Processing (CLSP), Johns Hopkins University Baltimore, MD, USA
VoxSumm: A Multilingual Corpus of Long-Form Spoken News for Joint Summarization and Translation
Aug 11, 2026Yejin Jeon, Marie Maltais, Virginia Ceccatelli +2Speech Translation Speech Language Models
Mila - Quebec AI Institute · McGill University, Canada · Google DeepMind +1
Breaking the Curse of Multilinguality in Many-to-Many Speech-to-Text Translation via a Resource-Aware Mixture of Speech Encoders
Aug 5, 2026Yexing Du, Kaiyuan Liu, Youcheng Pan +4Speech Translation Mixture of Experts
Harbin Institute of Technology, Harbin, China · Pengcheng Laboratory, Shenzhen, China
The Role of Disfluencies in Speech Translation
Aug 3, 2026Maike Züfle, Maria Teleki, Fabian Retkowski +5Speech Translation Speech Translation Evaluation
Karlsruhe Institute of Technology · Texas A&M University · ETH Zurich +1
SimulS2ST-Omni: Data-Efficient Streaming Speech-to-Speech Translation via Explicit Trajectory Supervision
Jul 22, 2026Rongshen He, Xinyu Liang, Dekun Chen +3Speech Translation Simultaneous Speech Translation
1The Chinese University of Hong Kong, Shenzhen
When to Use Extra Context: Evidence-Grounded Terminology Adaptation for Simultaneous Speech Translation
Jul 20, 2026Zeyu Yang, Satoshi NakamuraSpeech Translation Simultaneous Speech Translation
The Chinese University of Hong Kong, Shenzhen
Do LLMs Need Architectural Changes for Simultaneous Speech Translation? A Prefix-to-Prefix Data Driven Approach
Jul 14, 2026Junkun Chen, Jian Xue, Ming Tang +4Speech Translation Decoder-Only Language Models
Microsoft, USA
NAVER LABS Europe Submission to the Instruction-following 2026 Short Track
Jul 2, 2026Marcely Zanon Boito, Hemant Yadav, Jean-Luc Meunier +1Speech Translation Question Answering
1NAVER LABS Europe, France · 2IIIT Delhi, India
STEB: A Speech-to-Speech Translation Expressiveness Benchmark for Evaluating Beyond Translation Fidelity
Jun 24, 2026Sitong Cheng, Weizhen Bian, Songjun Cao +9Speech Translation Speech Translation Evaluation
Hong Kong University of Science and Technology · Tencent Youtu Lab · Shenzhen International Graduate School, Tsinghua University
Measuring User's Mental Models of Speech Translation in Human-AI Collaboration
Jun 23, 2026HyoJung Han, Nishant Balepur, Jordan Boyd-Graber +1Speech Translation Human-AI Collaboration
University of Maryland, College Park, USA
MLLP-VRAIN UPV system for the IWSLT 2026 Simultaneous Speech Translation task
Jun 15, 2026Jorge Iranzo-Sánchez, Gerard Mas-Mollà, Adrià Giménez +3Speech Translation Simultaneous Speech Translation
Machine Learning and Language Processing, VRAIN, Universitat Politècnica de València
Bridging the Usability Gap: Lessons from Interpreting Studies for Machine Interpreting Design
Jun 14, 2026Claudio FantinuoliSpeech Translation Speech Translation Evaluation
University of Mainz
Evaluating and Preserving Lexical Stress in English-to-Chinese Speech-to-Speech Translation
Jun 13, 2026Yuchen Song, Xi Chen, Mingze Li +1Speech Translation Speech Translation Evaluation
The Chinese University of Hong Kong, Shenzhen, China · Shenzhen Loop Area Institute, China
A Practical Evaluation Method for Long-Form Simultaneous Speech-to-Speech Translation
Jun 13, 2026Yulin Xue, Siqi Ouyang, Lei LiSpeech Translation Simultaneous Speech Translation
Carnegie Mellon University
Leveraging Audio-LLMs to Filter Speech-to-Speech Training Data
Jun 11, 2026Qixu Chen, Satoshi NakamuraSpeech Translation Audio-Language Models
School of Data Science, The Chinese University of Hong Kong, Shenzhen, China · School of Artificial Intelligence The Chinese University of Hong Kong, Shenzhen, China
NaturalFlow: Reducing Disruptive Pauses for Natural Speech Flow in Simultaneous Speech-to-Speech Translation
Jun 11, 2026Dongwook Lee, Youngho Cho, Sangkwon Park +2Speech Translation Simultaneous Speech Translation
IPAI and 2ECE, Seoul National University, Seoul, 08826, Korea · 2ECE, Seoul National University, Seoul, 08826, Korea · Department of AI, University of Seoul, Seoul, 02504, Korea
PiDA: Phonetically-Informed Data Augmentation for Robust Vietnamese Speech Translation
Jun 11, 2026Giang Son Nguyen, Tung X. Nguyen, Hieu Minh Truong +3Speech Translation Data Augmentation
VinUniversity, Vietnam · University of Technology Sydney, Australia · Monash University, Australia
Speech Meets ELF: Audio Conditional Continuous-Target Diffusion for Speech Recognition and Translation
Jun 9, 2026Xuanchen Li, Tianrui Wang, Yuheng Lu +11Speech Translation Automatic Speech Recognition
1Tianjin University · 2Shanghai Jiao Tong University · 3Zhipu AI

Oct 7, 2026J/K move · Enter open · S save

Dialect-Robust Speech Language Models with Synthetic Pseudo-Dialect Augmentation

Shunsuke Mitsumori, Tomoya Mizumoto, Yusuke Fujita

SB Intuitions · Waseda University, Tokyo, Japan

Speech Language Models Language Model Robustness Synthetic Data Augmentation Speech Translation

Open paper AskPDF arXiv

Speech Language Model (SLM) performance often degrades on dialects due to data scarcity. Conventional text-to-speech (TTS) augmentation struggles to cover diverse dialects as it requires a certain amount of real dialect speech. We propose synthesizing pseudo-dialect speech by converting LLM-generated dialect text via a standard-language TTS model, requiring zero real dialect speech. Additionally, we introduce intermediate standard-text prediction during training, acting as semantic normalization for downstream tasks. We evaluate dialect understanding via dialect-to-English speech translation across Japanese, German, and Chinese dialects. Compared to synthetic standard speech baselines, pseudo-dialect augmentation improves scores for Japanese (from 25.38 to 26.24) and German (from 31.57 to 32.47). Furthermore, the intermediate standard-text prediction effectively bridges the semantic gap, boosting performance to 28.26 for Japanese and from 11.67 to 16.37 for Chinese. These results suggest that our approach scales to various languages without requiring speech resources specific to each dialect.