Speech Translation

Follow this topic Ask about this topic

Latest papers 58

9 papers in the last four weeks, up 125% on the four weeks before. 0.1% of all new papers.

All topics

Speech Meets ELF: Audio Conditional Continuous-Target Diffusion for Speech Recognition and Translation
Jun 9, 2026Xuanchen Li, Tianrui Wang, Yuheng Lu +11Speech Translation Automatic Speech Recognition
1Tianjin University · 2Shanghai Jiao Tong University · 3Zhipu AI
HydraQE: OSU's Submission for the IWSLT 2026 Speech Translation Metrics Shared Task
Jun 7, 2026Kevin Krahn, Eric Fosler-LussierSpeech Translation Speech Translation Evaluation
Dept. of Computer Science and Engineering The Ohio State University
Ouvia: A User-centered Framework for Measuring Usability of Speech Translation in Real-World Communication Scenarios
Jun 4, 2026Giuseppe Attanasio, Beatrice Savoldi, Daniel Chechelnitsky +4Speech Translation Speech Translation Evaluation
Instituto de Telecomunicações · Fondazione Bruno Kessler · Carnegie Mellon University +3
Automatic Labelling of Speech Translation Errors
Jun 4, 2026Dominik Macháček, Maike Züfle, Ondrej KlejchAudio-Language Model Evaluation Speech Translation
Charles University · University of Edinburgh · 1Charles University 2University of Edinburgh +2
AlignAtt4LLM: Fast AlignAtt for Decoder-Only LLMs at IWSLT 2026 Simultaneous Speech Translation Task
Jun 2, 2026Quentin Fuxa, Dominik MacháčekSpeech Translation Decoder-Only Language Models
Independent Researcher · Charles University, MFF, ÚFAL · & University of Edinburgh
A Pocket Offline Model for Simultaneous Speech Translation as CUNI Submission to IWSLT 2026
Jun 2, 2026Aziz Sharipov Ortega, Dominik MacháčekSpeech Translation Simultaneous Speech Translation
Charles University, MFF, ÚFAL · & University of Edinburgh
Benchmarking Speech-to-Speech Translation Models
Jun 2, 2026Alkis Koudounas, Hayato Futami, Quentin Jodelet +3Speech Translation Speech Quality Assessment
Sony Group Corporation, Japan · Carnegie Mellon University, USA
DOA: Training-Free Decoder-Only Attention Policy for Long-Form Simultaneous Translation with SpeechLLMs
May 29, 2026Sara Papi, Luisa BentivogliSpeech Translation Decoder-Only Language Models
Fondazione Bruno Kessler, Italy
OpenSTBench: Beyond Semantic Evaluation for Speech Translation
May 29, 2026Yanjie An, Yuxiang Zhao, Yichi Zhang +5Speech Translation Speech Translation Evaluation
X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · University of the Chinese Academy of Sciences, Beijing, China · Shanghai Innovation Institute, Shanghai, China +1
Bandwidth-Efficient and Privacy-Preserving Edge-Cloud Many-to-Many Speech Translation
May 27, 2026Yexing Du, Kaiyuan Liu, Youcheng Pan +4Speech Translation Speech Processing
Harbin Institute of Technology, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China · Harbin Institute of Technology, Harbin, China
Why We Need Speech to Evaluate Speech Translation
May 27, 2026Maike Züfle, Danni Liu, Vilém Zouhar +1Speech Translation Speech Prosody
Karlsruhe Institute of Technology · ETH Zurich
Conditioned in Training, Dispensable at Inference: Typological Conditioning for Multilingual Speech Translation
May 15, 2026Yu Pan, Tianjiao Zhang, Yang Hou +5Speech Translation Speech Language Models
School of Information Science and Electrical Engineering, Kyushu University, Fukuoka 819-0395, Japan · Recho Inc., Tokyo, Japan · National Institute of Informatics, Tokyo, Japan +4
Streaming Speech-to-Text Translation with a SpeechLLM
May 14, 2026Titouan Parcollet, Shucong Zhang, Xianrui Zheng +1Speech Translation Simultaneous Speech Translation
Samsung, AI Center – Cambridge, United Kingdom
Language-free Experience at Expo 2025 Osaka
May 1, 2026Michael Paul, Kenji Imamura, Xiaolin Wang +2Speech Translation Simultaneous Speech Translation
National Institute of Information and Communications Technology (NICT), Japan
Hierarchical Policy Optimization for Simultaneous Translation of Unbounded Speech
Apr 22, 2026Siqi Ouyang, Shuoyang Ding, Oleksii Hrinchuk +4RL for Language Models Speech Translation
1Carnegie Mellon University · 2NVIDIA
MoVE: Translating Laughter and Tears via Mixture of Vocalization Experts in Speech-to-Speech Translation
Apr 19, 2026Szu-Chi Chen, I-Ning Tsai, Yi-Cheng Lin +2Speech Translation Mixture-of-Experts Language Models
National Taiwan University, Taipei, Taiwan · NVIDIA, Taiwan
NaijaS2ST: A Multi-Accent Benchmark for Speech-to-Speech Translation in Low-Resource Nigerian Languages
Apr 17, 2026Marie Maltais, Yejin Jeon, Min Ma +7Speech Translation Audio-Language Models
Mila - Quebec AI Institute · McGill University, Canada · Google DeepMind +6
Ti-Audio: The First Multi-Dialectal End-to-End Speech LLM for Tibetan
Apr 13, 2026Jialing Wang, Yue Zhao, Yuhao Zhang +5Speech Language Models Low-Resource Speech Recognition
Minzu University of China, Beijing, China · The Chinese University of Hong Kong, Shenzhen, China
AI-Driven Modular Services for Accessible Multilingual Education in Immersive Extended Reality Settings: Integrating Speech Processing, Translation, and Sign Language Rendering
Apr 7, 2026N. D. Tantaroudas, A. J. McCracken, I. Karachalios +1Speech Translation
Institute of Communications and Computer Systems (ICCS), Iroon Polytechneiou 9, 15773 Zografou, Athens, Greece · DASKALOS-APPS, 183 Rue de l’Abbé Griffon, 01960 Péronnas, France · National Technical University of Athens, Leof. Alimou, Katechaki, Zografou 15772, Athens, Greece +1
English to Central Kurdish Speech Translation: Corpus Creation, Evaluation, and Orthographic Standardization
Apr 1, 2026Mohammad Mohammadamini, Daban Q. Jaff, Josep Crego +2Speech Translation Low-Resource Language Processing
LIUM, Le Mans University, Le Mans, France · Erfurt University, Erfurt, Germany · Koya University, Koysinjaq, Iraq +1
Automated Gradient-Driven Parameter Sharing for Low-Resource Multilingual Speech-to-Text Translation
Mar 26, 2026Ruiyan Sun, Satoshi NakamuraSpeech Translation Low-Resource Language Processing
School of Data Science · School of Aritificial Intelligence, The Chinese University of Hong Kong,Shenzhen
Streaming Translation and Transcription Through Speech-to-Text Causal Alignment
Mar 12, 2026Roman Koshkin, Jeon Haesung, Lianbo Liu +4Speech Translation Streaming ASR
SB Intuitions, Tokyo, Japan
SimulU: Training-free Policy for Long-form Simultaneous Speech-to-Speech Translation
Mar 11, 2026Amirbek Djanibekov, Luisa Bentivogli, Matteo Negri +1Speech Translation Simultaneous Speech Translation
MBZUAI, United Arab Emirates · FBK, Italy
Chunk-wise Attention Transducers for Fast and Accurate Streaming Speech-to-Text
Feb 27, 2026Hainan Xu, Vladimir Bataev, Travis M. Bartley +1Streaming ASR Attention Mechanisms
NVIDIA Corporation.
Mitigating Structural Noise in Low-Resource S2TT: An Optimized Cascaded Nepali-English Pipeline with Punctuation Restoration
Feb 25, 2026Tangsang Chongbang, Pranesh Pyara Shrestha, Amrit Sarki +1Speech Translation Low-Resource MT
Department of Electronics and Computer Engineering, Institute of Engineering, Pulchowk Campus, Lalitpur, Bagmati, Nepal
Cross-Modal Robustness Transfer (CMRT): Training Robust Speech Translation Models Using Adversarial Text
Feb 12, 2026Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes +1Speech Translation Adversarial Robustness
Department of Advanced Computing Sciences Maastricht University
Simulstream: Open-Source Toolkit for Evaluation and Demonstration of Streaming Speech-to-Text Translation Systems
Dec 19, 2025Marco Gaido, Sara Papi, Mauro Cettolo +2Speech Translation Simultaneous Speech Translation
Fondazione Bruno Kessler, Trento, Italy
Efficient and Adaptive Simultaneous Speech Translation with Fully Unidirectional Architecture
Apr 16, 2025Biao Fu, Donglei Yu, Minpeng Liao +5Speech Translation Simultaneous Speech Translation
School of Informatics, Xiamen University · Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism · Alibaba Group Tongyi Lab

Jun 9, 2026J/K move · Enter open · S save

Speech Meets ELF: Audio Conditional Continuous-Target Diffusion for Speech Recognition and Translation

Xuanchen Li, Tianrui Wang, Yuheng Lu, Zikang Huang+10

1Tianjin University · 2Shanghai Jiao Tong University · 3Zhipu AI

Speech Translation Automatic Speech Recognition Language Modeling Diffusion Language Models

Open paper AskPDF arXiv

Speech-to-text (S2T) systems for recognition (ASR) and translation (S2TT) typically generate discrete text tokens. In contrast, continuous-target language modelling performs generation in a continuous space, yet its potential for S2T remains unexplored. To bridge this gap, we propose ELF-S2T, an audio-conditioned continuous-target generative model for S2T. Built upon the pre-trained Embedded Language Flows (ELF) backbone, ELF-S2T processes speech via a frozen Whisper encoder and a single linear projector, prepending the resulting audio condition to the noisy text latent for in-context, flow-matching denoising. To prevent the model from over-relying on its pre-trained text context, we introduce audio forcing during training, and further amplify the audio condition via classifier-free guidance at inference. Experiments on LibriSpeech and CoVoST2 show that ELF-S2T achieves competitive ASR and S2TT performance. Crucially, our error analysis reveals that, although ASR and S2TT errors look very different on the surface, both stem from the same underlying cause, a close distance confusion in the continuous latent space. This finding naturally aligns with the continuous representation generation paradigm, indicating a common semantic mapping process beneath recognition and translation. Our code and pretrained models are publicly available at https://github.com/Sslnon/ELF-S2T.