Quality-Aware Self-Correcting Speech Translation on an Edge Device

Zubair Ajmal Farooq; Diptesh Kanojia

cs.CLOct 6, 2026

Quality-Aware Self-Correcting Speech Translation on an Edge Device

Authors: Zubair Ajmal Farooq, Diptesh Kanojia

Organizations: School of CS and EE University of Surrey United Kingdom · Institute for People-Centred AI University of Surrey United Kingdom

Abstract

We present a fully offline speech-to-speech translation pipeline that runs on a Jetson Nano (4 GB) and corrects its own weak translations without retraining. A Whisper-tiny ASR feeds an Opus-MT translator; multilingual BERT cosine similarity acts as a Quality Estimation (QE) gate, triggering a secondary-pass correction when confidence falls below a pre-defined threshold $τ$ . We compare three correction methods: QE reranking (M1), Minimum Bayes-Risk decoding (M2), and constrained beam search (M3). On 1,012 FLORES-200 sentences (English-Spanish), M2 at $τ=0.90$ produces statistically significant improvements over greedy decoding on BLEU (+0.67, p<0.001), ChrF (+0.51, p<0.001), and COMET (+0.0020 at N=3, p=0.002); M1 yields no significant gains, and M3 is significantly worse than baseline (p>0.99). Our central finding is that QE functions effectively as a gate but poorly as a ranker: removing the QE model from candidate selection (M1 $\to$ M2) does not hurt quality and frees 680 MB from the critical path. Using a gain-to-edit ratio adapted from the post-editing-effort literature, we further show that smaller candidate pools (N=3) yield more surgical corrections with better semantic adequacy, while larger pools (N=10) maximise lexical reward. We release the system and demonstrate live translation across six language pairs.

Figures & tables

Method	$\Delta$ BLEU	$\Delta$ ChrF	$\Delta$ COMET
M1 (QE-rerank)	$-0.14$	$-0.11$	$-0.0004$
M2 (MBR)	$\mathbf{+0.67}^{***}$	$\mathbf{+0.51}^{***}$	$\mathbf{+0.0011}$
M3 (CBS)	$-0.96^{\dagger}$	$-0.86^{\dagger}$	$-0.0061^{\dagger}$

Table 1: Aggregate deltas over greedy baseline at

\tau{=}0.90

N{=}10

. Paired bootstrap,

n{=}1000

. ∗∗∗ :

p{<}0.001

. † : significantly worse than baseline (

p{>}0.99

, one-tailed).

\Delta

COMET for M2 at

N{=}10

is not individually significant (

p{=}0.107

); see § 6.2 .

$N$	$\Delta$ BLEU	$\Delta$ ChrF	$\Delta$ COMET
3	$+0.59^{***}$	$+0.37^{***}$	$\mathbf{+0.0020^{**}}$
5	$+0.47^{**}$	$+0.32^{**}$	$+0.0012$
10	$\mathbf{+0.67^{***}}$	$\mathbf{+0.51^{***}}$	$+0.0011$

Table 2: M2 (MBR) corrections at

\tau{=}0.90

across beam pool sizes. Paired bootstrap,

n{=}1000

. ∗∗ :

p{<}0.01

. ∗∗∗ :

p{<}0.001

\Delta

COMET at

N{=}5

and

N{=}10

are not significant at

\alpha{=}0.05

(one-tailed

p{=}0.082

and

0.107

$N$	GER (median)	Triggered
3	$\mathbf{+0.0053}$	485
5	$+0.0023$	542
10	$+0.0048$	580

Table 3: Median per-sentence GER for M2 at

\tau{=}0.90

. Triggered = number of sentences where QE fired and correction was applied (out of 1,012).

Figure 1: Improvement over greedy baseline across four metrics for M2 (MBR) at $\tau{=}0.90$ , with paired bootstrap significance ( $p{<}0.05$ ). $\Delta$ COMET and GER median are scaled $\times 100$ for joint visibility. Lexical metrics (BLEU, ChrF) peak at $N{=}10$ ; semantic metric (COMET) and per-sentence efficiency (GER median) peak at $N{=}3$ .

Figure 2: Per-stage latency comparison on the Jetson Nano. Sequential-load (top, $\sim$ 82 s) is dominated by per-stage model reload; single-load resident (bottom, $\sim$ 41 s) eliminates reload at the cost of higher peak memory.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

$\tau$	$N$	Trig%	BLEU-M1	BLEU-M2	BLEU-M3	ChrF-M1	ChrF-M2	ChrF-M3
0.75	3	0.1%	26.10 ( $+0.01$ )	26.10 ( $+0.01$ )	26.10 ( $+0.01$ )	54.85 ( $+0.00$ )	54.85 ( $+0.00$ )	54.87 ( $+0.02$ )
0.75	5	0.1%	26.10 ( $+0.01$ )	26.10 ( $+0.01$ )	26.10 ( $+0.01$ )	54.87 ( $+0.02$ )	54.85 ( $+0.00$ )	54.87 ( $+0.02$ )
0.75	10	0.1%	26.10 ( $+0.01$ )	26.10 ( $+0.01$ )	26.10 ( $+0.01$ )	54.86 ( $+0.01$ )	54.87 ( $+0.02$ )	54.86 ( $+0.01$ )
0.80	3	1.3%	26.10 ( $+0.01$ )	26.10 ( $+0.01$ )	26.11 ( $+0.02$ )	54.85 ( $+0.00$ )	54.85 ( $+0.00$ )	54.87 ( $+0.02$ )
0.80	5	1.3%	26.10 ( $+0.01$ )	26.09 ( $+0.00$ )	26.11 ( $+0.02$ )	54.86 ( $+0.01$ )	54.84 ( $-0.01$ )	54.87 ( $+0.02$ )
0.80	10	1.3%	26.09 ( $+0.00$ )	26.10 ( $+0.01$ )	26.10 ( $+0.01$ )	54.85 ( $+0.00$ )	54.87 ( $+0.02$ )	54.86 ( $+0.01$ )

Appendix

Table 4: Full

4\times 3

grid search results over 1,012 FLORES-200 sentences. Internal baselines: BLEU

26.09

, ChrF

54.85

. M1 = QE-Rerank, M2 = MBR, M3 = CBS. Parenthesised values are deltas from baseline. Bold marks the strongest M2 configuration reported in the main body.

Explore similar work

Jun 7, 2026cs.CL

HydraQE: OSU's Submission for the IWSLT 2026 Speech Translation Metrics Shared Task

We present HydraQE, our contribution to the IWSLT 2026 Speech Translation Metrics shared task. HydraQE is an end-to-end, reference-free quality estimation (QE) system for speech translation built on a Qwen3-ASR backbone, which accepts source audio and a translation hypothesis as joint input. Hidden states from all backbone layers are combined via a learnable sparsemax scalar mix, then re-encoded by a lightweight bidirectional Transformer to enable full cross-modal interaction prior to pooling into a shared embedding. Three independent prediction heads are trained on complementary supervision signals: human direct assessment (DA) annotations, MetricX-24 pseudo-labels, and xCOMET pseudo-labels. To address the scarcity of human-annotated data, we train on a combination of synthetically corrupted examples and silver pseudo-labeled machine translation outputs, using a curriculum that begins on synthetic and silver data and gradually shifts toward human-annotated examples. HydraQE outperforms cascaded text-based baselines and prior direct speech QE systems, demonstrating that end-to-end speech translation QE is competitive with cascaded approaches.

Kevin Krahn, Eric Fosler-Lussier

Dept. of Computer Science and Engineering The Ohio State University

Speech Translation Image Quality Assessment

Ask PDF

May 15, 2026cs.CL

CompactQE: Interpretable Translation Quality Estimation via Small Open-Weight LLMs

Current state-of-the-art Quality Estimation (QE) in machine translation relies on massive, proprietary LLMs, raising data privacy concerns. We demonstrate that smaller, open-source LLMs (<30B parameters) are a viable, cost-effective and privacy-preserving alternative. Using a single-pass prompting strategy, our models simultaneously generate quality scores, MQM error annotations, suggested error corrections, and full post-editions. Our analysis shows these models achieve highly competitive system-level correlations with human judgments that outperform traditional neural metrics, fine-tuned models, and human inter-annotator agreement, effectively approximating the capabilities of much larger proprietary LLMs.

Kamil Guttmann, Zofia Fraś, Artur Nowakowski +1

Laniqo, Pozna´n, Poland · Faculty of Mathematics and Computer Science, Adam Mickiewicz University, Pozna´n, Poland

Machine Translation Quality Machine Translation Inter-Annotator Agreement Image Quality Assessment

Ask PDF

Apr 15, 2025cs.CL

AskQE: Question Answering as Automatic Evaluation for Machine Translation

How can a monolingual English speaker determine whether an automatic translation in French is good enough to be shared? Existing MT error detection and quality estimation (QE) techniques do not address this practical scenario. We introduce AskQE, a question generation and answering framework designed to detect critical MT errors and provide actionable feedback, helping users decide whether to accept or reject MT outputs even without the knowledge of the target language. Using ContraTICO, a dataset of contrastive synthetic MT errors in the COVID-19 domain, we explore design choices for AskQE and develop an optimized version relying on LLaMA-3 70B and entailed facts to guide question generation. We evaluate the resulting system on the BioMQM dataset of naturally occurring MT errors, where AskQE has higher Kendall's Tau correlation and decision accuracy with human ratings compared to other QE metrics.

Dayeon Ki, Kevin Duh, Marine Carpuat

University of Maryland · Johns Hopkins University

Machine Translation Quality Machine Translation

Ask PDF