A capable voice assistant must do more than recognize words; it must understand requests, reason over context, sustain conversations, and act through tools. We present Apriel Voice 27B Preview, an open speech-language model that unifies these capabilities within a shared reasoning backbone while retaining text and image input modalities. Built on the post-trained Gemma 4 26B-A4B checkpoint, Apriel Voice directly processes speech and generates both responses and tool calls without requiring a separate dialogue model. We introduce a two-stage training procedure to adapt the model to speech. The first stage establishes the speech interface, while the second jointly adapts the projector and backbone through supervised fine-tuning, text replay, and source-dependent cross-entropy and forward Kullback--Leibler divergence supervision. Automatically constructed speech-conditioned dialogues provide conversational training data, while local teacher serving, overlapping teacher and student computation, and fused full-vocabulary losses enable efficient online distillation. In thinking mode, Apriel Voice achieves a score of 98.90 on Big Bench Audio, 81.13 on Speech-BFCL, and a task success rate of 65.51% on τ-Voice, with a mean latency of 1.91 seconds. It also achieves 51.27% English EVA accuracy and 74.13% experience pass@1. Finally, we deploy Apriel Voice 27B Preview on a custom serving platform to support interactive voice interactions.
Figures & tables
Figure 1: One reasoning backbone across speech, text, and images. Apriel Voice receives speech through an encoder and projector, retains the original text and image interfaces, and generates both responses and tool calls. Application components execute tools, return their results to the conversation, and stream response text to a replaceable synthesizer. The frozen teacher is used only during training.
Component
Stage one
Stage two
Speech encoder
Frozen
Frozen
Speech projector
Trainable
Trainable
Language backbone and router
Frozen
Trainable
Original vision tower and bridge
Frozen
Frozen
Online teacher
Frozen
Frozen
Speech synthesizer
External
External
Table 1: Component updates during speech adaptation. The projector has approximately 47.7 M parameters. The teacher supplies training supervision; the independently selected synthesizer is used at serving time and is not trained by this recipe.
Figure 2: Learning a speech interface, then broadening task capability. Stage one trains the projector with the backbone fixed. Stage two updates both on speech tasks, synthetic conversations, tool interactions, and text replay. Source-dependent CE, FKL, or combined supervision supplies task targets and teacher distributions. The speech encoder and original visual components remain frozen.
Training quantity
Value
Total rows
17.56 M
Packed bins
1.56 M
Average token-fill efficiency
92.5%
Tokens / loss-bearing tokens per epoch
17.5 B / 2.6 B
Completed epochs / total token exposure
3 / 52.6 B
Nodes / teacher GPUs / student GPUs
8 / 16 / 48
Table 2: Stage-two training scale. Rows denote training examples; bins include exposure weighting. M/B: million/billion. The duration describes the optimized training run.
Figure 3: From synthetic task dialogues to speech-conditioned training examples. The pipeline selects user spans, normalizes their spoken form, synthesizes them with a conversation-level reference voice, and applies fidelity checks. Assistant targets and tool messages remain text. The illustrated asset-management dialogue is schematic.
Figure 4: Local online supervision with one-batch lookahead. Schematic schedule in which teacher inference for the next batch overlaps the student’s current training work. Teacher–student communication stays within each node; student state is sharded across training workers. Workers complete pending teacher requests before saving checkpoints.
Figure 5: Multi-dimensional comparison of voice model performance. The radar plot summarizes each model across five axes: audio reasoning, τ -Voice responsiveness, τ -Voice accuracy, EVA-A, and EVA-X. Higher values are better on all axes. τ -Voice accuracy is measured as pass@1 percentage, while EVA-A and EVA-X report the model’s accuracy and experience scores on the EVA benchmark. Tau-Voice responsiveness converts response latency into a bounded 0–100 score as described in Equation 16 , where lower latency receives a higher score. Apriel Voice is shown as the green reference profile, and the remaining profiles are grouped by source category.
Model
Source
Type
Access
Gemini 3.8 Family
Proprietary
S2S
Closed
GPT-realtime-2.1
Proprietary
S2S
Closed
Flux + GPT-5.6
Proprietary
3-stage
Closed
Flux + Gemma4-26B
Mixed
3-stage
Partial
Parakeet + GPT-5.6
Mixed
3-stage
Partial
Parakeet + Gemma4-26B
Open-source
3-stage
Open
Table 3: Summary of baseline models: source, type, and access. We compare Apriel Voice against proprietary systems (S2S and 3-stage), mixed systems (3-stage cascaded), and open-source models (2-stage and 3-stage). 2-stage systems project speech into a language backbone; 3-stage systems cascade ASR, LLM, and TTS; S2S generates spoken output directly.
Metric
Proprietary
Open-Source
Flux
Gemini 3.8 Family
GPT-realtime-2.1
Parakeet
UltraVox-v0.6 (Qwen)
Gemma4-12B-IT
Audex-30B-A3B
Apriel Voice
Speech Recognition (WER %, lower is better)
LS test-clean
3.51
2.10
1.85
1.65
6.11
5.77
1.36
2.23
LS test-other
8.26
4.42
4.34
3.14
8.08
18.51
3.08
5.15
GigaSpeech
10.68
10.16
10.57
10.21
13.09
29.82
9.37
10.20
AMI-IHM
17.01
19.41
23.63
10.17
137.15
129.30
16.93
17.44
Table 4: Speech recognition and translation performance across Proprietary and Open-source systems. Recognition metrics (WER, %) and translation metrics (BLEU) are averaged separately. Lower WER is better; higher BLEU is better. LS = LibriSpeech, μ B = μ -Bench (en-US), DE/FR/ES = German/French/Spanish. ASR-only models are not evaluated on Speech Translation tasks and denoted as –.
Type
Source
Model
BFCL
BBA
IFEval
MTB
SS
AA
Openhermes
GSM8K
Avg.
3-stage
Prop.
Flux + GPT-5.6
71.12
96.8
53.91
77
68.88
68.6
71.6
84.53
74.06
Mixed
Flux + Gemma4-26B
76.43
98.4
88.15
84.75
84.82
75.6
83.4
91.05
85.33
Mixed
Parakeet + GPT-5.6
72.77
98
49.23
78.12
85.82
73.4
74
87.34
77.34
OSS
Parakeet + Gemma4-26B
78.35
98.2
87.48
88.62
84.96
77.2
86
92.19
86.63
S2S
Prop.
Gemini 3.8 Family
86.94
99.5
89
92.19
90.66
82.2
88.6
96.89
90.75
Prop.
GPT-realtime-2.1
80.24
94.6
88.49
78.44
85.48
78.4
84.2
93.18
85.38
Table 5: Comprehensive speech reasoning evaluation across model types and architectures. All scores use a 0–100 scale; higher is better. Apriel Voice 27B Preview is the highest-performing open-source 2-stage model (highlighted in green), outperforming other OSS baselines (highlighted in blue). Metrics: BFCL (Berkeley Function Calling Leaderboard), BBA (Big-Bench-Audio), IFEval (Instruction Following Evaluation), MTB (MT-Bench), SS (SpokenSquad), AA (AlpacaAudio), GSM8K (grade-school math). Type: 2-stage (speech projector + backbone), 3-stage (ASR + LLM + TTS cascade), S2S (end-to-end speech-to-speech). Source: Prop. (proprietary), Mixed (proprietary + open-source components), OSS (open-source). All models are evaluated in thinking mode and follow the individually suggested hyperparameter settings.
Type
Source
System
EVA
τ -Voice
EVA-A
EVA-X
pass@1
Latency (s)
S2S
Prop.
GPT-realtime-2.1
59.03
61.6
69.82
1.55
S2S
Prop.
Gemini 3.8 Live Extended Thinking
54.82
73.5
84.19
1.50
3-stage
Mixed
Flux + Gemma4-26B
36.07
29.22
65.92
2.92
3-stage
OSS
Parakeet + Gemma4-26B
39.97
3.93
67.78
3.01
2-stage
OSS
UltraVox-v0.6 (Qwen)
8.94
1.91
26.80
3.95
Table 6: Service accuracy and conversational experience. EVA-A/X and τ -Voice pass@1 scores use a 0–100 scale; latency is in seconds. System types: S2S (end-to-end speech-to-speech), 3-stage (ASR + LLM + TTS cascade), 2-stage (speech projector + backbone). Prop./OSS = Proprietary/Open-source. All 3-stage and 2-stage models use Cartesia Sonic 3.5 TTS [ 5 ] by default. Domain-specific results are provided in Appendix E .
Figure 6: EVA accuracy–experience trade-offs. EVA-A measures task accuracy on the x-axis, while EVA-X measures conversational experience on the y-axis; higher values indicate better performance on both metrics. Marker shapes denote model source category, including open-source, mixed, proprietary, and Apriel Voice. Apriel Voice achieves the highest EVA-X score among the compared models while maintaining competitive EVA-A accuracy, emphasizing conversational quality without leaving the accuracy frontier. The dashed curve traces the Pareto frontier of models that are not dominated in the EVA-A–EVA-X trade-off.
Figure 7: τ -Voice accuracy–latency trade-offs. τ -Voice pass@1 accuracy is shown on the x-axis, while Time-to-First-Audio (TTFA) is shown on the y-axis; higher accuracy and lower latency indicate better performance. Marker shapes denote model source category, including open-source, mixed, proprietary, and Apriel Voice. The shaded upper-right region highlights the preferred operating regime of high accuracy and low TTFA. Apriel Voice lies in the shaded optimal region, outperforming most open-source baselines.
Component
Parakeet + Gemma4-26B
Apriel Voice
Speech fidelity
99.78
98.89
Faithfulness
66.33
43.49
Task completion
60.0
69.94
Conciseness
84.99
86.80
Conversation progression
82.67
67.76
Turn taking
40.54
86.52
Table 7: EVA component means. Apriel Voice uses thinking mode. Component scores (%) are averaged equally across domains. EVA-A and EVA-X require their respective component criteria to pass jointly.
Benchmark
Gemma 4 26B-A4B (T)
Apriel Voice (T)
MMLU-Pro
85.27
84.42
GPQA Diamond [ 40 ]
77.80
80.40
AIME 2024
92.00
92.00
AIME 2025
81.67
81.67
MATH-500
92.76
92.88
AMC 2023
100.00
100.00
Table 8: Internal text benchmarks: Gemma 4 and Apriel Voice. All results use text input in thinking (T) mode unless noted. The τ2 mean covers three domains ( Tab. 14 ).
Tools
Gemma 4 26B-A4B (T)
Apriel Voice 27B Preview (T)
Apriel Voice 27B Preview (NT)
Oracle
22.7 / 57.8
23.0 / 57.6
14.7 / 44.7
+5 tools
23.4 / 60.2
24.7 / 61.1
16.0 / 46.2
+10 tools
23.5 / 60.0
23.6 / 59.9
15.9 / 44.9
+15 tools
25.9 / 60.9
23.7 / 59.4
14.2 / 44.8
Table 9: EnterpriseOps-Gym internal comparison [ 29 ] . Each cell gives task / verifier success (%; ↑ ). Extra tools are distractors. T/NT: thinking/non-thinking. Public reference tables appear in Tabs. 16 and 17 .
Benchmark
Gemma 4 26B
Apriel Voice 27B Preview
MM-IFEval [ 9 ]
77.13
74.75
OCRBench [ 26 ]
859
836
OCRBench v2 (EN, M) [ 12 ]
63.92
62.91
MathVista (M) [ 27 ]
80.50
79.60
MathVision (M) [ 47 ]
64.14
67.10
MMMU-Pro [ 52 ]
70.23
69.88
Table 10: Image capabilities after speech adaptation. Internal evaluations; ↑ for all displayed scores. EN denotes English; M denotes the MINI subset. OCRBench is scored out of 1000; the other rows use a 0–100 scale. The Gemma baseline is Gemma 4 26B-A4B; both Gemma and Apriel Voice use thinking mode. Document-extraction results appear in Tab. 13 .
Figure 8: Stage-two optimization. Cross-entropy and forward KL use separate scales. Faint curves show measured losses and solid curves show smoothed trends. Components need not decrease monotonically under mixed supervision. Figure 9 includes the combined objective.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Stage-two training losses. The combined objective, CE, and FKL use separate scales. Faint curves show measured losses; solid curves show smoothed trends. The component losses reflect both the sampled tasks and their supervision weights.
Stage
Audio / epoch
Epochs
Audio exposure
1: Interface
32.2k hrs
2
64.4k hrs
2: Broad SFT
61.5k hrs
3
184.6k hrs
Appendix
Table 11: Audio duration and training exposure. Per-epoch durations are 32,213 and 61,540 hours. Repetition gives 64,426 and 184,620 audio-hour exposures; k denotes thousand. Text contributes tokens without an assigned audio duration.
Supervision route
CE weight
FKL weight
Task learning: pure CE
1
0
Distillation: pure FKL
0
1
Weighted CE+FKL
αd>0
βd>0
Appendix
Table 12: Source-dependent objective selection. Internally generated responses supply task targets; the online teacher supplies full-vocabulary distributions on sources configured for distillation.
Benchmark
Gemma 4 26B (T)
Apriel Voice 27B Preview
GPT-4.1
OmniDocBench (EN) [ 36 ]
25.40
32.90
36.80
Appendix
Table 13: Document-understanding results. The OmniDocBench row covers English pages; the benchmark evaluates text, formulas, tables, and reading order [ 35 ] . The Gemma baseline is Gemma 4 26B-A4B; both Gemma and Apriel Voice use thinking mode.
Agent
Airline
Retail
Telecom
Avg.
Gemma 4 26B (T)
72.00
73.10
61.70
68.93
Gemma 4 26B (NT)
43.33
65.50
50.88
53.23
Apriel Voice 27B Preview (T)
76.00
69.59
67.25
70.95
Apriel Voice 27B Preview (NT)
38.67
58.77
50.00
49.14
Appendix
Table 14: Internal τ2 -Bench with GPT-5.2 as user simulator. Task success (%; ↑ ). T/NT: thinking/non-thinking. The Gemma baseline is Gemma 4 26B-A4B. Means weight the three domains equally.
Model
MMLU Pro
GPQA Diamond
τ2 Avg.
Gemma 4 31B
85.20
84.30
76.90
Gemma 4 26B-A4B
82.60
82.30
68.20
Gemma 4 12B
77.20
78.80
69.00
Appendix
Table 15: Published Gemma 4 text benchmarks. Officially reported results are extracted from [ 15 ] .
Model / setting
Task success (%)
Independent Artificial Analysis evaluations
Claude Fable 5 (with fallback)
51.1
Gemini 3.7 Flash (medium)
50.4
Gemini 3.5 Flash
50.1
DeepSeek V4 Pro 0813 (max)
49.6
Grok 4.6 (high)
48.3
Appendix
Table 16: EnterpriseOps-Gym oracle task success. All rows report strict task success (%; ↑ ) with oracle tools: every final-state verifier must pass. AA runs use Stirrup and three repeats per task [ 2 ] . Internal Gemma and Apriel Voice evaluations use a different harness and are grouped separately. T/NT: thinking/non-thinking. Verifier success and distractor-tool settings appear in Tab. 9 .
Model / setting
Task success (%)
Claude Opus 4.6
45.9
Claude Sonnet 4.6
42.2
Claude Opus 4.5
39.4
Gemini-3.1-Pro
38.7
Claude Sonnet 4.5
34.1
Gemini-3-Flash
34.0
Appendix
Table 17: EnterpriseOps-Gym repository reference models. Task success (%; ↑ ) with oracle tools on the full benchmark, from the official repository [ 41 ] . Think and Inst. denote thinking and instruction-tuned variants.
Type
Source
System
Airline
ITSM
HR
EVA-A
EVA-X
EVA-A
EVA-X
EVA-A
EVA-X
S2S
Prop.
GPT-realtime-2.1
54.0
51.3
72.9
68.8
50.2
64.7
S2S
Prop.
Gemini 3.8 Live Extended Thinking
62.67
70.0
60.0
75.4
41.8
75.1
3-stage
Mixed
Flux + Gemma4-26B
47.3
30.7
25.83
28.33
35.08
28.63
3-stage
OSS
Parakeet + Gemma4-26B
54.0
2.7
40.9
9.1
25.0
0.0
2-stage
OSS
UltraVox-v0.6 (Qwen)
15.33
0.0
7.08
2.92
4.42
2.81
Appendix
Table 18: EVA accuracy and experience by domain. EVA-A and EVA-X scores (higher is better) across airline, ITSM (information technology service management), and healthcare HR domains. All 2-stage and 3-stage models use thinking mode and Cartesia Sonic 3.5 [ 5 ] for consistency and fair comparisons. OSS baselines are highlighted in blue; Apriel Voice 27B Preview in green.
Type
Source
System
Airline
Retail
Telecom
Pass@1
Latency
Pass@1
Latency
Pass@1
Latency
S2S
Prop.
GPT-realtime-2.1
70.00
1.80
77.19
1.66
62.28
1.19
S2S
Prop.
Gemini 3.8 Live Extended Thinking
78.00
1.44
77.19
1.50
97.37
1.55
3-stage
Mixed
Flux + Gemma4-26B
65.30
3.24
79.82
2.87
52.63
2.64
3-stage
OSS
Parakeet + Gemma4-26B
70.00
3.37
78.95
3.00
54.39
2.65
2-stage
OSS
UltraVox-v0.6 (Qwen)
42.67
3.83
18.71
3.85
19.01
4.18
Appendix
Table 19: Task success and latency on τ -Voice by domain. Pass@1 (%) and latency (s) across airline, retail, and telecom domains. Apriel Voice 27B Preview uses thinking mode with Cartesia Sonic 3.5 [ 5 ] for consistency with reference systems. OSS baselines are highlighted in blue; Apriel Voice 27B Preview in green.
We introduce NemotronLabs VoiceChat, an open full-duplex speech-to-speech model with native tool-calling capabilities. NemotronLabs VoiceChat combines a streaming speech encoder and decoder-only language model with parallel specialized output streams for agent text and structured function calls, an auxiliary RNN-T branch for incremental user transcription, and a streaming TTS decoder. This design enables the model to listen, transcribe, reason, invoke tools, and speak within a unified streaming architecture while preserving the temporal behavior required for natural conversation. On Full-Duplex-Bench 1.0, NemotronLabs VoiceChat achieves the lowest pause-handling takeover rates among evaluated open-weight systems, 100% takeover following user interruptions, and a 4.33/5 post-interruption response-quality score. On Full-Duplex-Bench 1.5, it resumes its response after user backchannels in 93% of cases. NemotronLabs VoiceChat obtains a 55.1 normalized average on VoiceBench and, on Full-Duplex-Bench 3.0 (FDB 3.0), achieves 82.5% tool-selection F1, while argument accuracy and end-to-end tool execution remain areas for improvement. These results demonstrate that full-duplex interaction, speech recognition and generation, general language capabilities, and external tool use can be integrated in a single open speech-to-speech model without sacrificing real-time conversational behavior.
Voice assistants increasingly rely on Speech Language Models (SpeechLMs) to interpret spoken queries and execute complex tasks, yet existing benchmarks lack domain breadth, acoustic diversity, and compositional reasoning complexity to evaluate tool-calling performance. We introduce Audio2Tool, a large-scale dataset comprising approximately 30,000 queries designed to assess tool-calling capabilities of SpeechLMs across three primary domains: Smart Car, Smart Home, and Wearables. Our benchmark features a multi-tier complexity hierarchy, ranging from simple direct commands to complex multi-intent and needle-in-a-haystack extraction to isolate distinct failure modes. To ensure realism, we employ zero-shot voice cloning text-to-speech synthesis and diverse noise profiles to simulate in-the-wild conditions. Evaluations of state-of-the-art SpeechLMs and ASR-LLM pipelines show strong performance on simple commands but significant degradation under compositional and acoustic challenges. Code and dataset are publicly available on the project page: https://audio2tool.github.io/.
Speech-to-speech models handle turn-taking naturally but offer limited support for tool-calling or complex reasoning, while production ASR-LLM-TTS voice pipelines offer these capabilities but rely on silence timeouts, which lead to unnatural turn-taking. We present DualTurn, which narrows this gap through generative pretraining on dual-channel conversational audio. The model generates both speakers' future audio autoregressively, implicitly learning conversational dynamics without any labels, and is then fine-tuned to predict interpretable turn-taking signals that map directly to agent actions. DualTurn monitors both channels continuously, anticipating turn boundaries and producing five agent actions. On standard benchmarks, DualTurn (0.5B) outperforms both VAP on agent action prediction (wF1 0.633 vs. 0.389) and a 3.1B audio-text model on word-level turn prediction (AUC 0.930 vs. 0.880), while anticipating turn boundaries earlier with fewer interruptions.