G-STAR: End-to-End Global Speaker-Tracking Attributed Recognition
Authors: Jing Peng, Ziyi Chen, Haoyu Li, Yucheng Wang, Duo Ma, Mengtian Li, Yunfan Du, Dezhu Xu, Kai Yu, Shuai Wang
Organizations: Shanghai Jiao Tong University · Shenzhen Research Institute of Big Data · Nanjing University · Central Media Technology Institute, Huawei · ETH Zürich
We study timestamped speaker-attributed automatic speech recognition (SA-ASR) for long-form, multi-party speech with overlap. In this setting, chunk-wise inference must preserve meeting-level speaker identity consistency while producing time-stamped, speaker-labeled transcripts. Prior Speech-LLM systems tend to prioritize either local diarization or global labeling, lacking the ability to jointly model fine-grained temporal boundaries and robust cross-chunk identity linking. We propose G-STAR, an end-to-end framework that couples a cache-conditioned speaker-tracking module with a Speech-LLM transcription backbone. The tracker provides structured speaker cues with temporal grounding, and the LLM generates attributed text conditioned on these cues. G-STAR supports component-wise optimization and joint end-to-end training, enabling flexible learning under heterogeneous supervision and domain shift. Under chunk-wise decoding protocols, experiments on both oracle-segmented local evaluation and full-meeting global evaluation show strong speaker-attributed transcription performance.
Figures & tables
Figure 1: Illustration of Streaming Sortformer with AOSC. The cache stores arrival-order speaker states across chunks and reuses them to keep speaker identities consistent when speakers reappear.
Figure 2: Overview of G-STAR Structure. The left panel illustrates the proposed G-STAR model architecture, which features a dual-branch encoder design comprising ASR and speaker diarization modules to support chunk-wise inference. The right panel outlines the three-stage training strategy for G-STAR, progressing sequentially from Speech-Language Pre-training to local SA-ASR (Cross-modal Alignment) and finally to global SA-ASR (End-to-End Fine-tuning).
Group
Component
AMI
Fisher
MLC
Candor
cpWER/DER ↓
cpWER/DER ↓
cpWER/DER ↓
cpWER/DER ↓
NVIDIA cascade
Sortformer (SD) ( Park et al., 2024 )
– / 29.87
– / 18.33
– / 17.76
– / 30.92
Parakeet (ASR) ( Wang et al., 2025 )
24.62 / –
27.73 / –
25.90 / –
27.44 / –
E2E SA-ASR
Vibevoice-ASR ( Peng et al., 2026 )
30.51 / 31.99
15.18 / 17.68
21.74 / 14.01
22.12 / 30.89
MOSS-Diarizen ( MOSI.AI et al., 2026 )
25.13 / 32.20
11.69 / 21.61
14.16 / 10.58
16.38 / 31.76
G-STAR
Ours
24.86 / 19.00
10.29 / 8.18
13.90 / 6.49
14.54 / 17.56
Table 1: Performance comparison in the local setting (audio up to 20s) with oracle VAD/segmentation, reported in cpWER/DER (%). The NVIDIA default cascade is shown as a two-row group: Sortformer for SD/DER and Parakeet for ASR/cpWER. DER is computed with collar 0. Bold indicates the best result.
Group
Component
Fisher
MLC
Candor
AMI
cpWER/DER ↓
cpWER/DER ↓
cpWER/DER ↓
cpWER/DER ↓
NVIDIA cascade
Sortformer (SD) ( Park et al., 2024 )
– / 15.21
– / 21.92
– / 18.03
– / 28.35
Parakeet (ASR) ( Wang et al., 2025 )
24.41 / –
31.03 / –
26.92 / –
35.70 / –
Speech-LLM
Vibevoice-ASR ( Peng et al., 2026 )
25.03 / 27.15
25.41 / 19.83
27.24 / 25.68
34.19 / 39.95
Controlled
Late-fusion cascade
21.01 / 23.41
23.18 / 21.38
17.62 / 17.67
39.52 / 37.63
G-STAR
Ours
16.44 / 16.85
17.15 / 14.25
15.17 / 24.89
30.85 / 32.23
Table 2: Global (meeting-level) performance comparison, reported in cpWER/DER (%). The NVIDIA default cascade is shown as separate SD and ASR components; the controlled cascade uses the same main components as G-STAR but applies late fusion. DER is computed with collar 0. Bold indicates the best result.
System
2 speakers
3–4 speakers
Avg.
cpWER/DER ↓
cpWER/DER ↓
cpWER/DER ↓
Vibevoice-ASR ( Peng et al., 2026 )
11.10 / 14.83
54.48 / 38.33
47.64 / 34.62
Cluster-based Pipeline
23.56 / 14.20
41.14 / 30.76
38.37 / 28.15
G-STAR
10.42 / 4.86
38.85 / 28.59
34.37 / 24.88
Table 3: Global SA-ASR performance on the internal test set, reported in cpWER/DER (%). DER is computed with collar 0.5.
Figure 3: Effect of the interleave ratio K on MLC SA-ASR performance in the local setting. Smaller K injects speaker cues more frequently, while larger K provides sparser speaker conditioning.
Hierarchical CE loss
Interleave Temporal Fusion
AMI
Fisher
Candor
cpWER/DER ↓
cpWER/DER ↓
cpWER/DER ↓
×
✓
26.33 / 21.06
10.88 / 10.24
14.97 / 20.21
✓
×
28.63 / 21.28
14.23 / 9.02
18.30 / 18.10
✓
✓
24.86 / 19.00
10.29 / 8.18
14.54 / 17.56
Table 4: Ablation study of model configuration under local setting.