GTCRN-SS: Lightweight Causal Speech Separation with Residual Allocation and Reconstruction Refinement
Organizations: Dept. of Electrical Engineering, National Chi Nan University · Inst. of Information Science, Academia Sinica · Grad. Inst. of AI Interdisciplinary Applied Technology, National Taiwan Normal University
Abstract
Low-complexity causal speech separation must preserve speaker information and allocate residual mixture energy without relying on future feature frames. We present GTCRN-SS, a two-stage time-frequency separator that adapts the compact GTCRN enhancement backbone to fixed-output speech separation. The model combines internal residual allocation, reconstruction-conditioned refinement, and frame-aligned recurrent-state transfer. Its complex-mask estimator assigns non-target content to an unreturned residual branch, while the complementary speech estimate conditions a second stage whose final masks remain anchored to the original mixture. A lightweight low-rank adapter transfers causal recurrent evidence between recursive executions without adding another recurrent memory network. The neural graph excludes future feature frames and incurs only the fixed analysis look-ahead of the centered short-time Fourier transform. On noisy Libri2Mix, GTCRN-SS achieves 10.1 dB SDRi and 9.5 dB SI-SNRi with fewer than one million parameters and a low computational footprint. Independently trained ablations confirm the benefits of recurrent-state transfer and reconstruction conditioning, and corpus-level diagnostics show that the internal branch primarily captures residual rather than target-speech content. These results establish GTCRN-SS as an effective quality-efficiency operating point for causal monaural speech separation.
Figures & tables
| Method | SDRi (dB) | SI-SNRi (dB) | Params (M) | MACs (G) |
|---|---|---|---|---|
| DPRNN | 9.6 | 9.0 | 1.3 | 11.2 |
| SkiM | 9.9 | 9.2 | 3.6 | 11.1 |
| ReSepFormer | 4.1 | 3.3 | 8.0 | 6.3 |
| TF-GridNet-Causal | 11.5 | 10.8 | 11.4 | 179.2 |
| TFACM (Small) | 10.2 | 9.3 | 0.5 | 19.4 |
| GTCRN-SS (ours) | 10.1 | 9.5 | 0.9 | 6.1 |
| Information and allocation | Performance (dB) | |||||
|---|---|---|---|---|---|---|
| ID | Configuration | CE input | Stage-2 cue | Sink | SI-SNRi | SDRi |
| A | Full model | state | speech sum | yes | 9.5 | 10.1 |
| B | Without CE information | zero | speech sum | yes | 8.1 | 8.8 |
| C | Without reconstruction cue | state | zero | yes | 7.7 | 8.4 |
| D † | -branch exact consistency | state | mixture | no | 6.1 | 6.4 |
| Metric | Estimate | 95% interval |
|---|---|---|
| Residual correlation | 0.843 | [0.840, 0.845] |
| Max. speech correlation | 0.139 | [0.137, 0.140] |
| Correlation advantage | 0.704 | [0.700, 0.708] |
| Residual-preferred utterances | 99.9% | — |
Speech Processing
Speech processing develops computational methods for recognizing, synthesizing, analyzing, and transforming spoken audio.
See all 420 papers