Speech self-supervised learning aims to learn general-purpose representations for downstream speech tasks. However, current approaches rely on complex, carefully designed prediction targets. We challenge this necessity with GLaS-JEPA, a framework that directly predicts the current encoder's continuous representations at masked positions, without contrastive learning, discrete targets, or separate EMA target encoders. We prevent representation collapse using SIGReg representation-space regularization, eliminating the need for engineered target-generation mechanisms. Pretrained on 960 hours of LibriSpeech, our 57M-parameter model achieves a 6.89% WER on frozen-encoder SUPERB ASR and a 25.87% CER on slot filling, outperforming the best non-distilled sub-90M baselines by 43.1% and 22.0%, respectively. These results demonstrate that highly competitive speech representations can emerge from a radically simplified training recipe.
Figures & tables
Figure 1: Frozen SUPERB ASR versus parameter count. All plotted models are pretrained on LibriSpeech 960 h, except S-JEPA, pretrained on 83k h [ 27 , 16 ] . Dashed line: data2vec 2.0 WER (4.81%) [ 28 ] .
Figure 2: GLaS-JEPA: shared encoder and token-wise projector pα . MSE uses masked predictions and stop-gradient targets; SIGReg backpropagates through the full view. The projector is discarded downstream.
Figure 3: Time-conditional (left) and Shuffled Marginal (right), with SUPERB results for the 30M model. Colors/letters: utterances; subscripts: time indices. Outlines mark groups of B tokens.
Figure 4: Phone and speaker linear-probe test accuracy versus normalized encoder depth for GLaS-JEPA and WavLM Base.