cs.CLJan 11, 2026

Categorize Early, Integrate Late: Divergent Processing Strategies in Automatic Speech Recognition

Authors: Nathan RollPranav BhaleraoMartijn BarteldsArjun PawarYuka TatsumiTolulope OgunremiChen ShaniCalbert Graham+2 more

Organizations: Stanford University

Abstract

In speech language modeling, two architectures dominate the frontier: the Transformer and the Conformer. However, it remains unknown whether their comparable performance stems from convergent processing strategies or distinct architectural inductive biases. We introduce Architectural Fingerprinting, a probing framework that isolates the effect of architecture on representation, and apply it to a controlled suite of 24 pre-trained encoders (39M-3.3B parameters). Our analysis reveals divergent hierarchies: Conformers implement a "Categorize Early" strategy, resolving phoneme categories 29% earlier in depth and speaker gender by 16% depth. In contrast, Transformers "Integrate Late," deferring phoneme, accent, and duration encoding to deep layers (49-57%). These fingerprints suggest design heuristics: Conformers' front-loaded categorization may benefit low-latency streaming, while Transformers' deep integration may favor tasks requiring rich context and cross-utterance normalization.

Explore similar work

CardsList
  1. Design of the IBM Granite 5.0 TurboCTC ASR Model

    Sep 17, 2026Brian Kingsbury, George Saon, Masayuki Suzuki +5Muon Optimizer

  2. Contrastive Regularization for Accent-Robust ASR

    May 5, 2026Van-Phat Thai, Aradhya Dhruv, Duc-Thinh Pham +1Accented SpeechContrastive Learning