cs.LGNov 13, 2026

Neural Classifiers as Error-Correcting Codes

Authors: Sergey Salishev, Anton Makarov, Oleg Granichin, Renata Avros, Zeev Volkovich

Organizations: St. Petersburg State University, Saint Petersburg, 199034, Russia · Department of Software Engineering, Braude College, 21982, Karmiel, Israel

Abstract

A unified framework is proposed that characterizes supervised deep classification as the decoding of error-correcting codes in the Shannon sense. Cross-entropy optimization is shown to align softmax classifiers with maximum-likelihood decoding, where logits map to Hamming distances to class-specific codewords. This correspondence explains the emergence of standard divergence objectives (e.g., Kullback-Leibler, Jeffreys) and provides a geometric basis for the stability of decision boundaries. Furthermore, the geometries of Neural Collapse and LeJEPA are identified as optimal constellations for decoding under isotropic noise. Unlike classical Error-Correcting Output Codes, deep networks are shown to learn effective codebooks in the feature space implicitly. This insight is leveraged to derive novel criteria for lossless quantization, improved knowledge distillation, and architectural regularization based on code redundancy. Finally, this perspective is extended to self-supervised encoders, offering a cohesive explanation for the geometry of latent representations in modern neural models.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Neural Collapse by Design: Learning Class Prototypes on the Hypersphere

    May 19, 2026Panagiotis Koromilas, Theodoros Giannakopoulos, Mihalis A. Nicolaou +1Prototype-Based ClassificationHyperspherical Representation Learning

  2. The role of class encoding in neural collapse

    May 29, 2026Bastien Massion, Roy Makhlouf, Estelle MassartNeural Representation GeometryMulticlass Classification