Coalgebras for categorical deep learning: Representability and universal approximation

Dragan Mašulović

cs.LGMar 3, 2026

Coalgebras for categorical deep learning: Representability and universal approximation

Authors: Dragan Mašulović

Organizations: University of Novi Sad, Faculty of Sciences Department of Mathematics and Informatics Trg Dositeja Obradovića 3, 21000 Novi Sad, Serbia

Abstract

Categorical deep learning (CDL) has recently emerged as a framework that leverages category theory to unify diverse neural architectures. While geometric deep learning (GDL) is grounded in the specific context of invariants of group actions, CDL aims to provide domain-independent abstractions for reasoning about models and their properties. In this paper, we contribute to this program by developing a coalgebraic foundation for equivariant representation in deep learning, as classical notions of group actions and equivariant maps are naturally generalized by the coalgebraic formalism. Our first main result demonstrates that, given an embedding of data sets formalized as a functor from SET to VECT, and given a notion of invariant behavior on data sets modeled by an endofunctor on SET, there is a corresponding endofunctor on VECT that is compatible with the embedding in the sense that this lifted functor recovers the analogous notion of invariant behavior on the embedded data. Building on this foundation, we then establish a universal approximation theorem for equivariant maps in this generalized setting. We show that continuous equivariant functions can be approximated within our coalgebraic framework for a broad class of symmetries. This work thus provides a categorical bridge between the abstract specification of invariant behavior and its concrete realization in neural architectures.

Explore similar work

Feb 23, 2026

Quantitative Approximation Rates for Group Equivariant Learning

The universal approximation theorem establishes that neural networks can approximate any continuous function on a compact set. Later works in approximation theory provide quantitative approximation rates for ReLU networks on the class of

α

-Hölder functions

f: [0,1]^N \to \mathbb{R}

. The goal of this paper is to provide similar quantitative approximation results in the context of group equivariant learning, where the learned

α

-Hölder function is known to obey certain group symmetries. While there has been much interest in the literature in understanding the universal approximation properties of equivariant models, very few quantitative approximation results are known for equivariant models. In this paper, we bridge this gap by deriving quantitative approximation rates for several prominent group-equivariant and invariant architectures. The architectures that we consider include: the permutation-invariant Deep Sets architecture; the permutation-equivariant Sumformer and Transformer architectures; joint invariance to permutations and rigid motions using invariant networks based on frame averaging; and general bi-Lipschitz invariant models. Overall, we show that equally-sized ReLU MLPs and equivariant architectures are equally expressive over equivariant functions. Thus, hard-coding equivariance does not result in a loss of expressivity or approximation power in these models.

Jonathan W. Siegel, Snir Hordan, Hannah Lawrence +2

Department of Mathematics, Texas A&M University, College Station, TX, 77840, USA · Faculty of Mathematics, Technion- Israel Institute of Technology, Haifa, 3200003, Israel · Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 02139 USA

Equivariant Neural Networks Neural Network Approximation Theory Permutation-Invariant Learning

Ask PDF

Jul 4, 2026

Foundations of Equivariant Deep Learning: Unifying Graph and Sheaf Neural Networks

Symmetry is everywhere in nature and society. Geometric deep learning exploits symmetries in data to improve the performance and efficiency of deep learning systems. In this paper, we extend geometric deep learning to utilize richer symmetry structures. Specifically, we develop order-equivariant neural networks (OENN), which generalize standard graph message passing and sheaf neural networks via the theory of equivariant bundles over face posets (face categories). We (i) characterize all linear order-equivariant maps, (ii) build OENN layers, and (iii) prove universal approximation theorems (UATs) for continuous order-equivariant maps, which are new results even when restricted to sheaf neural networks (for which no UAT was known before). We illustrate the framework on graph and sheaf models. Our results can also be seen as extending the known UAT for graph neural networks to a more general setting that subsumes sheaf neural networks as well. In addition, we show that OENN can be extended further to CENN, Category-Equivariant Neural Network, which gives the general form of equivariant neural networks as well as of equivariant universal approximation theorems, allowing us to leverage categorical symmetry in data (e.g., non-invertible symmetries on multiple objects with compositional relations on those symmetries).

Yoshihiro Maruyama

Department of Mathematical and Information Sciences, Kyoto University, Kyoto, Japan.

Graph Neural Networks Equivariant Neural Networks Sheaf Neural Networks Neural Network Approximation Theory Equivariant Representation Learning

Ask PDF

Apr 8, 2026

Weaves, Wires, and Morphisms: Formalizing and Implementing the Algebra of Deep Learning

Despite deep learning models running well-defined mathematical functions, we lack a formal mathematical framework for describing model architectures. Ad-hoc notation, diagrams, and pseudocode poorly handle nonlinear broadcasting and the relationship between individual components and composed models. This paper introduces a categorical framework for deep learning models that formalizes broadcasting through the novel axis-stride and array-broadcasted categories. This allows the mathematical function underlying architectures to be precisely expressed and manipulated in a compositional manner. These mathematical definitions are translated into human manageable diagrams and machine manageable data structures. We provide a mirrored implementation in Python (pyncd) and TypeScript (tsncd) to show the universal aspect of our framework, along with features including algebraic construction, graph conversion, PyTorch compilation and diagram rendering. This lays the foundation for a systematic, formal approach to deep learning model design and analysis.

Vincent Abbott, Gioele Zardini

Laboratory for Information and Decision Systems, Massachusetts Institute of Technology

Topic pending

Ask PDF