cs.AINov 13, 2026

The Transformer Paradigm: Dynamic Processing and a New Principle of Inference

Authors: Marco Giunti, Fabrizia Giulia Garavaglia

Organizations: ALOPHIS – Applied Logic Philosophy and History of Science, University of Cagliari · Dipartimento di Pedagogia, Psicologia, Filosofia, University of Cagliari

Abstract

We argue that the Transformer revolution is not adequately understood as a merely quantitative expansion of deep learning, but as the emergence of a new paradigm based on a different principle of inference. We interpret the Transformer as a system in which token representations are transformed by parameterized transformations, or transforming concepts. During inference, some of these transformations are dynamically constructed from the current input: previously acquired transformations are first adapted and then applied. At the mechanical-structural level, this dynamic processing is realized through output-weight interconnections, whereby outputs of some networks determine the weights of others. The resulting mode of processing, which we call Sequence-level Interactive Dynamic Parallel Processing (SIDPP), provides a framework for explaining strong prompt sensitivity and opens research directions concerning interpretability, control, alternative AI architectures, neuromorphic systems, and possible biological realizations of dynamic processing.

Explore similar work

CardsList
  1. The Transformer Revolution, Part 1: Dynamic Processing through Output-Weight Interconnections

    Aug 4, 2026Marco Giunti, Fabrizia Giulia GaravagliaTransformer InferencePrompt Sensitivity

  2. Pattern Formation in Transformers

    Sep 29, 2026Erkan Turan, Gaspard Abel, Maks OvsjanikovTransformerTransformer Attention

  3. Invariant Learning Dynamics of Transformers in Inductive Reasoning Tasks

    Jul 13, 2026Tiberiu Musat, Tiago Pimentel, Nicolas Zucchet +1Transformer Training DynamicsIn-Context Learning