Transformer

Ask about this topic

Momentum

26 papers in the last four weeks, up 44% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 351

All topics
CardsList

  1. Cryptanalytic Extraction of Isolated Bias-Free GLU Feed-Forward Blocks by Antipodal Separation

    Aug 6, 2026Chunhui Shi, Xinwen FuTransformer FFNsModel Extraction Attacks

  2. Equivariant Music Transformer

    Aug 4, 2026Zixun Guo, Simon DixonRepresentation LearningTransformer

    Centre for Digital Music, Queen Mary University of London, United Kingdom

  3. Probing Character-level Transformers for the Spanish L-shaped Morphome

    Aug 4, 2026Akhilesh Kakolu Ramarao, Kevin Tang, Wiebke Petersen +1Transformer InterpretabilityTransformer

    Department of English Language and Linguistics, Heinrich Heine University Düsseldorf · Department of Linguistics, College of Liberal Arts and Sciences, University of Florida · Institute of Linguistics and Information Science, Heinrich Heine University Düsseldorf +1

  4. Provably Learning Multi-Head Attention with Queries

    Aug 4, 2026Sunyeop Kim, Insung Kim, Jian GuoSoftmax AttentionTransformer

    Korea University, Republic of Korea · Nanyang Technological University, Singapore

  5. Scaling an Autoregressive Transformer for Single-Cell Generation

    Aug 3, 2026Aleksandr Sharipov, Yusif Mukhtarov, Igor MolybogTransformerAutoregressive Generation

    HawAII, University of Hawai’i at Manoa

  6. Feed-Forward Steering in Transformer Residual Dynamics

    Aug 3, 2026Timur Mudarisov, Mikhail Burtsev, Radu StateTransformer InterpretabilityTransformer FFNs

    University of Luxembourg · 2London Institute of Mathematical Sciences

  7. Geometry-Guided Layerwise FFN Width Allocation in Transformers

    Aug 3, 2026Timur Mudarisov, Mikhail Burtsev, Radu StateTransformer FFNsTransformer

    University of Luxembourg · London Institute of Mathematical Sciences

  8. Riemannian Attention Mechanisms for Transformers: A Theoretical Framework and Architecture Design

    Aug 2, 2026Sen SongTransformerSelf-Attention

    Independent Researcher, Guangzhou, China

  9. Transcript-Managed Transformers: Monotone Multi-Agent Collapse and Universality with Two Pop-Enabled Transcripts

    Jul 31, 2026Sergey SalishevTransformer ExpressivityTransformer

    AI Foundry San Francisco, CA, USA

  10. Learning to Trace Seiberg Dualities

    Jul 30, 2026Jonathan J. Heckman, Shani Meynet, Alessandro Mininno +1TransformerScientific ML

    Department of Physics and Astronomy, University of Pennsylvania, 209 South 33rd Street, Philadelphia, PA 19104, USA · Department of Physics, University of Wisconsin–Madison, 1150 University Avenue, Madison, WI 53706, USA

  11. MUL-T: Decoding Spatial Cellular Architecture in Multiplexed Tissue Images

    Jul 30, 2026Farzaneh Seyedshahi, Kai Rakovic, Adalberto Claudio Quiros +2TransformerGenerative Modeling

    Cancer Research UK Scotland Institute, Glasgow, UK · University of Glasgow, Glasgow, UK

  12. Generalization Bounds on Optimal Control for Transformer Training and Wasserstein Distributional Robustness

    Jul 30, 2026Kağan Akman, Naci Saldi, Serdar YükselTransformerWasserstein Distributionally Robust Optimization

    Department of Mathematics, Bilkent University, 06800, Ankara, Turkey · Department of Mathematics and Statistics, Queen’s University, Kingston, K7L 3N8, Ontario, Canada

  13. A Compositional Theory of Causally Masked Transformers

    Jul 29, 2026Franz Nowak, Ryan Cotterell, Reda BoumasmoudTransformer ExpressivityTransformer

    1ETH Zürich

  14. Placement Is Free, Composition Is Not: The Latin Square as a Provably-Balanced Construction for Heterogeneous Sequence-Mixer Stacks

    Jul 29, 2026Taebong Kim, Youngsik Hong, Minsik Kim +3Transformer

    VIDRAFT AI Research · QuantumOS, Seoul, Republic of Korea

  15. Localized Adaptation Reveals Distinct Learning Signatures in Transformers

    Jul 28, 2026Rebecca Ramnauth, Brian ScassellatiFine-TuningTransformer

    Yale University

  16. Physics Transformer: Tailoring Transformer for General PDE Prediction

    Jul 27, 2026Guoze Sun, Rui Zhang, Jiankai Tang +4TransformerPDE Surrogate Modeling

    Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China · School of Mechanics and Engineering Science, Peking University, Beijing, China

  17. SpecFormer: Mitigating Embedding and Attention Collapse via Spectral-Aware Transformer for Recommendation

    Jul 27, 2026Yu Cui, Yi Xu, Jiahao Wang +6Representation CollapseTransformer

    Zhejiang University Hangzhou, China · Alibaba Group Beijing, China

  18. Multimodal Auto-regressive Transformer Surrogate for Modeling Variable Operations and Quantifying Uncertainty in Geological Carbon Storage

    Jul 26, 2026Yifu Han, Louis J. DurlofskyNeural Surrogate ModelingUncertainty Quantification

    Department of Energy Science and Engineering, Stanford University, Stanford, CA, 94305, USA

  19. BERT-based Models vs. Large Language Models for Low-Resource Named Entity Recognition: A Comparative Study on Marathi

    Jul 25, 2026Hariom Ingle, Ronit Ghode, Ishwari Gondkar +2Named Entity RecognitionLLM Evaluation

    Department of Information Technology, PICT, Pune, India · L3Cube Labs, Pune, India · Indian Institute of Technology Madras, Chennai, India

  20. CAPT: A Multi-task Continuous Autoregressive Transformer enabling Cross-dataset and Cross-species Transfer for Calcium Population Dynamics

    Jul 25, 2026Xinhong Xu, Yimeng Zhang, Yuanlong ZhangTransformerNeural Decoding

    School of Life Sciences Tsinghua University Beijing, China

  21. Structure over Depth: A Single-Block Spatio-Temporal Transformer for Multi-Entity Reasoning

    Jul 25, 2026Narthana Sivalingam, Santhirarajah Sivasthigan, Buddhi Wijenayake +3Spatiotemporal ReasoningHuman Activity Recognition

    Department of Electrical and Electronic Engineering University of Peradeniya Peradeniya, Sri Lanka

  22. Sensitivity Analysis of GRU, LSTM and Transformer Encoder in Classification of Automated Driving Systems

    Jul 25, 2026Bidhya Shrestha, Christos PapadopoulosTransformerTime Series Classification

    University of Memphis

  23. Hidden Boundary Motion in Transformer Optimization: Function-Space Orthogonalization of Affine Weight and Bias Updates

    Jul 24, 2026Zhang Gongyue, Sheng Yixuan, Liu donghan +3TransformerNeural Network Optimization

    arXiv:2607.22927v1

  24. Interior interpretability with attention rollout: contraction and propagation profiles in Transformers

    Jul 24, 2026Umberto Biccari, Qian Huang, Enrique ZuazuaTransformer InterpretabilityTransformer

  25. Multimodal Surface EMG Hand Gesture Recognition Using Query-Based Transformers for Prosthetic Control

    Jul 24, 2026Federico Del Pup, Elisa Tentori, Manfredo AtzoriEMG-Based Gesture RecognitionHybrid CNN-Transformer Architectures

    Department of Information Engineering, University of Padua, Padua, 35131, Italy · Padova Neuroscience Center, University of Padua, Padua, 35129, Italy · Department of Biomedical Sciences, University of Padua, Padua, 35131, Italy +2

  26. Hierarchical Grading in Large Language Models

    Jul 23, 2026T. ShaskaTransformerLLM Training

  27. Multi-modal transformer for signal classification in nanopore blockade experiments

    Jul 22, 2026Sandro Kuppel, Julian Hoßbach, Samuel Tovey +1TransformerTime Series Classification

    Institute for Computational Physics University of Stuttgart Allmandring 3, 70569 Stuttgart, Germany

  28. The Giant Hippocampus: From Structural Monoculture to a System of Systems

    Jul 22, 2026Jaeho SeolTransformerInductive Bias

    Independent Researcher

  29. Multi-Head Attention Residuals

    Jul 22, 2026Cheng Luo, Zefan Cai, Junjie HuTransformerAttention Residual Connections

    Independent Researcher · University of Wisconsin–Madison

  30. Elicitation without Backpropagation: Steering Model Behavior by Optimizing the Latent Posterior

    Jul 21, 2026Garrett Baker, Vinayak Pathak, Daniel Murfet +1Prompt LearningTransformer

    Timaeus · CORAL · Monash University