Transformer

Ask about this topic

Momentum

26 papers in the last four weeks, up 44% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 351

All topics
CardsList

  1. Enhancing Transformer Representations of Symbolic ODE Expressions

    Sep 21, 2026Xiyue Fan, Adam Prugel-Bennett, Stuart E. MiddletonTransformerPositional Encoding

    University of Southampton

  2. Design of the IBM Granite 5.0 TurboCTC ASR Model

    Sep 17, 2026Brian Kingsbury, George Saon, Masayuki Suzuki +5TransformerAutomatic Speech Recognition

    IBM Research

  3. TIO-Former: Ultra-Lightweight 6-Directional ToF-Inertial Odometry for Nano-UAVs via a Streaming Causal Transformer

    Sep 15, 2026Yang Liu, Yifan He, Wenhao Zhao +9Inertial OdometryTransformer

    Zhejiang University, Hangzhou, China · Differential Robotics, Hangzhou, China

  4. Investigating Temporal Motion Features for Pose-to-Text Indian Sign Language Translation

    Sep 14, 2026Manav Dhamecha, Praveen Kumar Chandaliya, Pruthwik MishraSign Language TranslationTransformer

    Sardar Vallabhbhai National Institute of Technology, Surat, India

  5. Type Diversity Enables Transformers to Generalise Compositionally

    Sep 14, 2026Anssi Moisio, Mathias Creutz, Mikko KurimoTransformerSemantic Parsing

    Aalto University · University of Helsinki

  6. A New Transformer-Based Approach for Audio-Based Kinship Verification and a New Uncontrolled Mandarin Kinship Speech Dataset

    Sep 12, 2026Qiyang Sun, Langqing Zhang, Yupei Li +1Transformer

    Department of Computing Imperial College London London, United Kingdom · Chair of Health Informatics TUM University Hospital Munich, Germany

  7. "Transforming" LHCb: self-supervised maps of heavy-flavour decays

    Sep 8, 2026Marko Stamenkovic, Greg LandsbergTransformerSelf-Supervised Learning

    Department of Physics, Brown University, Providence, Rhode Island 02912, U.S.A.

  8. Disentangled Global-Local Feature Learning with E-Branchformer for Audio Deepfake Detection

    Sep 8, 2026Phuong Tuan Dat, Ho Bao Thu, Nguyen Tran Trung +2Audio Deepfake DetectionTransformer

    Department of Electrical and Computer Engineering, National University of Singapore · School of Communication and Information Technology, Hanoi University of Science and Technology

  9. Length Generalization for Transformers via Compression

    Sep 8, 2026Georg Zetzsche, Hongjian Jiang, Andy Yang +4Transformer ExpressivityTransformer

    Max Planck Institute for Software Systems (MPI-SWS) · RPTU Kaiserslautern-Landau · University of Notre Dame

  10. Global Divergence, Local Convergence: Representation Geometry in SSMs and Transformers

    Sep 8, 2026Amit Ben-Artzy, Roy SchwartzTransformerRepresentation Geometry in Language Models

    Computer Science, The Hebrew University of Jerusalem

  11. A Transformer-Based Delta Expression Encoder for Psilocybin Transcriptional Response: Architecture, Representations, and Biological Validation

    Sep 8, 2026Sai JayakumarTransformerSingle-Cell Perturbation Modeling

    Stanford School of Medicine, Biomedical Data Science Program, Stanford, CA

  12. Graph Machine: Towards Better Pretraining via Edges

    Sep 2, 2026Lintai HouTransformerAutoregressive Language Modeling

    Iter Labs

  13. oHC: Orthogonal Hyper-Connections on SO(4) via Quaternions

    Sep 2, 2026Haoqiang Guo, Xuyi Chen, Bo Ke +5Hyper-ConnectionsTransformer

    The Hong Kong University of Science and Technology · Baidu Inc.

  14. AGI Maze Prediction Datasets: A Compact Benchmark for Learning World Dynamics with Transformers

    Sep 2, 2026Alexey PotapovWorld ModelsAction-Conditioned World Models

    SingularityNET Foundation

  15. If It Moves, Radar Knows: A Physics-Aware Radar Transformer for Class-Agnostic Moving-Object Detection

    Sep 2, 2026Yinghao Sun, Shuguang Li, Jinliang Shao +1Object DetectionAutonomous Driving

    School of Automation Engineering of the University of Electronic Science and Technology of China (UESTC), Chengdu, Sichuan, P. R. China

  16. OR-Transformer: Scaling Real-Time Decision-Making to 1,000 Items

    Sep 1, 2026Shuze Daniel Liu, David Simchi-Levi, Claire Chen +2Inventory ControlTransformer

    Massachusetts Institute of Technology · Purdue University · California Institute of Technology +1

  17. One-Layer Transformer Provably Learns Multiclass One-Nearest Neighbor in Context

    Sep 1, 2026Skanda Athreya, Yutong WangTransformerIn-Context Learning

    James B. Conant High School · Hoffman Estates, IL · Illinois Institute of Technology +1

  18. Measuring Optimal Transport in Transformer Depth

    Sep 1, 2026Alexandre QuemyTransformerNeural Network Training Dynamics

    Hother Labs

  19. Can LLMs Use Relational Transformer Embeddings?

    Aug 31, 2026Francisco Galuppo Azevedo, Clarissa Lima LouresTransformerLLM Fine-Tuning

    Kunumi Institute, Brazil · Universidade Federal de Minas Gerais, Belo Horizonte, Minas Gerais, Brazil.

  20. Transformer-Based Flow Shop Scheduling Using MILP-Generated Training Data

    Aug 30, 2026Roderich WallrathTransformerCombinatorial Optimization

    University of Twente, Faculty of Science and Technology, Sustainable Process Technology, Process Design and Optimization, Drienerlolaan 5, 7522 NB Enschede, The Netherlands

  21. TANGO: Treating Tokens as Operators

    Aug 22, 2026Joshua NunleyTransformer FFNsTransformer

    Department of Informatics Luddy School of Informatics, Computing, and Engineering Cognitive Science Program Indiana University Bloomington

  22. Algebraic Decomposition Theory for Transformer Length Generalization

    Aug 13, 2026Andy Yang, Blerta Veseli, Corentin Barloy +5TransformerLength Generalization

    University of Notre Dame · 2Saarland University · 3Ruhr University Bochum +4

  23. Rethinking Normalization Placement for LLMs: Post-Norm under Curriculum Depth Growing

    Aug 13, 2026Sheng Ren, Yadong Wang, Naiqiang Tan +7TransformerLayer Normalization

    Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China · Didichuxing Co. Ltd, Beijing, China

  24. Paths: Prompt-aware Spatio-temporal Transformer with Hierarchical Multi-modal Fusion for RGB-Event Video Person Re-Identification

    Aug 13, 2026Yakun Huo, Yingquan Wang, Yangyang Liu +4Cross-Modal Representation LearningTransformer

    Dalian University of Technology Dalian, China

  25. Geometric and Behavioral Stratification in Transformer Residual Streams

    Aug 12, 2026Nelson GudaTransformer InterpretabilityTransformer

  26. Cracks in the Foundation: Seemingly Minor Architectural Choices Impact Long Context Extension

    Aug 10, 2026Amanda Bertsch, Luca Soldaini, Matthew R. Gormley +4TransformerLong-Context Language Modeling

    Ai2 · Carnegie Mellon University · University of Washington

  27. Training-Free Universal Approximation by Prompting Random Transformers

    Aug 10, 2026Alexander Hsu, Rongjie LaiTransformer ExpressivityPrompt Learning

    Department of Mathematics, Purdue University

  28. MixFormer: Linear Transformer with Mixture of Memory Experts

    Aug 10, 2026Yu Guo, Lei DuanMemory-Augmented Neural NetworksTransformer

    School of Computer Science, Sichuan University Chengdu, China · School of Artificial Intelligence, Sichuan University Chengdu, China

  29. Why Post-Norm Transformers Collapse: Attention Amplification and Gradient Repair Failure

    Aug 10, 2026Xingjian Wang, Qingyu Han, Xiaodong Luo +1TransformerSelf-Attention

    The Chinese University of Hong Kong, Shenzhen · Shenzhen Research Institute of Big Data

  30. Post-Grokking Collapse at the Representation-Readout Interface in Muon-Trained Transformers

    Aug 7, 2026Ali Janati, Kaoutar El Maghraoui, Anass BelfatmiTransformerMuon Optimizer

    Data Science Institute, Columbia University, New York, NY, USA · Department of Computer Science, Columbia University, New York, NY, USA · CentraleSupélec, Gif-sur-Yvette, France