Transformer

Ask about this topic

Momentum

26 papers in the last four weeks, up 44% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 351

All topics
CardsList

  1. Robustness of Transformer-Based Fluence Map Prediction Under Clinically Realistic Perturbations

    Apr 28, 2026Ujunwa Mgboh, Rafi Ibn Sultan, Joshua Kim +2Distribution Shift RobustnessTransformer

    Wayne State University, Detroit, MI, USA · Henry Ford Health, Detroit, MI

  2. Investigation into In-Context Learning Capabilities of Transformers

    Apr 28, 2026Rushil Chandrupatla, Leo Bangayan, Sebastian LengNeural Network GeneralizationTransformer

  3. StratFormer: Adaptive Opponent Modeling and Exploitation in Imperfect-Information Games

    Apr 28, 2026Andy Caen, Mark H. M. Winands, Dennis J. N. J. SoemersOpponent ModelingGame-Playing Agents

    Department of Advanced Computing Sciences, Maastricht University, Maastricht, The Netherlands

  4. GEGLU-Transformer for IMU-to-EMG Estimation with Few-Shot Adaptation

    Apr 28, 2026Miroljub Mihailovic, Luca Tonin, Stefano Tortora +1TransformerWearable Sensing

    Department of Information Engineering, University of Padua, 35131 Padua, Italy

  5. Scaling Probabilistic Transformer via Efficient Cross-Scale Hyperparameter Transfer

    Apr 28, 2026Penghao Kuang, Haoyi Wu, Kewei TuTransformerNeural Network Optimization

    School of Information Science and Technology, ShanghaiTech University

  6. Training Transformers as a Universal Computer

    Apr 28, 2026Ruize Xu, Chenxiao Yang, Yanhong Li +1TransformerAlgorithmic Reasoning

    The University of Chicago · Toyota Technological Institute at Chicago · Allen Institute for AI

  7. ADE: Adaptive Dictionary Embeddings -- Scaling Multi-Anchor Representations to Large Language Models

    Apr 27, 2026Orhan Demirci, Sezer Aptourachman, Aydın KayaText EmbeddingsWord Embeddings

    Department of Computer Engineering Hacettepe University Ankara, Turkey

  8. A systematic literature Review for Transformer-based Software Vulnerability detection

    Apr 27, 2026Fiza Naseer, Javed Ali Khan, Muhammad Yaqoob +2TransformerSoftware Vulnerability Detection

  9. Graph Memory Transformer (GMT)

    Apr 26, 2026Nicola Zanarini, Niccolò Ferrari, Evelina LammaMemory-Augmented Neural NetworksTransformer FFNs

    Bonfiglioli Engineering s.r.l. · Department of Engineering, University of Ferrara, Via Saragat 1, 44122 Ferrara, Italy · NAIS s.r.l., Via Maria Callas 4, 40131 Bologna, Italy

  10. Transformer as an Euler Discretization of Score-based Variational Flow

    Apr 26, 2026Huadong LiaoRepresentation LearningTransformer

  11. Adaptive Head Budgeting for Efficient Multi-Head Attention

    Apr 24, 2026Bilal Faye, Abdoulaye Mbaye, Hanane Azzag +1TransformerTransformer Attention

  12. VecHeart: Holistic Four-Chamber Cardiac Anatomy Modeling via Hybrid VecSets

    Apr 21, 2026Yihong Chen, Pascal FuaTransformer3D Reconstruction

    CVLAB, EPFL

  13. Neural posterior estimation of the neutrino direction in IceCube using transformer-encoded normalizing flows on the sphere

    Apr 21, 2026R. Abbasi, M. Ackermann, J. Adams +411Normalizing FlowsTransformer

  14. Nexusformer: Nonlinear Attention Expansion for Stable and Inheritable Transformer Scaling

    Apr 21, 2026Weijie Zhao, Mingquan Liu, Bolun Wang +4TransformerLanguage Model Scaling Laws

    Luxi Tech, Shenzhen, China · The School of Computer Science, China University of Geosciences, 388 Lumo Road, Wuhan, 430074, Hubei, China.

  15. One-Block Transformer (1BT) for EEG-Based Cognitive Workload Assessment

    Apr 21, 2026Stefanos Gkikas, Christian Arzate Cruz, Thomas Kassiotis +3Efficient Neural Network InferenceTransformer

    Honda Research Institute Japan · Wako City, Japan · Department of Electronic Engineering +6

  16. Towards a Foundation-Model Paradigm for Aerodynamic Prediction in Three-dimensional Design

    Apr 20, 2026Yunjia Yang, Babak Gholami, Caglar Gurbuz +2Surrogate ModelingTransformer

    Technical University of Munich, Garching, D-85748, Germany · BMW Group, Bremer Str. 6, Munich, D-80788, Germany

  17. In-Context Learning Under Regime Change

    Apr 18, 2026Carson Dudley, Yutong Bi, Xiaofeng Liu +1TransformerDistribution Shift Detection

    Department of Mathematics, University of Michigan · Department of Electrical Engineering and Computer Science, University of Michigan · Michigan Institute for AI and Data in Society, University of Michigan

  18. Adaptive Computation Depth via Learned Token Routing in Transformers

    Apr 18, 2026Ahmed Abdelmuniem Abdalla MohammedEfficient Transformer InferenceTransformer

    Independent Researcher

  19. What Is the Minimum Architecture for Prolepsis? Early Irrevocable Commitment Across Tasks in Small Transformers

    Apr 16, 2026Éric JacopinTransformer InterpretabilityAttention Head Analysis

    Cosmic AI

  20. Expressivity of Transformers: A Tropical Geometry Perspective

    Apr 16, 2026Ye Su, Yong LiuTransformer ExpressivityTransformer

    Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China · University of Chinese Academy of Sciences, Beijing 101407, China · Gaoling School of Artificial Intelligence, Renmin University of China, Beijing 100872, China

  21. Crystalite: A Lightweight Transformer for Efficient Crystal Modeling

    Apr 2, 2026Tin Hadži Veljković, Joshua Rosenthal, Ivor Lončarić +1Diffusion TransformerTransformer

    1UvA-Bosch Delta Lab · University of Amsterdam · 3Ruđer Bošković Institute

  22. A Benchmark of State-Space Models vs. Transformers and BiLSTM-based Models for Historical Newspaper OCR

    Apr 1, 2026Merveilles Agbeti-Messan, Pierrick Tranouez, Stéphane Nicolas +2Selective SSMsTransformer

    LITIS EA4108, University of Rouen Normandy, France · LITIS EA4108, INSA of Rouen Normandy, France

  23. ExFusion: Efficient Transformer Training via Multi-Experts Fusion

    Mar 30, 2026Jiacheng Ruan, Daize Dong, Xiaoye Qu +5TransformerMixture of Experts

    School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, No 800, Dongchuan Road, Minhang District, Shanghai, China, 200240 · Shanghai Artificial Intelligence Laboratory, Shanghai, China, 200240 · Huazhong University of Science and Technology, Hubei, China, 430074 +3

  24. On the Ability of Transformers to Verify Plans

    Mar 20, 2026Yash Sarrof, Yupei Du, Katharina Stein +3TransformerSymbolic Planning

    Saarland University · Australian National University · University of Toulouse / LAAS-CNRS

  25. The Phasor Transformer: Resolving Attention Bottlenecks on the Unit Circle

    Mar 18, 2026Dibakar SigdelTransformerTime Series Forecasting

    Mindverse Computing LLC, WA 98087

  26. Measuring the Redundancy of Decoder Layers in SpeechLLMs

    Mar 5, 2026Adel Moumen, Guangzhi Sun, Philip C WoodlandLLM PruningTransformer

    Department of Engineering, University of Cambridge, UK

  27. Retrievit: In-context Retrieval Capabilities of Transformers, State Space Models, and Hybrid Architectures

    Mar 3, 2026Georgios Pantazopoulos, Malvina Nikandrou, Ioannis Konstas +1In-Context RetrievalTransformer

    University of Edinburgh · Heriot-Watt University

  28. MetaOthello: A Controlled Study of Multiple World Models in Transformers

    Feb 26, 2026Aviral Chawla, Galen Hall, Juniper LovatoTransformer InterpretabilityWorld Model Learning

    Vermont Complex Systems Institute, University of Vermont, Burlington, VT · University of Michigan, Ann Arbor, MI

  29. Routing Absorption in Sparse Attention: Why Random Gates Are Hard to Beat

    Feb 11, 2026Keston Aquino-MichaelsTransformerSelf-Attention

    No Way Labs

  30. Revisiting the Shape Convention of Transformer Language Models

    Feb 6, 2026Feng-Ting Liao, Guan-Ting Yi, Tzu-Quan Lin +2Efficient Transformer InferenceTransformer FFNs

    MediaTek Research · National Taiwan University