Transformer

Ask about this topic

Momentum

26 papers in the last four weeks, up 44% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 351

All topics
CardsList

  1. The Energy Consumption of Transformer Fine-Tuning: A Roofline-Inspired Scaling Model

    Jun 22, 2026Mansour Zoubeirou a MayakiEnergy-Efficient MLTransformer

    Université Lumière Lyon 2, CNRS, Ecole Centrale de Lyon, INSA Lyon, Universite Claude Bernard Lyon 1, LIRIS, UMR5205, 69007 Lyon, France

  2. Scalable Physics-Inspired Transformers for Spin Glasses

    Jun 22, 2026Lu Zhong, Wenli Duan, Jing Liu +2Transformer

    Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu 611731, China · School of Physical Science and Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China · Institute of Theoretical Physics, Chinese Academy of Sciences, Beijing 100190, China +4

  3. BranchShine: Compact Raw-Audio-to-IPA Transcription with a RoPE E-Branchformer Encoder

    Jun 22, 2026Nikhil Navas, Sergio Chevtchenko, Talisson Damiao +1TransformerMultilingual ASR

    1International Centre for Neuromorphic Systems, Western Sydney University · 2Neurabuild

  4. SOHET: Sequence Of Heterogeneous Events Transformer with Self-Supervised Pre-Training

    Jun 19, 2026Kees Jan de Vries, Mustafa Radha, Mathijs de JongTransformerSelf-Supervised Pre-Training

    Booking.com, Amsterdam, The Netherlands

  5. Scalable Hierarchical Attention Transformers for Multi-Turn Jailbreak Detection in Long Conversations

    Jun 19, 2026Chenhui Hu, Muhammed Salih, Sudipto Guha +1TransformerMulti-Turn Jailbreak Attacks

    Zscaler, Inc.

  6. FiLM-Coordinated Dual-Branch Transformer for Global-Local Dependency Modeling in Language Modeling

    Jun 19, 2026Zhiqiang Zhou, Xu Ling, Junliang DaiTransformerLanguage Modeling

  7. Learning through Internalization

    Jun 18, 2026Nikolaos Tsilivis, Nirmit Joshi, Marko Medvedev +2TransformerOOD Generalization

    New York University · Toyota Technological Institute at Chicago · University of Chicago

  8. Comparing Transformers and Hybrid Models at the Token Level

    Jun 18, 2026Yanhong Li, William MerrillLLM EvaluationTransformer

    Allen Institute for AI

  9. HEPTv2: End-to-End Efficient Point Transformer for Charged Particle Reconstruction

    Jun 18, 2026Siqi Miao, Shitij Govil, Jack P. Rodgers +5Efficient Transformer InferenceTransformer

    School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA. · Department of Physics and Astronomy, Purdue University, West Lafayette, IN, USA. · Department of Physics, University of California San Diego, La Jolla, CA, USA. +1

  10. OnDeFog: Online Decision Transformer under Frame Dropping

    Jun 18, 2026Daiki Yotsufuji, Kenta Nishihara, Shoma Shimizu +2Reinforcement LearningTransformer

    Yokohama National University, Kanagawa, Japan

  11. Efficiently Representing Algorithms With Chain-of-Thought Transformers

    Jun 18, 2026Yanhong Li, Anej Svete, Ashish Sabharwal +1TransformerCoT Reasoning

    Allen Institute for AI · ETH Zürich

  12. Massive Activations Are Architecturally Robust: A Controlled Scratch/Commitment Residual Stream Test

    Jun 17, 2026Maruthi VemulaTransformer

    University of North Carolina at Chapel Hill

  13. Algebraic Dead Directions in LayerNorm Transformers: A Forward-Pass-Only Diagnostic at LLM Scale

    Jun 17, 2026Tejas Pradeep Shirodkar, P. J. NarayananInformation GeometryTransformer

    IIIT, Hyderabad

  14. QC-GAN: A Parameter-Efficient Quaternion Conformer GAN for High-Fidelity Speech Enhancement

    Jun 17, 2026Shogo Yamauchi, Hideaki Tamori, Makoto Sakai +2TransformerComplex-Valued Neural Networks

    The Asahi Shimbun Company, Japan · Tokyo Woman’s Christian University, Japan

  15. Variable-Width Transformers

    Jun 16, 2026Zhaofeng Wu, Oliver Sieberling, Shawn Tan +3TransformerDecoder-Only Language Models

    ãMIT · èMIT-IBM Watson AI Lab

  16. HLS-GPT: A Generative Pretrained Transformer (GPT) for Continental-Scale NASA Harmonized Landsat and Sentinel-2 (HLS) Reflectance Reconstruction Across All Bands on Arbitrary Dates

    Jun 16, 2026Junjie Li, Hankui K. Zhang, David P. RoyTransformerRemote Sensing

    Geospatial Sciences Center of Excellence, Department of Geography and Geospatial Sciences, South Dakota State University, Brookings, SD 57007, USA · Department of Geography, Environment, and Spatial Sciences, & Center for Global Change and Earth Observations, Michigan State University, MI 48824, USA

  17. Functional Equivalence in Attention: A Comprehensive Study with Applications to Linear Mode Connectivity

    Jun 16, 2026Viet-Hoang Tran, Vinh Khanh Bui, Van-Hoan Trinh +2Rotary Positional EmbeddingsTransformer

    National University of Singapore · Center for AI Research, VinUniversity · Technical University of Munich +1

  18. TIGER: Inverting Transformer Gradients via Embedding-Subspace Distance Optimization

    Jun 16, 2026William Kalikman, Ivo Petrov, Dimitar I. Dimitrov +1Gradient Inversion AttacksTransformer

    ETH Zürich · INSAIT, Sofia University "St. Kliment Ohridski"

  19. Taming Curvature: Architecture Warm-Up for Stable Transformer Training

    Jun 15, 2026Sameera Ramasinghe, Ajanthan Thalaiyasingam, Hadi Mohaghegh Dolatabadi +6Edge of StabilityTransformer

    Pluralis Research

  20. Attention is Just Another Name for Coupling? A Fast-Slow ODE Perspective on Hierarchical Pretraining

    Jun 15, 2026Zhengyuan GaoDynamical SystemsTransformer

    Independent scholar

  21. CraBERT: Efficient Phoneme Encoder Pre-Training via Cascade Fusion of Subword Representations for Text-to-Speech

    Jun 15, 2026Dong Yang, Yuki Saito, Wataru Nakata +1Language Model PretrainingRepresentation Learning

    The University of Tokyo, Japan

  22. NeuronFabric: A Software Reference Architecture for On-Chip Transformer Training with Local Adam

    Jun 15, 2026Evgeny UkladchikovDeep Learning OptimizationTransformer

    Independent Researcher

  23. Scaling Adaptive Depth with Norm-Agnostic Residual Networks

    Jun 15, 2026Tomás Figliolia, Beren MillidgeEfficient Neural Network InferenceDynamic Neural Networks

    Zyphra San Francisco, CA

  24. David vs. Goliath in Next Activity Prediction: Argmax vs. LSTM, Transformer, and LLM

    Jun 14, 2026Hans Weytjens, Ingo WeberLLM EvaluationPredictive Process Monitoring

    Technical University of Munich, School of CIT, Germany · Fraunhofer Gesellschaft, Munich, Germany

  25. Transformers Learn the Mestre-Nagao Heuristic

    Jun 13, 2026Pranav Venkata KondaTransformer InterpretabilityTransformer

  26. Simplifying the Modeling of Arbitrary Conditionals in Natural Language

    Jun 12, 2026Yinhan Lu, Eric Elmoznino, Léo Gagnon +3TransformerAutoregressive Language Modeling

    Mila — Quebec AI Institute · McGill University · Université de Montréal

  27. How Linear Is a Transformer Feed-Forward Block? Per-Block Linear Recoverability Is Learned, Not Architectural

    Jun 12, 2026Stuart WhippTransformer FFNsTransformer Expressivity

    Independent Research

  28. HiST: A Hierarchical Sparse Transformer for Cross-Modal Spatial Transcriptomics Modeling

    Jun 12, 2026Weiyi Wu, Xinwen Xu, Xingjian Diao +4Computational PathologySpatial Transcriptomics

    Dartmouth College · Mass General Hospital · New Jersey Institute of Technology +1

  29. Illumination-Robust Camera-Based Heart-Rate Estimation for Physiological Sensing in Robots

    Jun 10, 2026Zhi Wei Xu, Torbjörn E. M. NordlingRemote PhotoplethysmographyDistribution Shift Robustness

    Department of Mechanical Engineering, National Cheng Kung University, Tainan, Taiwan.

  30. Vision Transformers for Face Recognition Need More Registers

    Jun 10, 2026Tahar Chettaoui, Guray Ozgur, Eduarda Caldeira +2Transformer InterpretabilityTransformer

    Fraunhofer Institute for Computer Graphics Research IGD, Germany · Department of Computer Science, TU Darmstadt, Germany