Transformer Architectures

Ask about this topic

Recent momentum

-24%

52 papers in the last 28 days · 0.8% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Also in Transformers

Neighbouring directions

Quadratic AttentionDynamic Sparse AttentionTransformer AttentionTransformer EncoderTransformer LayersSoftmax AttentionUnified TransformerMulti-Head Self-Attention

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

20 new papers

A weekly snapshot of new work published in Transformer Architectures.

Period ending 2026-09-14

1 new paper

A weekly snapshot of new work published in Transformer Architectures.

Period ending 2026-09-07

2 new papers

A weekly snapshot of new work published in Transformer Architectures.

684 papers

Latest in Transformer Architectures

Browse all topics →
CardsList

  1. A Coulomb Particle Model for Learning Kernel Attention in Transformers

    Jul 26, 2026Masoud Badiei Khuzani, Sharath Honnaiah, Atiq Islam +2Transformer AttentionFeature Learning

  2. Memory Efficient Audio Synthesis with Decoupled Temporal Depth Diffusion Transformers

    Jul 26, 2026Dongseong Hwang, Prasanth Yadla, Kaan Elgin +8Audio TokensZero-Shot Text-To-Speech

  3. ATLAS: Automated Approximation of Transformers for Efficient Homomorphic Inference in One Hour

    Jul 26, 2026Jianhang Xie, Sicheng Tan, Vishnu Naresh Boddeti +1Homomorphic EncryptionTransformer Architectures

  4. Structure over Depth: A Single-Block Spatio-Temporal Transformer for Multi-Entity Reasoning

    Jul 25, 2026Narthana Sivalingam, Santhirarajah Sivasthigan, Buddhi Wijenayake +3Spatio-Temporal TransformersTemporal Dynamics

  5. The Entropic Bound for Transformers: Why Static Rank Fails and Attention-Native Rank Recovers

    Jul 25, 2026Byeong Hoon YoonLow-Rank FactorizationTransformer Architectures

  6. Hidden Boundary Motion in Transformer Optimization: Function-Space Orthogonalization of Affine Weight and Bias Updates

    Jul 24, 2026Zhang Gongyue, Sheng Yixuan, Liu donghan +3AdamAffine

  7. Learning to Prepare Molecular Ground States with Transformer Models

    Jul 24, 2026Alex Koziell-Pipe, Jasmine Brewer, Jem Guhit +14Quantum Circuit SimulationN$-Qubit Stabilizer States

  8. LithoFormer: A Robust Framework for Stratigraphic Inference via Transformers

    Jul 24, 2026Shwetha Salimath, Francesca Bugiotti, Sylvain Wlodarczyk +1Lithology ClassificationGeological Semantic Understanding

  9. Interior interpretability with attention rollout: contraction and propagation profiles in Transformers

    Jul 24, 2026Umberto Biccari, Qian Huang, Enrique ZuazuaTransformer AttentionModel Interpretability

  10. Geometric 2D Scene Graph Generation

    Jul 24, 2026Christoph Jahn, Urs Waldmann, Bastian GoldlueckeHierarchical 3D Scene GraphsTransformer Architectures

  11. Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting

    Jul 24, 2026Wan Zhang, Qinjie Lin, Chan Lee +3Time SeriesMixformer

  12. Optimizing Transformer Neural Network for Real-Time Outlier Detection on FPGAs

    Jul 24, 2026Ilia Sobakinskikh, Paul Alexander BilokonTime-Series Anomaly DetectionHybrid Transformer

  13. Multimodal Surface EMG Hand Gesture Recognition Using Query-Based Transformers for Prosthetic Control

    Jul 24, 2026Federico Del Pup, Elisa Tentori, Manfredo AtzoriHand Gesture RecognitionMultimodal Physiological Signals

  14. RED-PIM: Reducing Data Movement for Transformers using Processing-in-Memory

    Jul 23, 2026Zahra Yousefijamarani, Alaa AlameldeenTransformer AttentionTransformer Architectures

  15. Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs

    Jul 23, 2026Yidu Wu, Xiang Wang, Kejie Zhao +3LLM Inference EfficiencyLarge Language Model Pretraining

  16. Agree on the Model, Verify the Inference: GKR Protocols for HND-Based Transformer Inference

    Jul 23, 2026Xiaolong Liang, Juanjuan Li, Rui Qin +1Homomorphic EncryptionTransformer Architectures

  17. Scaling Interpretable Transformers with Parity Bottleneck Layers

    Jul 22, 2026Andrew Mack, Kraig Yuheng Tou, Mark Henry +2Transformer ArchitecturesModel Interpretability

  18. The Giant Hippocampus: From Structural Monoculture to a System of Systems

    Jul 22, 2026Jaeho SeolHuman Visual CortexNeural Architectures

  19. Appearance Pointers -- Multimodal Region Control of Diffusion Transformers

    Jul 21, 2026Rahul Sajnani, Yulia Gryaditskaya, Radomír Měch +2Multimodal Diffusion TransformersAppearance

  20. Breaking Feedback-Blindness: Utility-Augmented Transformer for Sequential Decision Making

    Jul 21, 2026Yuyang Shen, Shan Dai, Daimin ChenTransformer ArchitecturesSequential Decision Making

  21. Elicitation without Backpropagation: Steering Model Behavior by Optimizing the Latent Posterior

    Jul 21, 2026Garrett Baker, Vinayak Pathak, Daniel Murfet +1PosteriorElicitation

  22. Beyond Transformers: Linear Attention Policy for Open-Vocabulary Object Goal Navigation

    Jul 21, 2026Jiahong Zhang, Yifan Lin, Yandong Zhang +6Object Goal NavigationNavigation

  23. On the Diverse Dynamical Behaviors Arising in Deep Linear Transformers

    Jul 20, 2026Sixu Li, Thomas Jacob Maranzatto, Jan Peszek +5Transformer ArchitecturesSoftmax Attention

  24. DuSPiT: Dual-Branch Sub-Patch Pixel Diffusion Transformer

    Jul 20, 2026Yunpeng Bai, Yossi Gandelsman, Michaël GharbiMultimodal Diffusion TransformersLarge-Scale Image Generation

  25. Adaptive Multi-Expert Graph Transformer for Interpretable EEG-Based Diagnostics

    Jul 20, 2026Maryam Rahimimovassagh, Md Elias Hossain, Ivan Garibay +1Electroencephalography Spatial Super-ResolutionFunctional Networks

  26. A Controlled Study of Attention-Only Transformers

    Jul 20, 2026Henry Ndubuaku, Karen Mosoyan, Jakub Mroz +5Transformer AttentionTransformer Encoder

  27. Mobius Learning: Cyclic Depth Folding in Transformers

    Jul 20, 2026Tongtian ZhuTransformer Architectures

  28. DA-Fusion: Deformable Attention-Based RGB-D Fusion Transformer for Unseen Object Instance Segmentation

    Jul 20, 2026Yesol Park, Hye-Jung Yoon, Juno Kim +1Robotic PerceptionGeneralizable Robotic Manipulations

  29. Planning with Transformers: Chain of Computation and Structured Context Windows

    Jul 20, 2026Ehsan Futuhi, Nathan R. SturtevantLarge Language Model PlanningTransformer Architectures

  30. An Adjoint-Sensitivity Framework for Lost-in-the-Middle Phenomena in Causal Residual Transformers

    Jul 20, 2026Cheng Huan, Hongwei YuanTransformer Residual StreamsModel Sensitivity

  31. An Analysis of Residual-Stream Geometry Across Transformer Depth

    Jul 20, 2026Sunit Bhattacharya, Ravi Shankar KolliTransformer Residual StreamsResidual Stream

  32. Brain-Aligned Multi-Stream Video Transformers with Sparse Self-Selection

    Jul 20, 2026Amir Hosein Fadaei, Mahyar Maleki, Mohammad-Reza A. DehaqaniVideo UnderstandingTransformer Architectures

  33. Can Transformers Really Do It All? On the Compatibility of Inductive Biases Across Tasks

    Jul 20, 2026Damien Teney, Liangze Jiang, Hemanth Saratchandran +1Transformer ArchitecturesInductive Bias

  34. Pixel-Space Diffusion Transformers

    Jul 20, 2026Renye Yan, Jikang Cheng, You Wu +8Multimodal Diffusion TransformersVisual Tokens

  35. A Weisfeiler-Leman Characterization of Global-Attention Graph Transformers for Mixed-Integer Linear Programs

    Jul 20, 2026Md Abrar Jahin, Craig A. Knoblock, Jay PujaraQuadratic AttentionTransformer Architectures

  36. Staged Depth-Pruning Distillation of a Flow-Matching Text-to-Speech Teacher: A Compact Hindi Speech Synthesizer

    Jul 19, 2026Sivateja TrikutamZero-Shot Text-To-SpeechTransformer Architectures

  37. Histopathological Spectrum-Guided Prostate Stratification via Segmentation-Assisted Diagnostic Transformer

    Jul 19, 2026Leyang Li, Lihua Chen, Huangang Hu +7Lesion SegmentationComputational Pathology

  38. The Geometry of Semantic Space: A Continuous Geometric Framework for the Transformer Architecture

    Jul 19, 2026Zhihua LiangTransformer ArchitecturesRotary Position Embedding

  39. What does a Bayes-filtered transformer believe? A predictive Monte Carlo approach

    Jul 19, 2026Afiq Abdillah Effiezal Aswadi, Haotong Ma, Susan WeiPosteriorNext-Token Prediction

  40. First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers

    Jul 18, 2026Irina Piontkovskaia, Sergey NikolenkoTransformer ArchitecturesUnseen-Task Adaptation

  41. In-context learning of closed form solution to simple linear regression task using transformer with linear self-attention

    Jul 17, 2026Katsuyuki HagiwaraIn-Context LearningTransformer Architectures

  42. A Benchmark for Electrical Load Forecasting Across Grid Levels: Time-Series Transformers Outperform Established Methods

    Jul 17, 2026Matthias Hertel, Sebastian Pütz, Jonathan Kolar +3Electricity Price ForecastingTime Series Foundation Models

  43. Boundary-Seeking GAN-Augmented TabTransformer for Adversarially Robust Intrusion Detection

    Jul 17, 2026Raihan Sultan Pasha Basuki, Aliyah KurniasihIntrusion DetectionGenerative Adversarial Network

  44. T^2MLR: Transformer with Temporal Middle-Layer Recurrence

    Jul 16, 2026Ziyang Cai, Xingyu Zhu, Yihe Dong +2Efficient Latent ReasoningTransformer Architectures

  45. Towards Hierarchical Structure Understanding of Newspaper Images

    Jul 16, 2026William Mocaër, Solène Tarride, Thomas Constum +7HierarchicalHeadlines

  46. FlashDecoder: Real-Time Latent-to-Pixel Streaming Decoder with Transformers

    Jul 16, 2026Minguk Kang, Suha KwakLarge Language Model DecoderTransformer Architectures

  47. Beyond Implicit Force: Evaluating Explicit Force-Torque Proxies in Action Chunking with Transformers

    Jul 16, 2026King Hang Wong, Lingqiao Liu, Feras DayoubActuatorTeleoperation

  48. MxGPS: Multiplex Graph Transformers for a Power Grid Foundation Model

    Jul 15, 2026Charilaos Papaioannou, Ioannis Tsantilas, Dimitris Giannakakos +9Ac Optimal Power FlowHeterogeneous Graph Transformers

  49. DeepLoop: Depth Scaling for Looped Transformers

    Jul 15, 2026Shuzhen Li, Yifan Zhang, Jiacheng Guo +2Transformer ArchitecturesSearch Depth

  50. 2D Rotary Position Embedding for Scene Text Recognition with Transformers

    Jul 15, 2026Zobeir RaisiScene Text RecognitionRotary Position Embedding

  51. Toward Localizing and Repairing Bias in Transformer Attention Heads

    Jul 14, 2026Sigma JahanTransformer AttentionTransformer Architectures

  52. Do We Really Need Transformers for Global Spatial Information Extraction in Traffic Forecasting?

    Jul 14, 2026Qihang Zhang, Siyao Zhang, Letao Kang +3Accurate Traffic ForecastingSpatial Attention

  53. Invariant Learning Dynamics of Transformers in Inductive Reasoning Tasks

    Jul 13, 2026Tiberiu Musat, Tiago Pimentel, Nicolas Zucchet +1Transformer ArchitecturesTrainable Parameter

  54. From Expressivity to Sample Complexity: Narrow Teachers for Transformers via C-RASP

    Jul 13, 2026Michael Rizvi-Martel, Satwik Bhattamishra, Guillaume Rabusseau +1Transformer ArchitecturesSample Complexity

  55. Higher-Order Cell Tracking Transformer

    Jul 13, 2026Jordão Bragantini, Ilan Theodoro, Loïc A. RoyerTransformer ArchitecturesLineages

  56. Sparse Inter-Layer Dependencies of Transformer FFN Neurons

    Jul 13, 2026Johannes Knittel, Hanspeter PfisterFeed-Forward NetworkTransformer Architectures

  57. RepTran: Search-Based Repair of Transformer Models

    Jul 13, 2026Yuta Ishimoto, Paolo Arcaini, Fuyuki Ishikawa +3Transformer ArchitecturesArtificial Intelligence Systems

  58. Controlling Motion Transfer in Diffusion Transformers via Attention Heads

    Jul 13, 2026Sunyoung Jung, Jiwoo Park, Yoonseok Choi +3Motion-Focused RepresentationsTransformer Architectures

  59. Incremental Transformer for Surrogate-Based Inverse Design of Geopolymer Mixtures

    Jul 12, 2026Giansalvo Cirrincione, Filippo GrassiaGenerative Inverse DesignInverse Design

  60. LayerNorm as Implicit Gain Control in Looped Transformers

    Jul 12, 2026Matthias M. M. BuehlmaierBatch NormalizationTransformer Layers