Transformer Architectures

Ask about this topic

Recent momentum

-24%

52 papers in the last 28 days · 0.8% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Also in Transformers

Neighbouring directions

Quadratic AttentionDynamic Sparse AttentionTransformer AttentionTransformer EncoderTransformer LayersSoftmax AttentionUnified TransformerMulti-Head Self-Attention

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

20 new papers

A weekly snapshot of new work published in Transformer Architectures.

Period ending 2026-09-14

1 new paper

A weekly snapshot of new work published in Transformer Architectures.

Period ending 2026-09-07

2 new papers

A weekly snapshot of new work published in Transformer Architectures.

684 papers

Latest in Transformer Architectures

Browse all topics →
CardsList

  1. Adaptive Head Budgeting for Efficient Multi-Head Attention

    Apr 24, 2026Bilal Faye, Abdoulaye Mbaye, Hanane Azzag +1Multi-Head Self-AttentionTransformer Architectures

  2. Learning Reactive Human Motion Generation from Paired Interaction Data Using Transformer-Based Models

    Apr 24, 2026Masato Soga, Ryuki TakebayashiHuman Motion GenerationMotion Generation

  3. Dissociating Decodability and Causal Use in Bracket-Sequence Transformers

    Apr 24, 2026Aryan Sharma, Cutter Dawes, Shivam RavalTransformer EncoderTransformer Architectures

  4. Universal Transformers Need Memory: Depth-State Trade-offs in Adaptive Recursive Reasoning

    Apr 23, 2026Grigory SapunovRecurrent ModelTransformer Architectures

  5. To See the Unseen: on the Generalization Ability of Transformers in Symbolic Reasoning

    Apr 23, 2026Nevena Lazić, Liam Fowl, András György +1Transformer EncoderTransformer Architectures

  6. Sculpt4D: Generating 4D Shapes via Sparse-Attention Diffusion Transformers

    Apr 23, 2026Minghao Yin, Wenbo Hu, Jiale Xu +24D GenerationMultimodal Diffusion Transformers

  7. EdgeFormer: local patch-based edge detection transformer on point clouds

    Apr 23, 2026Yifei Xie, Zhikun Tu, Tong Yang +2Spatial PatchingTransformer Architectures

  8. an interpretable vision transformer framework for automated brain tumor classification

    Apr 23, 2026Chinedu Emmanuel Mbonu, Tochukwu Sunday Belonwu, Okwuchukwu Ejike Chukwuogo +1Brain Tumor ClassificationSelf-Supervised Vision Transformers

  9. Hyperloop Transformers

    Apr 23, 2026Abbas Zeitoun, Lucas Torroba-Hennigen, Yoon KimTransformer ArchitecturesStable Transport

  10. The Recurrent Transformer: Greater Effective Depth and Efficient Decoding

    Apr 23, 2026Costin-Andrei Oncescu, Depen Morwani, Samy Jelassi +3Transformer ArchitecturesKey-Value Cache

  11. climt-paraformer: Stable Emulation of Convective Parameterization using a Temporal Memory-aware Transformer

    Apr 22, 2026Shuochen Wang, Nishant Yadav, Joy Merwin Monteiro +1Global Climate ModelsEmulators

  12. Working Memory Constraints Scaffold Learning in Transformers under Data Scarcity

    Apr 22, 2026Pranava Madhyastha, Dagmar AdamcovaTransformer ArchitecturesWorking Memory

  13. Explicit Dropout: Deterministic Regularization for Transformer Architectures

    Apr 22, 2026Vidhi Agrawal, Illia Oleksiienko, Alexandros IosifidisDropoutTransformer Attention

  14. LaplacianFormer:Rethinking Linear Attention with Laplacian Kernel

    Apr 22, 2026Zhe Feng, Sen Lian, Changwei Wang +5Quadratic AttentionSoftmax Attention

  15. PASTA: A Patch-Agnostic Twofold-Stealthy Backdoor Attack on Vision Transformers

    Apr 21, 2026Dazhuang Liu, Yanqi Qiao, Rui Wang +2Vision TransformerBackdoor Attacks

  16. Benign Overfitting in Adversarial Training for Vision Transformers

    Apr 21, 2026Jiaming Zhang, Meng Ding, Shaopeng Fu +2Self-Supervised Vision TransformersAdversarial Training

  17. GRAFT: Geometric Refinement and Fitting Transformer for Human Scene Reconstruction

    Apr 21, 2026Pradyumna YM, Yuxuan Xue, Yue Chen +3Articulated ObjectsTransformer Architectures

  18. Multimodal Transformer for Sample-Aware Prediction of Metal-Organic Framework Properties

    Apr 21, 2026Seunghee Han, Jaewoong Lee, Jihan KimMolecular Property PredictionCryo-Electron Microscopy

  19. Multi-view Crowd Tracking Transformer with View-Ground Interactions Under Large Real-World Scenes

    Apr 21, 2026Qi Zhang, Jixuan Chen, Kaiyi Zhang +3Multi-Object TrackingMulti-View

  20. Multimodal embodiment-aware navigation transformer

    Apr 21, 2026Louis Dezons, Quentin Picard, Rémi Marsal +2Embodied NavigationObject Goal Navigation

  21. ConvVitMamba: Efficient Multiscale Convolution, Transformer, and Mamba-Based Sequence modelling for Hyperspectral Image Classification

    Apr 20, 2026Mohammed Q. AlkhatibHyperspectral Image ClassificationVision Transformer

  22. One Step Forward and K Steps Back: Better Reasoning with Denoising Recursion Models

    Apr 20, 2026Chris Cameron, Wangzheng Wang, Nikita Ivanov +3Denoising Diffusion Probabilistic ModelImage Denoising

  23. Towards Understanding the Robustness of Sparse Autoencoders

    Apr 20, 2026Ahson Saiyed, Sabrina Sadiekh, Chirag AgarwalAdversarial JailbreaksTransformer Architectures

  24. Linear-Time and Constant-Memory Text Embeddings Based on Recurrent Language Models

    Apr 20, 2026Tobias Grantner, Emanuel Sallinger, Martin FlechlDomain-Invariant Text EmbeddingsRecurrent Model

  25. How Much Cache Does Reasoning Need? Depth-Cache Tradeoffs in KV-Compressed Transformers

    Apr 20, 2026Xiao WangKey-Value Cache CompressionKey-Value Cache

  26. Towards Generalizable Deepfake Image Detection with Vision Transformers

    Apr 19, 2026Kaliki V Srinanda, M Manvith Prabhu, Hemanth K Mogilipalem +4Explainable Deepfake DetectionSelf-Supervised Vision Transformers

  27. NaviFormer: A Deep Reinforcement Learning Transformer-like Model to Holistically Solve the Navigation Problem

    Apr 18, 2026Daniel Fuertes, Andrea Cavallaro, Carlos R. del-Blanco +2Trajectory PredictionDeep Reinforcement Learning

  28. Adaptive Computation Depth via Learned Token Routing in Transformers

    Apr 18, 2026Ahmed Abdelmuniem Abdalla MohammedTransformer ArchitecturesMultilayer Perceptrons

  29. SAVE: A Generalizable Framework for Multi-Condition Single-Cell Generation with Gene Block Attention

    Apr 18, 2026Jiahao Li, Jiayi Dong, Peng Ye +3Single-Cell Rna SequencingHigh-Fidelity Conditional Generation

  30. Closing the Theory-Practice Gap in Spiking Transformers via Effective Dimension

    Apr 17, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuNeuromorphic HardwareTransformer Architectures

  31. Stability and Generalization in Looped Transformers

    Apr 16, 2026Asher LabovichTransformer ArchitecturesFixed-Point Iteration

  32. What Is the Minimum Architecture for Prolepsis? Early Irrevocable Commitment Across Tasks in Small Transformers

    Apr 16, 2026Éric JacopinTransformer ArchitecturesSmall Models

  33. Expressivity of Transformers: A Tropical Geometry Perspective

    Apr 16, 2026Ye Su, Yong LiuTransformer AttentionMulti-Head Self-Attention

  34. FreqTrack: Frequency Learning based Vision Transformer for RGB-Event Object Tracking

    Apr 16, 2026Jinlin You, Muyu Li, Xudong ZhaoRgbtMulti-Object Tracking

  35. ExFusion: Efficient Transformer Training via Multi-Experts Fusion

    Mar 30, 2026Jiacheng Ruan, Daize Dong, Xiaoye Qu +5Mixture-of-Experts ModelsTransformer Architectures

  36. When Perplexity Lies: Generation-Focused Distillation of Hybrid Sequence Models

    Mar 27, 2026Juan Gabriel Kostelec, Qinghai GuoGenerative Pretrained TransformersGenerative Quality

  37. On the Ability of Transformers to Verify Plans

    Mar 20, 2026Yash Sarrof, Yupei Du, Katharina Stein +3Classical PlanningTransformer Architectures

  38. Human-like Object Grouping in Self-supervised Vision Transformers

    Mar 14, 2026Hossein Adeli, Seoyoung Ahn, Andrew Luo +3Self-SupervisedObject-Centric Learning

  39. AccelAes: Accelerating Diffusion Transformers for Training-Free Aesthetic-Enhanced Image Generation

    Mar 13, 2026Xuanhua Yin, Chuanzhi Xu, Haoxian Zhou +2Text-To-ImageImage Generation

  40. HiAP: A Multi-Granular Stochastic Auto-Pruning Framework for Vision Transformers

    Mar 12, 2026Andy Li, Aiden Durrant, Milan Markovic +1Self-Supervised Vision TransformersVision Transformer

  41. Feature-level Interaction Explanations in Multimodal Transformers

    Mar 4, 2026Yeji Kim, Housam Khalifa Bashier Babiker, Mi-Young Kim +1Multimodal TransformersExplainability

  42. Retrievit: In-context Retrieval Capabilities of Transformers, State Space Models, and Hybrid Architectures

    Mar 3, 2026Georgios Pantazopoulos, Malvina Nikandrou, Ioannis Konstas +1Transformer ArchitecturesPositional Encoding

  43. MetaOthello: A Controlled Study of Multiple World Models in Transformers

    Feb 26, 2026Aviral Chawla, Galen Hall, Juniper LovatoTransformer ArchitecturesMultimodal Transformers

  44. Transformers converge to invariant algorithmic cores

    Feb 26, 2026Joshua S. SchiffmanTransformer ArchitecturesMemorization

  45. Bridging Latent Reasoning and Target-Language Generation via Retrieval-Transition Heads

    Feb 25, 2026Shaswat Patel, Vishvesh Trivedi, Yue Han +2Multilingual Language ModelsEfficient Latent Reasoning

  46. How Transformers Reject Wrong Answers: Rotational Dynamics of Factual Constraint Processing

    Feb 25, 2026Javier MarínTransformer EncoderTransformer Architectures

  47. Incremental Learning of Sparse Attention Patterns in Transformers

    Feb 22, 2026Oğuz Kaan Yüksel, Rodrigo Alvarez Lucendo, Nicolas FlammarionTransformer AttentionTransformer Architectures

  48. Central Dogma Transformer II: An AI Microscope for Understanding Cellular Regulatory Mechanisms

    Feb 9, 2026Nobuyuki OtaGene-Regulatory NetworksRegulation

  49. Language-Guided Transformer Tokenizer for Human Motion Generation

    Feb 9, 2026Sheng Yan, Yong Wang, Xin Du +2Human Motion GenerationTokenization

  50. Revisiting the Shape Convention of Transformer Language Models

    Feb 6, 2026Feng-Ting Liao, Guan-Ting Yi, Tzu-Quan Lin +2Transformer ArchitecturesNeural Architectures

  51. A Constrained Optimization Perspective of Unrolled Transformers

    Jan 24, 2026Javier Porras-Valenzuela, Samar Hadou, Alejandro RibeiroTransformer ArchitecturesConstrained Optimization

  52. From Local Windows to Adaptive Candidates via Individualized Exploratory: Rethinking Attention for Image Super-Resolution

    Jan 13, 2026Chunyu Meng, Wei Long, Shuhang GuGenerative Super-ResolutionSelf-Supervised Vision Transformers

  53. The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Models

    Jan 9, 2026Nora Graichen, Iria de-Dios-Flores, Gemma BoledaSyntactic StructureBert-Based Models

  54. TexTailor: Inference-Time Textual Guidance Tailoring for Multimodal Diffusion Transformers

    Jan 5, 2026Binglei Li, Mengping Yang, Zhiyu Tan +2Multimodal Diffusion TransformersText-To-Image

  55. SwinIFS: Landmark Guided Swin Transformer For Identity Preserving Face Super Resolution

    Jan 4, 2026Habiba Kausar, Saeed Anwar, Omar Jamal Hammad +1Diffusion-Based Super-Resolution FrameworkBlind Face Restoration

  56. GeoTransolver: Learning Physics on Irregular Domains Using Multi-scale Geometry Aware Physics Attention Transformer

    Dec 23, 2025Corey Adams, Rishikesh Ranade, Ram Cherukuri +1Representational GeometryFinite Element Method

  57. Probing Spectrum-Like Organization of States of Mind in Transformer Representation Spaces

    Dec 23, 2025Sophie ZhaoTransformer ArchitecturesRepresentation Quality

  58. Flexible Gravitational-Wave Parameter Estimation with Transformers

    Dec 2, 2025Annalena Kofler, Maximilian Dax, Stephen R. Green +6Gravitational WavesDiscrete Wavelet Transform

  59. Understanding the Staged Dynamics of Transformers in Learning Latent Structure

    Nov 24, 2025Rohan Saha, Farzane Aminmansour, Alona FysheTransformer ArchitecturesLatent Structure

  60. A Space-Time Transformer for Precipitation Nowcasting

    Nov 14, 2025Levi Harris, Tianlong ChenPrecipitationAdverse Weather