Transformer Architectures

Ask about this topic

Recent momentum

-24%

52 papers in the last 28 days · 0.8% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Also in Transformers

Neighbouring directions

Quadratic AttentionDynamic Sparse AttentionTransformer AttentionTransformer EncoderTransformer LayersSoftmax AttentionUnified TransformerMulti-Head Self-Attention

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

20 new papers

A weekly snapshot of new work published in Transformer Architectures.

Period ending 2026-09-14

1 new paper

A weekly snapshot of new work published in Transformer Architectures.

Period ending 2026-09-07

2 new papers

A weekly snapshot of new work published in Transformer Architectures.

684 papers

Latest in Transformer Architectures

Browse all topics →
CardsList

  1. SAFE-DiT: Semantics-Aware Fast-path Execution for High-Resolution Diffusion Transformers

    Jun 28, 2026Xuanhua Yin, Yuxuan Jia, Chuanzhi Xu +1Multimodal Diffusion TransformersSelf-Supervised Vision Transformers

  2. Adaptive Financial Transformer with Regime-Gated Attention for Stock Return Prediction

    Jun 28, 2026Dishan SarkarTransformer AttentionTransformer Architectures

  3. Generalization Analysis of Transformers in Distribution Regression

    Jun 28, 2026Peilin Liu, Ding-Xuan ZhouTransformer ArchitecturesGeneralization Bounds

  4. Do Diabetic Foot Ulcer Segmentation Models Generalize? A Cross-Dataset Benchmark of CNN and Transformer Architectures

    Jun 27, 2026Abderrahmane BenfatahCross-Dataset BenchmarkConvolutional Neural Networks

  5. From Black-Box to Clinical Insight: A Multi-Stage Explainable Framework for Speech-Based Cognitive Impairment Detection

    Jun 26, 2026Yasaman Haghbin, Sina Rashidi, Ali Zolnour +6Mild Cognitive ImpairmentCognitive Diagnosis

  6. Mitigating Position Bias in Transformers via Layer-Specific Positional Embedding Scaling

    Jun 26, 2026Changze Lv, Zhenghua Wang, Yiran Ding +9Rotary Position EmbeddingTransformer Architectures

  7. AI-Generated Image Recognition via Fusion of CNNs and Vision Transformers

    Jun 26, 2026Xuan-Bach Mai, Hoang-Minh Nguyen-Huu, Quoc-Nghia Nguyen +3Synthetic Image DetectionAuthenticity

  8. The Context-Ready Transformer

    Jun 25, 2026Mahesh GodavartiTransformer LayersTransformer Architectures

  9. Developmental approach reveals the statistical learning of Neural Language Models: Transformers generalize from the most abstract statistical patterns

    Jun 25, 2026Wang Bojun, Holly Jenkins, Elizabeth WonnacottTransformer Architectures

  10. Prism Transformer: Progressive Head Schedules for Hierarchical Attention Processing

    Jun 25, 2026Shubham AggarwalTransformer ArchitecturesPrism

  11. Transformer-Based Classification of Bacterial Raman Spectra with LOOCV

    Jun 25, 2026Jamile Mohammad Jafari, Thomas BocklitzRamanTransformer Architectures

  12. Generating Special Triangulations with Transformers

    Jun 25, 2026Charles Arnal, Jacky H. T. Yip, François Charton +1TriangulationsPolytopes

  13. CascadeFormer: Depth-Tapered Transformers Motivated by Gradient Fan-in Asymmetry

    Jun 25, 2026Huzama Ahmad, Cao Viet Hai Nam, Se-Young YunTransformer ArchitecturesMixformer

  14. Lifelong In-Context Learning with Transformers Requires Parametric Forms of Attention

    Jun 24, 2026Luke McDermott, Robert W. Heath, Rahul ParhiIn-Context LearningTransformer Architectures

  15. AeroCast: Probabilistic 3D Trajectory Prediction for Non-Cooperative Aerial Obstacles via Transformer-MDN Architecture

    Jun 23, 2026Syed Izzat Ullah, Jose BacaAerotrackTransformer Architectures

  16. An LLM-based Two-Stage Transformer Framework for Cross-Domain Bearing Fault Diagnosis with Limited Data

    Jun 23, 2026Jinghan Wang, Feng Cheng, Wentao Wu +3Fault DiagnosisPredictive Maintenance

  17. SURGELLM: Rethinking Multi-Task Evaluation through Task-Aware Feature Gating with Class-Balanced Normalization

    Jun 23, 2026Noor Islam S. Mohammad, Ulug BayazitNatural Language ProcessingMulti--Task Learning

  18. Prediction of Viscoelastic Droplet Impact Dynamics Using a Vision Transformer-Based Approach

    Jun 22, 2026Diego A. de Aguiar, Cassio M. Oishi6-Dof Viscoelastic MechanismSelf-Supervised Vision Transformers

  19. Tapered Language Models

    Jun 22, 2026Reza Bayat, Ali Behrouz, Aaron CourvilleTransformer Architectures

  20. Scaling Linear Mode Connectivity and Merging to Billion Parameter Pretrained Transformers

    Jun 22, 2026Tianyi Li, Zhiqiang ShenGenerative Pretrained TransformersTransformer Architectures

  21. The Energy Consumption of Transformer Fine-Tuning: A Roofline-Inspired Scaling Model

    Jun 22, 2026Mansour Zoubeirou a MayakiTransformer ArchitecturesLarge Models

  22. UniverSat: Resolution- and Modality-Agnostic Transformers for Earth Observation

    Jun 22, 2026Yohann Perron, Guillaume Astruc, Nicolas Gonthier +2Cloud-Aware Earth Observation SequencesEarth Observation

  23. Privacy-Preserving Person Re-Identification from Temporal Sequences with Transformer and Hungarian Optimization

    Jun 22, 2026Raphaël Delécluse, Hazem Wannous, Laurent GuimasRe-IdentificationRgb-Depth Cameras

  24. IViT: A Novel Interpretable Visual Transformer for Skin Disease Detection

    Jun 22, 2026Haibiao Li, Di Lin, Xue Jiang +3Skin Lesion ClassificationSelf-Supervised Vision Transformers

  25. Performance and Interpretability of Convolutional, Transformer, and Hybrid Deep Learning Models in Colorectal Histology Classification

    Jun 21, 2026Reza BozorgpourHybrid TransformerComputational Pathology

  26. First-Token Broadcasters: Mechanistic Origins of Language Identity and Distributed Robustness in Transformers

    Jun 21, 2026Arjun Pillai, Christian Hoang, Anjelo Jann LarozaEnglishInstruction-Tuned Models

  27. From Convolution to Transformer: A Comparative Study of U-Net Variants for Brain Tumor and Retinal Vessel Segmentation

    Jun 20, 2026Khoa Pham, Sindhuja Penchala, Jiacheng Li +2U-NetRetinal Vessel Segmentation

  28. ScalePredictor: Instance-aware Scale Learning for Accurate Quantization of Vision Transformers

    Jun 20, 2026Changjun Li, Runqing Jiang, Lian Xu +3Vision TransformerQuantizer

  29. Keyless Attention: Value-Space Routing and Value-Only Caching for Efficient Transformers

    Jun 20, 2026Xin GaoQuadratic AttentionTransformer Architectures

  30. LIG: Layer-wise Integrated Gradients for Within-Layer Flow Analysis in Transformers

    Jun 19, 2026Eight Suzuki, Hideitsu Hino, Noboru MurataTransformer LayersTransformer Architectures

  31. Few-Shot Hyperspectral Aphid Detection via FastGAN Synthetic Data Generation, Transformer-Based Classification and Explainable AI

    Jun 19, 2026Ali SaeidanHyperspectral Image ClassificationTomato Diseases

  32. Grouped Query Experts: Mixture-of-Experts on GQA Self-Attention

    Jun 18, 2026Vishesh Tripathi, Abhay KumarTransformer ArchitecturesMixture-of-Experts Models

  33. Comparing Transformers and Hybrid Models at the Token Level

    Jun 18, 2026Yanhong Li, William MerrillTransformer ArchitecturesToken-Level Uncertainty

  34. Efficiently Representing Algorithms With Chain-of-Thought Transformers

    Jun 18, 2026Yanhong Li, Anej Svete, Ashish Sabharwal +1Transformer ArchitecturesChain-of-Thought Reasoning

  35. Algebraic Dead Directions in LayerNorm Transformers: A Forward-Pass-Only Diagnostic at LLM Scale

    Jun 17, 2026Tejas Pradeep Shirodkar, P. J. NarayananBatch NormalizationTransformer Architectures

  36. Scaling Generative Foundation Models for Chest Radiography with Rectified Flow Transformers

    Jun 17, 2026Fabio De Sousa Ribeiro, Emma A. M. Stanley, Charles Jones +7Chest RadiographyMedical Image Generation

  37. A Hybrid LSTM--Vision Transformer Architecture for Predicting HRRR Forecast Errors

    Jun 17, 2026David Aaron Evans, Jay C. Rothenberger, Kara J. Sulia +2Numerical Weather PredictionPrecipitation

  38. Graph Grounded Cross Attention Transformer Neural Network for Structurally Constrained Full Event Sequence Generation in Predictive Process Monitoring

    Jun 17, 2026Fang Wang, Ernesto DamianiHeterogeneous Graph TransformersSequence Modeling

  39. Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers

    Jun 16, 2026Sajad Movahedi, Vera Milovanović, Shlomo Libo Feigin +5Transformer ArchitecturesFeed-Forward Network

  40. Robustness of Similarity-based Positional Encoding Under Rotations: Theoretical Analysis and Experimental Validation

    Jun 16, 2026Andrea Santomauro, Luigi Portinale, Giorgio LeonardiPositional EncodingRotation-Invariant Descriptors

  41. Functional Equivalence in Attention: A Comprehensive Study with Applications to Linear Mode Connectivity

    Jun 16, 2026Viet-Hoang Tran, Vinh Khanh Bui, Van-Hoan Trinh +2Positional EncodingTransformer Attention

  42. TIGER: Inverting Transformer Gradients via Embedding-Subspace Distance Optimization

    Jun 16, 2026William Kalikman, Ivo Petrov, Dimitar I. Dimitrov +1Federated LearningTransformer Architectures

  43. AoiZora: Topology-Aware Auto-Parallel Optimization for Inference of Diffusion Transformers

    Jun 16, 2026Kaijian Wang, Yuanyuan Xu, Fanjiang Ye +5AcceleratorShard

  44. Reconfigurable Computing Challenge: Transformer for Jet Tagging on Versal AI Engines

    Jun 16, 2026Gram Koski, Sean Lipps, Zhenghua Ma +2Transformer ArchitecturesAccelerator

  45. Discrete Autoregressive Transformer for Generative Mechanism Synthesis

    Jun 16, 2026Anar Nurizada, Anurag PurwarAutomated Synthesis TechniquesKinematic Chain

  46. The Discrete-Log Clock: How a Transformer Learns Modular Multiplication

    Jun 16, 2026Huu Danh NguyenModular AdditionTransformer Architectures

  47. Taming Curvature: Architecture Warm-Up for Stable Transformer Training

    Jun 15, 2026Sameera Ramasinghe, Ajanthan Thalaiyasingam, Hadi Mohaghegh Dolatabadi +6Curvature-Aware Quadratic BanksTransformer Architectures

  48. Attention is Just Another Name for Coupling? A Fast-Slow ODE Perspective on Hierarchical Pretraining

    Jun 15, 2026Zhengyuan GaoQuadratic AttentionModel Pretraining

  49. Adaptive inference and function vectors in deep transformers

    Jun 15, 2026Ravin Raj, Gautam ReddyTransformer ArchitecturesTransformer Attention

  50. NeuronFabric: A Software Reference Architecture for On-Chip Transformer Training with Local Adam

    Jun 15, 2026Evgeny UkladchikovTransformer ArchitecturesField-Programmable Gate Arrays

  51. Taylor-Calibrate: Principled Initialization for Hybrid Linear Attention Distillation

    Jun 15, 2026Zhongzhu Zhou, Qingyang Wu, Junxiong Wang +4Transformer Architectures

  52. Explainable Flood Segmentation on Sentinel-1 SAR1 Imagery Using CNN and Transformer Architectures

    Jun 15, 2026Arundhuti Banerjee, David DaouFlood DetectionSen1Floods11

  53. LiFT: Local Search via Linear Programming for Overfitting-Controlled Transformers

    Jun 15, 2026Abhishek Shukla, Anikeit Khanna, Ankur Sinha +1FinetuningTransformer Architectures

  54. teasr: training-efficient any-step diffusion transformer for real-world image super-resolution

    Jun 15, 2026Xiang Gao, Chenxin Zhu, Yushun Fang +2Generative Super-ResolutionMultimodal Diffusion Transformers

  55. Long-Context Modeling via GSS-Transformer Hybrid Architecture with Learnable Mixing

    Jun 15, 2026Kuzey Torlak, Hüseyin Arda Arslan, Anıl Dervişoğlu +2Efficient Long-Context InferenceHybrid Transformer

  56. The Reservoir Attention Network: Cross-Pass State in Pretrained Transformers via Content-Addressable Reservoir Injection

    Jun 14, 2026Emma LeonhartReservoir ComputingTransformer Architectures

  57. MoECa: Aligning Feature Reuse with Expert Decomposition in Diffusion Transformers

    Jun 14, 2026Maoliang Li, Haojing Chen, Jiayu Chen +4Mixture-Of-Expert InferenceMultimodal Diffusion Transformers

  58. Controlled Dynamics Attractor Transformer

    Jun 13, 2026Cheng Zhang, Minnan Luo, Zesheng Yang +3Heterogeneous Graph TransformersAttractors

  59. Transformers Learn the Mestre-Nagao Heuristic

    Jun 13, 2026Pranav Venkata KondaTransformer EncoderTransformer Architectures

  60. Beyond Importance: Interchange-Sobol Sensitivity Reveals Task-Specific Content Channels in Transformer Components

    Jun 12, 2026Yifeng Guo, Jin-Hong Du, Xiang ChenTransformer ArchitecturesImportance