Transformers

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

42 new papers

A weekly snapshot of new work published in Transformers.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Transformers.

Inside this field

Focused directions

1,208 papers

Latest in Transformers

Browse all topics →
CardsList

  1. PAMNet: Cycle-aware Phase-Amplitude Modulation Network for Multivariate Time Series Forecasting

    May 1, 2026Yingbo Zhou, Yutong Ye, Zhiwei Ling +5Multivariate Time Series ForecastingGlobal Attention Mechanism

  2. Hierarchical Fault Detection and Diagnosis for Transformer Architectures

    Apr 30, 2026Sigma Jahan, Saurabh Singh Rajput, Tushar Sharma +1Fault DiagnosisTransformer Architectures

  3. Sampling two-dimensional spin systems with transformers

    Apr 30, 2026Piotr Białas, Piotr Korcyl, Tomasz Stebel +2Two-Dimensional Ising ModelSamplers

  4. ZAYAN: Disentangled Contrastive Transformer for Tabular Remote Sensing Data

    Apr 30, 2026Al Zadid Sultan Bin Habib, Tanpia Tasnim, Md. Ekramul Islam +1Tabular LearningContrastive Learning

  5. Sentiment Analysis of AI Adoption in Indonesian Higher Education Using Machine Learning and Transformer-Based Models

    Apr 30, 2026Happy Syahrul Ramadhan, Ahmad Sahidin Akbar, Karin Yehezkiel Sinaga +3Sentiment AnalysisEducation

  6. Spectral Dynamic Attention Network for Hyperspectral Image Super-Resolution

    Apr 30, 2026Tengya Zhang, Feng Gao, Lin Qi +2Hyperspectral Image Super-ResolutionDynamic Sparse Attention

  7. PINN-Cast: Exploring the Role of Continuous-Depth NODE in Transformers and Physics Informed Loss as Soft Physical Constraints in Short-term Weather Forecasting

    Apr 30, 2026Hira Saleem, Flora Salim, Cormac PurcellNumerical Weather PredictionConventional Physics-Informed Neural Networks

  8. Better Models, Faster Training: Sigmoid Attention for single-cell Foundation Models

    Apr 29, 2026Vijay Sadashivaiah, Georgios Dasoulas, Judith Mueller +1Softmax AttentionWireless Foundation Models

  9. Stochastic Scaling Limits and Synchronization by Noise in Deep Transformer Models

    Apr 29, 2026Andrea Agazzi, Giuseppe Bruno, Eloy Mosig García +2Transformer ArchitecturesStochastic Differential Equations

  10. Unifying Sparse Attention with Hierarchical Memory for Scalable Long-Context LLM Serving

    Apr 29, 2026Zihan Zhao, Baotong Lu, Shengjie Lin +8Dynamic Sparse AttentionGraphics Processing Unit Memory

  11. Exploring the Potential of Probabilistic Transformer for Time Series Modeling: A Report on the ST-PT Framework

    Apr 29, 2026Zhangzhi Xiong, Haoyi Wu, You Wu +3Time SeriesSpatio-Temporal Transformers

  12. Zero-Shot to Full-Resource: Cross-lingual Transfer Strategies for Aspect-Based Sentiment Analysis

    Apr 29, 2026Jakob Fehle, Nils Constantin Hellwig, Udo Kruschwitz +1Cross-Lingual TransferSentiment Analysis

  13. Star-Fusion: A Multi-modal Transformer Architecture for Discrete Celestial Orientation via Spherical Topology

    Apr 29, 2026May Hammad, Menatallh HammadSatelliteSpatio-Temporal Transformers

  14. Folding Tensor and Sequence Parallelism for Memory-Efficient Transformer Training & Inference

    Apr 29, 2026Vasu Shyam, Anna Golubeva, Quentin AnthonyTensor ProgramsParallel

  15. AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving

    Apr 28, 2026Zhongkai Yu, Haotian Ye, Chenyang Zhou +9Graphics Processing Unit MemoryQuadratic Attention

  16. PPG-Based Affect Recognition with Long-Range Deep Models: A Measurement-Driven Comparison of CNN, Transformer, and Mamba Architectures

    Apr 28, 2026Karim Alghoul, Hussein Al Osman, Abdulmotaleb El SaddikConvolutional Neural NetworksAffective Computing

  17. Carbon-Taxed Transformers: A Green Compression Pipeline for Overgrown Language Models

    Apr 28, 2026Ajmain Inqiad Alam, Palash Roy, Chanchal K. Roy +2Large Language Model CompressionContext Compression

  18. Investigation into In-Context Learning Capabilities of Transformers

    Apr 28, 2026Rushil Chandrupatla, Leo Bangayan, Sebastian LengIn-Context LearningIn-Context

  19. Barriers to Universal Reasoning With Transformers (And How to Overcome Them)

    Apr 28, 2026Oliver Kraus, Yash Sarrof, Yuekun Yao +2Transformer ArchitecturesChain-of-Thought Reasoning

  20. Emergent Self-Attention from Astrocyte-Gated Associative Memory Dynamics

    Apr 28, 2026Arnau Vivet, Alex ArenasHopfield NetworksAttractors

  21. QFlash: Bridging Quantization and Memory Efficiency in Vision Transformer Attention

    Apr 28, 2026Sehyeon Oh, Yongin Kwon, Jemin LeeQuadratic AttentionQuantization

  22. Transformer Approximations from ReLUs

    Apr 27, 2026Jerry Yao-Chieh Hu, Mingcheng Lu, Yi-Chen Lee +1Gumbel-Softmax RelaxationRectified Linear Unit

  23. Learning to Rotate: Temporal and Semantic Rotary Encoding for Sequential Modeling

    Apr 27, 2026Hailing Cheng, Daqi Sun, Xinyu LuRotary Position EmbeddingPositional Encoding

  24. UVTran: Accurate Hole-Filling Parameterization with Transformers

    Apr 27, 2026JunFeng ZhangB-SplineTransformer Architectures

  25. Salca: A Sparsity-Aware Hardware Accelerator for Efficient Long-Context Attention Decoding

    Apr 27, 2026Wang Fan, Wei Cao, Xi Zha +5Efficient Long-Context InferenceDynamic Sparse Attention

  26. Kwai Summary Attention Technical Report

    Apr 27, 2026Chenglong Chu, Guorui Zhou, Guowang Zhang +35Efficient Long-Context InferenceQuadratic Attention

  27. SolarTformer: A Transformer Based Deep Learning Approach for Short Term Solar Power Forecasting

    Apr 27, 2026Ankan Basu, Jyotiraditya Roy, Aditya Datta +2Energy ForecastingSolar

  28. SparseContrast: Dynamic Sparse Attention for Efficient and Accurate Contrastive Learning in Medical Imaging

    Apr 27, 2026Paarth Prasad, Ruchika MalhotraMedical ImagingChest X-Ray

  29. Architecture Determines Observability of Transformers

    Apr 27, 2026Thomas CarmichaelAutoregressive TransformersTransformer Architectures

  30. Graph Memory Transformer (GMT)

    Apr 26, 2026Nicola Zanarini, Niccolò Ferrari, Evelina LammaTransformer EncoderHeterogeneous Graph Transformers

  31. ELSA: Exact Linear-Scan Attention for Fast and Memory-Light Vision Transformers

    Apr 26, 2026Chih-Chung Hsu, Xin-Di Ma, Wo-Ting Liao +1Quadratic AttentionReal-Time Inference

  32. DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence

    Apr 26, 2026DeepSeek-AI, Anyi Xu, Bangcai Lin +316Efficient Long-Context InferenceDeepseek

  33. Transformer as an Euler Discretization of Score-based Variational Flow

    Apr 26, 2026Huadong LiaoTransformer ArchitecturesRepresentation Learning

  34. Rank, Head-Channel Non-Identifiability, and Symmetry Breaking: A Precise Analysis of Representational Collapse in Transformers

    Apr 26, 2026Giansalvo CirrincioneTransformer ArchitecturesBatch Normalization

  35. Hardware-Efficient Softmax and Layer Normalization with Guaranteed Normalization for Edge Devices

    Apr 26, 2026Dawon Choi, Hana Kim, Ji-Hoon KimBatch NormalizationGumbel-Softmax Relaxation

  36. Applications of the Transformer Architecture in AI-Assisted English Reading Comprehension

    Apr 26, 2026Ping LiTransformer Architectures

  37. Supernodes and Halos: Loss-Critical Hubs in LLM Feed-Forward Layers

    Apr 26, 2026Audrey Cherilyn, Houman SafaaiFeed-Forward NetworkTransformer Layers

  38. TEMPO: Transformers for Temporal Disease Progression from Cross-Sectional Data

    Apr 25, 2026Hongtao Hao, Joseph L. AusterweilPatient TrajectoriesBiomarker

  39. BSViT: A Burst Spiking Vision Transformer for Expressive and Efficient Visual Representation Learning

    Apr 25, 2026Hongxiang Peng, Dewei Bai, Hong QuSelf-Supervised Vision TransformersBinary Spikes

  40. Scaling Point-in-Time Language Models

    Apr 24, 2026Bryan Kelly, Semyon Malamud, Johannes Schwab +1Transformer EncoderLookahead

  41. Adaptive Head Budgeting for Efficient Multi-Head Attention

    Apr 24, 2026Bilal Faye, Abdoulaye Mbaye, Hanane Azzag +1Multi-Head Self-AttentionTransformer Architectures

  42. SpikingBrain2.0: Brain-Inspired Foundation Models for Efficient Long-Context and Cross-Platform Inference

    Apr 24, 2026Yuqi Pan, Jinghao Zhuang, Yupeng Feng +16Dynamic Sparse AttentionEfficient Long-Context Inference

  43. HubRouter: A Pluggable Sub-Quadratic Routing Primitive for Hybrid Sequence Models

    Apr 24, 2026Abhinaba BasuTransformer AttentionAdaptive Routing

  44. Learning Reactive Human Motion Generation from Paired Interaction Data Using Transformer-Based Models

    Apr 24, 2026Masato Soga, Ryuki TakebayashiHuman Motion GenerationMotion Generation

  45. Dissociating Decodability and Causal Use in Bracket-Sequence Transformers

    Apr 24, 2026Aryan Sharma, Cutter Dawes, Shivam RavalTransformer EncoderTransformer Architectures

  46. Universal Transformers Need Memory: Depth-State Trade-offs in Adaptive Recursive Reasoning

    Apr 23, 2026Grigory SapunovRecurrent ModelTransformer Architectures

  47. Beyond N-gram: Data-Aware X-GRAM Extraction for Efficient Embedding Parameter Scaling

    Apr 23, 2026Yilong Chen, Yanxi Xie, Zitian Gao +10N$-Grams FeaturesQuadratic Attention

  48. To See the Unseen: on the Generalization Ability of Transformers in Symbolic Reasoning

    Apr 23, 2026Nevena Lazić, Liam Fowl, András György +1Transformer EncoderTransformer Architectures

  49. Sculpt4D: Generating 4D Shapes via Sparse-Attention Diffusion Transformers

    Apr 23, 2026Minghao Yin, Wenbo Hu, Jiale Xu +24D GenerationMultimodal Diffusion Transformers

  50. EdgeFormer: local patch-based edge detection transformer on point clouds

    Apr 23, 2026Yifei Xie, Zhikun Tu, Tong Yang +2Spatial PatchingTransformer Architectures

  51. an interpretable vision transformer framework for automated brain tumor classification

    Apr 23, 2026Chinedu Emmanuel Mbonu, Tochukwu Sunday Belonwu, Okwuchukwu Ejike Chukwuogo +1Brain Tumor ClassificationSelf-Supervised Vision Transformers

  52. Hyperloop Transformers

    Apr 23, 2026Abbas Zeitoun, Lucas Torroba-Hennigen, Yoon KimTransformer ArchitecturesStable Transport

  53. Robust Cross-Domain WiFi Fall Detection via Physics-Driven Attention-Enhanced Transformers

    Apr 23, 2026Yingzhe Wang, Cunhua Pan, Ruijing Liu +4Robust Fall DetectionWifi

  54. Sparse Forcing: Native Trainable Sparse Attention for Real-time Autoregressive Diffusion Video Generation

    Apr 23, 2026Boxun Xu, Yuming Du, Zichang Liu +7Autoregressive Video DiffusionAutoregressive Video Generation

  55. The Recurrent Transformer: Greater Effective Depth and Efficient Decoding

    Apr 23, 2026Costin-Andrei Oncescu, Depen Morwani, Samy Jelassi +3Transformer ArchitecturesKey-Value Cache

  56. A Hybridizable Neural Time Integrator for Stable Autoregressive Forecasting

    Apr 22, 2026Brooks Kinch, Xiaozhe Hu, Yilong Huang +6Autoregressive TransformersLong-Horizon Forecasting

  57. Preconditioned DeltaNet: Curvature-aware Sequence Modeling for Linear Recurrences

    Apr 22, 2026Neehal Tumma, Noel Loo, Daniela RusRecurrent ModelSequence Modeling

  58. climt-paraformer: Stable Emulation of Convective Parameterization using a Temporal Memory-aware Transformer

    Apr 22, 2026Shuochen Wang, Nishant Yadav, Joy Merwin Monteiro +1Global Climate ModelsEmulators

  59. Working Memory Constraints Scaffold Learning in Transformers under Data Scarcity

    Apr 22, 2026Pranava Madhyastha, Dagmar AdamcovaTransformer ArchitecturesWorking Memory

  60. LayerTracer: A Joint Task-Particle and Vulnerable-Layer Analysis framework for Arbitrary Large Language Model Architectures

    Apr 22, 2026Yuhang Wu, Qinyuan Liu, Qiuyang Zhao +1Transformer LayersCircuit Tracing