Transformers

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

42 new papers

A weekly snapshot of new work published in Transformers.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Transformers.

Inside this field

Focused directions

1,208 papers

Latest in Transformers

Browse all topics →
CardsList

  1. Pixel-Space Diffusion Transformers

    Jul 20, 2026Renye Yan, Jikang Cheng, You Wu +8Multimodal Diffusion TransformersVisual Tokens

  2. A Weisfeiler-Leman Characterization of Global-Attention Graph Transformers for Mixed-Integer Linear Programs

    Jul 20, 2026Md Abrar Jahin, Craig A. Knoblock, Jay PujaraQuadratic AttentionTransformer Architectures

  3. An Explainable FFT-Based Spatial-Frequency Fusion Framework for Deepfake Detection

    Jul 19, 2026Pamela Kirui, Cho Hyuk, Qingzhong Liu +1Explainable Deepfake DetectionDeepfake Detection

  4. Kernelized Linear Attention: Breaking the Capacity Wall with Symmetric Cones

    Jul 19, 2026Ayoub Ghriss, Sourav ChakrabortyQuadratic AttentionDense Associative Memory

  5. Visible-Light Imaging Diagnosis of Neutral Particle Emission Tomography in the Tokamak Divertor: An Efficient Transformer-based Surrogate Model

    Jul 19, 2026Xiao Wang, Hao Si, Qiang Chen +8Plasma Shape ControlMultimodal Diffusion Transformers

  6. Staged Depth-Pruning Distillation of a Flow-Matching Text-to-Speech Teacher: A Compact Hindi Speech Synthesizer

    Jul 19, 2026Sivateja TrikutamZero-Shot Text-To-SpeechTransformer Architectures

  7. Histopathological Spectrum-Guided Prostate Stratification via Segmentation-Assisted Diagnostic Transformer

    Jul 19, 2026Leyang Li, Lihua Chen, Huangang Hu +7Lesion SegmentationComputational Pathology

  8. The Geometry of Semantic Space: A Continuous Geometric Framework for the Transformer Architecture

    Jul 19, 2026Zhihua LiangTransformer ArchitecturesRotary Position Embedding

  9. Physics-Guided Masked Multi-Task Network for Edge-Friendly Battery Health Diagnostics from Sto-chastically Fragmented Charging Profiles

    Jul 19, 2026Shuhao Chen, Tianyu Shi, Chengyi TuLithium-Ion BatteriesUseful Life Prediction

  10. Dynamic Loss Balancing for Joint SOH and RUL Prediction of Lithium-Ion Batteries via a Rotary SOH-Injected Prior Battery Transformer

    Jul 19, 2026Shuhao Chen, Tianyu Shi, Yiwen Huang +1Lithium-Ion BatteriesUseful Life Prediction

  11. What does a Bayes-filtered transformer believe? A predictive Monte Carlo approach

    Jul 19, 2026Afiq Abdillah Effiezal Aswadi, Haotong Ma, Susan WeiPosteriorNext-Token Prediction

  12. HyBDM: Multi-Scale Hybrid Experts for Time Series Forecasting with Bidirectional Dependency Modeling

    Jul 18, 2026Wenqiang Ma, Chen Cheng, Xue Cheng +1Multivariate Time Series ForecastingLong-Range Temporal Dependencies

  13. First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers

    Jul 18, 2026Irina Piontkovskaia, Sergey NikolenkoTransformer ArchitecturesUnseen-Task Adaptation

  14. SpecLA: Efficient Speculative Decoding for Linear-Attention Models

    Jul 18, 2026Zhibin Wang, Xuying Han, Zhaohua Yang +5Speculative DecodingQuadratic Attention

  15. FVAttn: Adaptive Sparse Attention with Runtime Load Balancing for Video Generation

    Jul 17, 2026Hao Liu, Chenghuan Huang, Ye Huang +7Dynamic Sparse AttentionPre-Trained Video Diffusion Models

  16. Frontier Language Models Struggle to Copy: Text Can Be Better Viewed in 2D

    Jul 17, 2026Haodong Wen, Yiran Zhang, Yingfa Chen +1Positional EncodingTransformer Layers

  17. In-context learning of closed form solution to simple linear regression task using transformer with linear self-attention

    Jul 17, 2026Katsuyuki HagiwaraIn-Context LearningTransformer Architectures

  18. A Benchmark for Electrical Load Forecasting Across Grid Levels: Time-Series Transformers Outperform Established Methods

    Jul 17, 2026Matthias Hertel, Sebastian Pütz, Jonathan Kolar +3Electricity Price ForecastingTime Series Foundation Models

  19. DiTango: Cost-Effective Parallel Diffusion Generation with Selective Attention State Reuse

    Jul 17, 2026Yuyang Chen, Runxin Zhong, Zan Zong +3Multimodal Diffusion TransformersParallel

  20. Boundary-Seeking GAN-Augmented TabTransformer for Adversarially Robust Intrusion Detection

    Jul 17, 2026Raihan Sultan Pasha Basuki, Aliyah KurniasihIntrusion DetectionGenerative Adversarial Network

  21. T^2MLR: Transformer with Temporal Middle-Layer Recurrence

    Jul 16, 2026Ziyang Cai, Xingyu Zhu, Yihe Dong +2Efficient Latent ReasoningTransformer Architectures

  22. Towards Hierarchical Structure Understanding of Newspaper Images

    Jul 16, 2026William Mocaër, Solène Tarride, Thomas Constum +7HierarchicalHeadlines

  23. FlashDecoder: Real-Time Latent-to-Pixel Streaming Decoder with Transformers

    Jul 16, 2026Minguk Kang, Suha KwakLarge Language Model DecoderTransformer Architectures

  24. Autoregressive Modeling of Film with Applications in Video Montage

    Jul 16, 2026Marcelo Sandoval-Castañeda, Fabian Caba Heilbron, Shiry Ginosar +4Autoregressive Video GenerationFilms

  25. Beyond Implicit Force: Evaluating Explicit Force-Torque Proxies in Action Chunking with Transformers

    Jul 16, 2026King Hang Wong, Lingqiao Liu, Feras DayoubActuatorTeleoperation

  26. Screening Is Effective for Visual Recognition

    Jul 15, 2026Shunya Shimomura, Kazuhiro HottaVision TransformerSoftmax Attention

  27. MxGPS: Multiplex Graph Transformers for a Power Grid Foundation Model

    Jul 15, 2026Charilaos Papaioannou, Ioannis Tsantilas, Dimitris Giannakakos +9Ac Optimal Power FlowHeterogeneous Graph Transformers

  28. DeepLoop: Depth Scaling for Looped Transformers

    Jul 15, 2026Shuzhen Li, Yifan Zhang, Jiacheng Guo +2Transformer ArchitecturesSearch Depth

  29. 2D Rotary Position Embedding for Scene Text Recognition with Transformers

    Jul 15, 2026Zobeir RaisiScene Text RecognitionRotary Position Embedding

  30. Adaptive Filtering of the KV Cache: Diagnosing and Correcting Structural-Role Bias in LLM Inference

    Jul 14, 2026Soumil MandalKey-Value Cache EvictionKey-Value Cache

  31. Toward Localizing and Repairing Bias in Transformer Attention Heads

    Jul 14, 2026Sigma JahanTransformer AttentionTransformer Architectures

  32. AVQ-Attention: Adaptive Vector-Quantized Attention

    Jul 14, 2026Winfried van den dool, Patrick Forré, Amir Habibian +2Softmax AttentionCodebooks

  33. A JoLT for the KV Cache: Near-Lossless KV Cache Compression via Joint Tucker and JL-Residual Allocation for LLMs

    Jul 14, 2026Rahul Krishnan, Volker SchulzKey-Value Cache CompressionKey-Value Cache

  34. Do We Really Need Transformers for Global Spatial Information Extraction in Traffic Forecasting?

    Jul 14, 2026Qihang Zhang, Siyao Zhang, Letao Kang +3Accurate Traffic ForecastingSpatial Attention

  35. Forgetful Attention: A Trainable Support-Vector Memory with Certified Selection and Exact Unlearning

    Jul 13, 2026Vishwajith RameshSoftmax AttentionForgetting

  36. Invariant Learning Dynamics of Transformers in Inductive Reasoning Tasks

    Jul 13, 2026Tiberiu Musat, Tiago Pimentel, Nicolas Zucchet +1Transformer ArchitecturesTrainable Parameter

  37. From Expressivity to Sample Complexity: Narrow Teachers for Transformers via C-RASP

    Jul 13, 2026Michael Rizvi-Martel, Satwik Bhattamishra, Guillaume Rabusseau +1Transformer ArchitecturesSample Complexity

  38. Higher-Order Cell Tracking Transformer

    Jul 13, 2026Jordão Bragantini, Ilan Theodoro, Loïc A. RoyerTransformer ArchitecturesLineages

  39. Sparse Inter-Layer Dependencies of Transformer FFN Neurons

    Jul 13, 2026Johannes Knittel, Hanspeter PfisterFeed-Forward NetworkTransformer Architectures

  40. RepTran: Search-Based Repair of Transformer Models

    Jul 13, 2026Yuta Ishimoto, Paolo Arcaini, Fuyuki Ishikawa +3Transformer ArchitecturesArtificial Intelligence Systems

  41. Desc++: Efficient Descriptor Enhancement for Data Association in Existing Visual SLAM Systems

    Jul 13, 2026Ting-Wei Ou, Huang-Ting Lin, Kuu-Young YoungOrb-Slam2Simultaneous Localization And Mapping

  42. Controlling Motion Transfer in Diffusion Transformers via Attention Heads

    Jul 13, 2026Sunyoung Jung, Jiwoo Park, Yoonseok Choi +3Motion-Focused RepresentationsTransformer Architectures

  43. LiteTopK: Exploiting the Curse of Dimensionality for a Fused Indexer-TopK Kernel in Long-Context Sparse Attention

    Jul 13, 2026Ziqi Yin, Jianyang Gao, Peiqi Yin +2Dynamic Sparse AttentionTop-K

  44. MMA-Former: Multi-Window Mixture-of-Head Attention Transformer for Adaptive PNI Prediction in 3D MRI

    Jul 13, 2026Youngung Han, Induk Um, Kyeonghun Kim +9Magnetic Resonance ImagingProgressive Attention Fusion

  45. Incremental Transformer for Surrogate-Based Inverse Design of Geopolymer Mixtures

    Jul 12, 2026Giansalvo Cirrincione, Filippo GrassiaGenerative Inverse DesignInverse Design

  46. Is Energy Guidance All You Need? Training-Free Norm Injection for Driving World Models

    Jul 12, 2026Xiyan Su, Frank Diermeyer, Markus LienkampVideo World ModelsWorld Models

  47. LayerNorm as Implicit Gain Control in Looped Transformers

    Jul 12, 2026Matthias M. M. BuehlmaierBatch NormalizationTransformer Layers

  48. From Self-Attention to Connection Laplacian: A Unified Operator View of Transformers

    Jul 12, 2026Binbin Lin, Wei Chen, Yalun Li +3Transformer AttentionTransformer Architectures

  49. AutoNorm: Understanding Adaptive Normalization in Transformers through Differentiable Gating

    Jul 12, 2026Piyush Kaushik Bhattacharyya, Divyanshu Rai, Swastik Singh +3Batch NormalizationNormalization

  50. The RG-Flow Transformer: Encoding Scale-Free Dynamics in Scarce EEG

    Jul 11, 2026Dibakar SigdelSleep Stage ClassificationTransformer Architectures

  51. BAT-RM: A Boundary-Aware Transformer with Region-Aware Multi-Directional Mamba for Clinically Deployed Cervical Cancer Radiotherapy Auto-Contouring

    Jul 11, 2026Istiak Ahmed, Kazi Shahriar Sanjid, Galib Ahmed +16RadiotherapyContours

  52. Gradient-Skipping Relevance Propagation for Efficient Explainability of Vision Transformers

    Jul 11, 2026Christopher Buratti, Michele Marchetti, Federica Parlapiano +3Vision TransformerExplainability

  53. MLPs are Hebbians: Constructing Efficient Fact-Storing MLPs for Transformers

    Jul 10, 2026Roberto Garcia, Jerry Liu, Ronny Junkins +3Large Language Model MemoryTransformer Architectures

  54. Do Transformer Temporal Heads and Post-Pooling Motion Gates Help CorrNet-based CSLR? An Empirical Study

    Jul 10, 2026Lisi Wang, Zhidong Xiao, Jianjun PengSign Language RecognitionTransformer Encoder

  55. PAC-ACT: Post-training Actor-Critic for Action Chunking Transformers

    Jul 10, 2026Yujie Pang, Zudong LiAction ChunksSoft Actor-Critic

  56. CoCoT-EEG: Contrastive-Pretrained Multiscale Convolutional Transformer for EEG Decoding

    Jul 10, 2026Gabriel Mahuas, Victoria Shevchenko, Ugo Tanielian +2Electroencephalography Foundation ModelsTransformer Encoder

  57. STEEL: Sparsity-Aware Fused Attention for Energy-Efficient Long-Sequence Inference on AMD's XDNA NPU

    Jul 10, 2026Victor J. B. Jung, Gagandeep Singh, Joseph Melber +3Neural Processing UnitsDynamic Sparse Attention

  58. TSR-Ego: Temporally Guided Stereo Refinement Framework for Egocentric 3D Human Pose Estimation

    Jul 10, 2026Md Mushfiqur Azam, John Quarles, Kevin Desai3D Pose EstimatorStereo Matching

  59. COBS: Cumulant Order Block Sparse Attention

    Jul 10, 2026Alexander Tian, Aditya Ghai, Sanjit Neelam +2Dynamic Sparse AttentionLongbench

  60. MOSAIC: Adaptive Inter-layer Composition for Efficient Heterogeneous Vision-Language Models

    Jul 10, 2026Yuncheng Yang, Feiyang Ye, Shixian Luo +7Large Vision Language ModelsReal-Time Inference