Transformers

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

42 new papers

A weekly snapshot of new work published in Transformers.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Transformers.

Inside this field

Focused directions

1,206 papers

Latest in Transformers

Browse all topics →
CardsList

  1. vToken: Token-Level Virtualization for Reclaimable KV Caches

    Aug 13, 2026Yuanhang Gao, Xiangrui Yang, Yuanfeng Chen +4Key-Value CacheGraphics Processing Unit Memory

  2. Virtual Temperature Sensors in Power Transformers Using Neural Ordinary Differential Equations

    Aug 13, 2026Berk Hadzhamolla, Alexander Johannes Stasik, Signe Riemer-SørensenUnified TransformerNeural Ordinary Differential Equations

  3. UniCon-Former: Unified Convolution Transformer is All You Need for Hand Gesture Recognition

    Aug 13, 2026Mallika Garg, Debashis Ghosh, Pyari Mohan PradhanHand Gesture RecognitionHybrid Transformer

  4. MergeOver: Post-Training Token Merging for Recursive Vision Transformers

    Aug 13, 2026Junseo Kim, Uraz Odyurt, Amirreza YousefzadehSelf-Supervised Vision TransformersVision Transformer

  5. Spatially-Grounded Text-to-Video Generation via Inference-Time Gradient-Free Optimization

    Aug 13, 2026Guillaume Jeanneret, Mathis Koroglu, Hugo Caselles-Dupré +2Video Temporal GroundingTransformer Architectures

  6. Semantic Steering for Controllable Generation: Tuning-Free Concept Erasure in Multimodal Diffusion Transformers

    Aug 13, 2026Qiao Li, Xiaomeng Fu, Yuanshu Zhao +3Multimodal Diffusion TransformersConcept Erasure

  7. SCOPE: Subspace Clustering with Online Per-Head Top-K Estimation for Sparse Video Attention

    Aug 13, 2026Qi Zhao, Qirui Li, Hanlin Tang +10Dynamic Sparse AttentionPre-Trained Video Diffusion Models

  8. On the Expressive Power of Transformers

    Aug 13, 2026Phokion Kolaitis, Rik SenguptaTransformer ArchitecturesExpressivity

  9. Class Activation Mapping in Explainable Computer Vision: A Method-Centered Review of CNN, Transformer, and Foundation-Model-Era Visual Explanations

    Aug 12, 2026AmirHossein Eshghi, Hamid Saadatfar, Seyyed Ali Hoseini +2Etsi-Compliant Class Activation MapExplainability

  10. Geometric and Behavioral Stratification in Transformer Residual Streams

    Aug 12, 2026Nelson GudaTransformer Residual StreamsResidual Stream

  11. A Neighborhood Attention Transformer Network for Enhanced 3D Segmentation of the Left Anterior Descending Artery

    Aug 12, 2026Rafi Ibn Sultan, Chengyin Li, Yiannos Demetriou +6Whole-Heart SegmentationRadiotherapy

  12. HSTGFormer: Hyper Spatial-Temporal Graph Transformer for 3D Human Pose Estimation

    Aug 12, 2026Ruochen Li, Shuang Chen, Wenke E +23D Pose EstimatorSpatio-Temporal Transformers

  13. LoSA: Near-Lossless Sparse Attention for Training-Free Video Diffusion Acceleration

    Aug 12, 2026Enhuai Liu, Yunke Wang, Yutong Wang +2Dynamic Sparse AttentionPre-Trained Video Diffusion Models

  14. Disentangling the Expressivity of RoPE

    Aug 12, 2026Selim Jerad, Anej Svete, Jiaoda Li +1Rotary Position EmbeddingExpressivity

  15. Lapis: Laplacian Spiking Attention via First-Spike Timing and Membrane Leakage

    Aug 12, 2026Kaiwen Tang, Jiaqi Zheng, Zixuan Zhu +3Binary SpikesSoftmax Attention

  16. Small-Scale Experiments: Are We There Yet?

    Aug 12, 2026Nicholas Lourie, Kyunghyun Cho, Karen Ullrich +1Scaling LawsHyperparameter Optimization

  17. LabelFusion-TS: Fusing Large Language Models, Transformer Encoders, and Financial Time Series for Monetary-Policy Stance Classification

    Aug 12, 2026Michael Schlee, Fabian Lukassen, Christoph WeisserFinancial Sentiment AnalysisModernbert

  18. G0.5: One Autoregressive Stream for Robot Reasoning and Action

    Aug 12, 2026Yicheng Liu, Zibin Dong, Baijun Ye +24Vision-Language-Action ModelsExecutable Robot Behavior

  19. Low-Interaction-Rank Learning: Unifying Multiplicative Dual-Encoder Heads

    Aug 12, 2026Zijian Zhao, Sen LiEncoder-Decoder ArchitecturesLow-Rank Factorization

  20. Three Tokens Force Exponential Feature Rank in Nonnegative Kernel Attention

    Aug 11, 2026Vicente OpazoQuadratic AttentionGumbel-Softmax Relaxation

  21. ODE-Based Transformer Decoders for Iterative Sign Language Translation

    Aug 11, 2026Tuğçe Kızıltepe, Hacer Yalim KelesSign Language RecognitionTransformer Encoder

  22. A Quantum Roadmap for Softmax Attention: Exact Born-Rule Analogs for Softmax Attention on the Probability Simplex

    Aug 11, 2026Eric A. F. Reinhardt, Adam J. HauserSoftmax AttentionQubit

  23. TACTICL: Task-Aware Compression of Tabular ICL Models

    Aug 11, 2026Mykhailo Koshil, Matthias Feurer, Katharina EggenspergerTabular Foundation ModelsContext Compression

  24. Neural Introspection Gating for Adaptive KV-Cache Reuse in Vision-Language-Action Models

    Aug 11, 2026Zhijie Wu, Kento Kawaharazuka, Kei OkadaCacheVisual Tokens

  25. Assessing Reliability of BERT-Based Models on Question Answering Tasks

    Aug 11, 2026Pooja Yadav, Priyanka Harjule, Basant Agarwal +1Large Language Model ReliabilityBert-Based Models

  26. Off-Axis, On Purpose: Where a Transformer Computes Concepts and Why it Does So

    Aug 10, 2026Mark OskinTransformer Architectures

  27. Training-Free Universal Approximation by Prompting Random Transformers

    Aug 10, 2026Alexander Hsu, Rongjie LaiSoftmax AttentionApproximation

  28. MixFormer: Linear Transformer with Mixture of Memory Experts

    Aug 10, 2026Yu Guo, Lei DuanEfficient Long-Context InferenceSequence Modeling

  29. Why Post-Norm Transformers Collapse: Attention Amplification and Gradient Repair Failure

    Aug 10, 2026Xingjian Wang, Qingyu Han, Xiaodong Luo +1Batch NormalizationTransformer Encoder

  30. Linearized 2-Simplicial Attention

    Aug 10, 2026Aritra Das, Dhruman Gupta, Debayan GuptaQuadratic AttentionDownstream Reasoning

  31. SwiftQK: Fast and Communication-Efficient Tensor Parallelism for Query-Key Normalization

    Aug 10, 2026Gyudong Kim, Wonjun Han, Young Geun KimTensor ProgramsQuadratic Attention

  32. Full-bandwidth transformer

    Aug 9, 2026Xi Wang, Ziyang Cai, Zheng Zhan +5Autoregressive TransformersTransformer Architectures

  33. DistillCache: KL-Guided Adaptive KV-Cache Eviction for Memory-Efficient LLM Inference

    Aug 9, 2026Asaad AlthoubiKey-Value Cache EvictionCache

  34. Deployable Per-Instance Multi-Layer Activation Steering for Large Language Models

    Aug 9, 2026Muhammad Faishal Adly Nelwan, Alfan Farizki WicaksonoActivation SteeringModel Activations

  35. RippleKV: Cross-Layer KV Cache Allocation via Perturbation Propagation

    Aug 9, 2026Dongjie Xu, Kai Qian, Julius +6Key-Value Cache CompressionCache

  36. Position Encoding in Transformers: From Absolute and Relative Methods to Rotary Position Embeddings and Long-Context Scaling

    Aug 9, 2026Jiguo LiPositional EncodingRotary Position Embedding

  37. Where Is the Bee? Detecting Tiny Pollinators with a Single Collaborative-Head Transformer

    Aug 9, 2026Junsu Kim, Seungryul BaekBeesKeyframe Selection

  38. Rethinking Attention Locality in Spiking Transformers

    Aug 9, 2026Zeqi Zheng, Zizheng Zhu, Yuping Yan +3Transformer AttentionBinary Spikes

  39. Gated Spatial Redundancy Projection for Pathology Transformer Attentions

    Aug 8, 2026Zhiyuan Yang, Jiahao Cheng, Vincent Quoc-Huy Trinh +1Whole-Slide ImagesComputational Pathology

  40. RotaryQuant: Fitting 120B MoE Models on Consumer Hardware via Fused Compressed-Space Attention

    Aug 8, 2026Anthony. Lui, Mohamed. Elsaied, N. P. SavaniMixed-Precision QuantizationQuadratic Attention

  41. Spectral Outliers Reveal Dominant Learned Structure in Transformer Attention

    Aug 8, 2026Kasun Dewage, Marianna Pensky, Suranadi De Silva +1Transformer AttentionOutliers

  42. Transformer Geometry Observatory TGO-IV: Developmental Topology Observatory

    Aug 7, 2026Kaustubh Kapil, Kishor P. UplaTransformer ArchitecturesTopological Data Analysis

  43. Fluid-DiT: Graph-Free Diffusion Transformers for Fluid Flow Simulations Learning

    Aug 7, 2026Shentong Mo, Guolin KeFluid DynamicsMultimodal Diffusion Transformers

  44. Transformers Struggle to Use Their Emergent World Models: Revisiting the Tower of Hanoi, and the Illusion of Thinking

    Aug 7, 2026Devin Pereira, Willem ZuidemaLarge Language Model ReasoningPuzzle

  45. Autonomy-of-Heads: Data-Free Sparse Attention from Frozen Query-Key Geometry

    Aug 7, 2026Yehan Yang, Junyuan Shang, Yang Li +3Dynamic Sparse AttentionLLM Inference Efficiency

  46. Investigating Quantum-Embedded Transformers on Classical Datasets for Cross-Modality Classification

    Aug 7, 2026Hao-Yuan ChenHybrid Quantum Neural NetworksMultimodal Classification

  47. Graph Machine: Exploring Edge Mechanisms as an Inductive Bias

    Aug 7, 2026Lintai HouHeterogeneous Graph TransformersUnified Inductive Bias

  48. Faster Query-Key Learning Sharpens Attention in Self-Attention Models

    Aug 7, 2026Rahul Vashisht, Harish G. RamaswamySoftmax AttentionModel Interpretability

  49. HSMLA: Hierarchical Softmax Multi-scale Linear Attention for Efficient Vision Transformers

    Aug 7, 2026Dong Liu, Yanxuan Yu, Renata Borovica-Gajic +2Self-Supervised Vision TransformersVision Transformer

  50. Retrofitting Linear Attention into Diffusion Language Models

    Aug 6, 2026Jinha Kim, Younghun Roh, Jaeyeon KimDiffusion Language ModelsQuadratic Attention

  51. Beyond Attention: Signed Integrated Gradients Attribution in a BiomeGPT-Style Microbiome Transformer

    Aug 6, 2026Oren NelsonTraining Data AttributionTransformer Attention

  52. Is Self-Pretraining really useful to improve diagnosis in medical Time Series?

    Aug 6, 2026Omar Coser, Antonio Orvieto, Paolo Soda +1Time-Series ClassificationSelf-Supervised

  53. Beyond Sequence Order: Syntax-Informed Positional Embeddings for Transformers

    Aug 6, 2026Haris Riaz, Hyungji Kim, Mihai SurdeanuPositional EncodingTransformer Architectures

  54. MACRO: Markov Chain Routing of Transformer Layers

    Aug 6, 2026Paweł Batorski, Abtin Pourhadi, Akylgali Aitaza +2Large Language Model RoutingTransformer Layers

  55. Runtime Observability for Heterogeneous Attention Memory

    Aug 6, 2026Fanzhe Wei, Li Liu, Ziyang Wang +1Quadratic AttentionKey-Value Cache

  56. One Ranking, Any Budget: Matryoshka Evidence-to-Context Frame Selection for Long-Video Understanding

    Aug 6, 2026Wang Chen, Yu Chen, Xiang Wang +3Matryoshka Representation LearningRelevance

  57. QEvict: Recoverable Quantized KV Eviction for Attention-Drift-Robust Long-Context Decoding

    Aug 5, 2026Ayushman Garg, Akshita Gupta, Shaswata Bhattacharya +3Key-Value Cache EvictionKey-Value

  58. RepairFormer: Automated Repair of Structured Inputs Using Transformers

    Aug 5, 2026Ovi Paul, Tom J King, Ali ShokriCorrupted ObservationsCode Refactoring

  59. A Survey of Adversarial Efficiency Degradation for Vision Transformer by Exploiting Input-adaptive Optimization

    Aug 5, 2026Anadi Goyal, Nandish Chattopadhyay, Anupam Chattopadhyay +1Vision TransformerVisual Token Pruning

  60. Training-Free Hashing-Based Attention via Binary Principal Components

    Aug 5, 2026Daohai Yu, Zhanpeng Zeng, Keyu Chen +6Dynamic Sparse AttentionQuadratic Attention