Transformers

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

42 new papers

A weekly snapshot of new work published in Transformers.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Transformers.

Inside this field

Focused directions

1,208 papers

Latest in Transformers

Browse all topics →
CardsList

  1. AttentionBender: Manipulating Cross-Attention in Video Diffusion Transformers as a Creative Probe

    Apr 22, 2026Adam Cole, Mick GriersonHigh-Fidelity Video GenerationCross-Attention

  2. Explicit Dropout: Deterministic Regularization for Transformer Architectures

    Apr 22, 2026Vidhi Agrawal, Illia Oleksiienko, Alexandros IosifidisDropoutTransformer Attention

  3. DynamicRad: Content-Adaptive Sparse Attention for Long Video Diffusion

    Apr 22, 2026Yongji Long, Shijun Liang, Jintao Li +1Dynamic Sparse AttentionSpatial Attention

  4. LaplacianFormer:Rethinking Linear Attention with Laplacian Kernel

    Apr 22, 2026Zhe Feng, Sen Lian, Changwei Wang +5Quadratic AttentionSoftmax Attention

  5. Simplified Sparse Attention via Gist Tokens

    Apr 22, 2026Yuzhen Mao, Michael Y. Li, Emily B. FoxDynamic Sparse AttentionEfficient Long-Context Inference

  6. PASTA: A Patch-Agnostic Twofold-Stealthy Backdoor Attack on Vision Transformers

    Apr 21, 2026Dazhuang Liu, Yanqi Qiao, Rui Wang +2Vision TransformerBackdoor Attacks

  7. Cognitive Alignment At No Cost: Inducing Human Attention Biases For Interpretable Vision Transformers

    Apr 21, 2026Ethan KnightsSelf-Supervised Vision TransformersVision Transformer

  8. Benign Overfitting in Adversarial Training for Vision Transformers

    Apr 21, 2026Jiaming Zhang, Meng Ding, Shaopeng Fu +2Self-Supervised Vision TransformersAdversarial Training

  9. GRAFT: Geometric Refinement and Fitting Transformer for Human Scene Reconstruction

    Apr 21, 2026Pradyumna YM, Yuxuan Xue, Yue Chen +3Articulated ObjectsTransformer Architectures

  10. Volume Transformer: Revisiting Vanilla Transformers for 3D Scene Understanding

    Apr 21, 2026Kadir Yilmaz, Adrian Kruse, Tristan Höfer +23D Scene Understanding3D Object Understanding

  11. Multimodal Transformer for Sample-Aware Prediction of Metal-Organic Framework Properties

    Apr 21, 2026Seunghee Han, Jaewoong Lee, Jihan KimMolecular Property PredictionCryo-Electron Microscopy

  12. Multi-view Crowd Tracking Transformer with View-Ground Interactions Under Large Real-World Scenes

    Apr 21, 2026Qi Zhang, Jixuan Chen, Kaiyi Zhang +3Multi-Object TrackingMulti-View

  13. Multimodal embodiment-aware navigation transformer

    Apr 21, 2026Louis Dezons, Quentin Picard, Rémi Marsal +2Embodied NavigationObject Goal Navigation

  14. Nexusformer: Nonlinear Attention Expansion for Stable and Inheritable Transformer Scaling

    Apr 21, 2026Weijie Zhao, Mingquan Liu, Bolun Wang +4MixformerSoftmax Attention

  15. FG2^2-GDN: Enhancing Long-Context Gated Delta Networks with Doubly Fine-Grained Control

    Apr 21, 2026Pingwei Sun, Yuxuan Hu, Jianchao Tan +6Global Attention MechanismEfficient Long-Context Inference

  16. ConvVitMamba: Efficient Multiscale Convolution, Transformer, and Mamba-Based Sequence modelling for Hyperspectral Image Classification

    Apr 20, 2026Mohammed Q. AlkhatibHyperspectral Image ClassificationVision Transformer

  17. One Step Forward and K Steps Back: Better Reasoning with Denoising Recursion Models

    Apr 20, 2026Chris Cameron, Wangzheng Wang, Nikita Ivanov +3Denoising Diffusion Probabilistic ModelImage Denoising

  18. Towards Understanding the Robustness of Sparse Autoencoders

    Apr 20, 2026Ahson Saiyed, Sabrina Sadiekh, Chirag AgarwalAdversarial JailbreaksTransformer Architectures

  19. Linear-Time and Constant-Memory Text Embeddings Based on Recurrent Language Models

    Apr 20, 2026Tobias Grantner, Emanuel Sallinger, Martin FlechlDomain-Invariant Text EmbeddingsRecurrent Model

  20. Depth Registers Unlock W4A4 on SwiGLU: A Reader/Generator Decomposition

    Apr 20, 2026Ziyang LiuW4A4 QuantizationQuadratic Attention

  21. Decision-Aware Attention Propagation for Vision Transformer Explainability

    Apr 20, 2026Sehyeong Jo, Gangjae Jang, Haesol ParkVision TransformerTransformer Attention

  22. How Much Cache Does Reasoning Need? Depth-Cache Tradeoffs in KV-Compressed Transformers

    Apr 20, 2026Xiao WangKey-Value Cache CompressionKey-Value Cache

  23. OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models

    Apr 20, 2026Yiwei Zhang, Xuesong Chen, Jin Gao +5Autonomous Hard Drive DisassemblyVision-Language-Action Models

  24. Why Advanced Encoders Lag on Sparse Retrieval? The Answer and an Approach to Bridging Vocabulary Gaps

    Apr 20, 2026Zhichao Geng, Yang YangRetrieval LayerDynamic Sparse Attention

  25. Towards Generalizable Deepfake Image Detection with Vision Transformers

    Apr 19, 2026Kaliki V Srinanda, M Manvith Prabhu, Hemanth K Mogilipalem +4Explainable Deepfake DetectionSelf-Supervised Vision Transformers

  26. Capacity-Controlled Global Attention for Graph Transformers

    Apr 19, 2026Yang Liu, Dongxin Guo, Tom Zheng +3Heterogeneous Graph TransformersTransformer Attention

  27. Bit-Flip Vulnerability of Shared KV-Cache Blocks in LLM Serving Systems

    Apr 19, 2026Yuji Yamamoto, Satoshi MatsuuraKv-Cache BlocksSystematic Fp16 Divergence

  28. Decomposing the Depth Profile of Fine-Tuning

    Apr 19, 2026Jayadev BillaGradient FlowTransformer Layers

  29. NaviFormer: A Deep Reinforcement Learning Transformer-like Model to Holistically Solve the Navigation Problem

    Apr 18, 2026Daniel Fuertes, Andrea Cavallaro, Carlos R. del-Blanco +2Trajectory PredictionDeep Reinforcement Learning

  30. Open-TQ-Metal: Fused Compressed-Domain Attention for Long-Context LLM Inference on Apple Silicon

    Apr 18, 2026Sai VegasenaSystematic Fp16 DivergenceLLM Inference Efficiency

  31. The CTLNet for Shanghai Composite Index Prediction

    Apr 18, 2026Haibin JiaoMultivariate Time Series ForecastingHybrid Transformer

  32. Adaptive Computation Depth via Learned Token Routing in Transformers

    Apr 18, 2026Ahmed Abdelmuniem Abdalla MohammedTransformer ArchitecturesMultilayer Perceptrons

  33. SAVE: A Generalizable Framework for Multi-Condition Single-Cell Generation with Gene Block Attention

    Apr 18, 2026Jiahao Li, Jiayi Dong, Peng Ye +3Single-Cell Rna SequencingHigh-Fidelity Conditional Generation

  34. Closing the Theory-Practice Gap in Spiking Transformers via Effective Dimension

    Apr 17, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuNeuromorphic HardwareTransformer Architectures

  35. LACE: Lattice Attention for Cross-thread Exploration

    Apr 16, 2026Yang Li, Zirui Zhang, Yang Liu +1Large Language Model ReasoningParallel

  36. Stability and Generalization in Looped Transformers

    Apr 16, 2026Asher LabovichTransformer ArchitecturesFixed-Point Iteration

  37. AdaSplash-2: Faster Differentiable Sparse Attention

    Apr 16, 2026Nuno Gonçalves, Hugo Pitorro, Vlad Niculae +4Dynamic Sparse AttentionEfficient Long-Context Inference

  38. The Illusion of Equivalence: Systematic FP16 Divergence in KV-Cached Autoregressive Inference

    Apr 16, 2026Ranjith Chodavarapu, Lei XuSystematic Fp16 DivergenceKey-Value Cache Quantization

  39. Dispatch-Aware Ragged Attention for Pruned Vision Transformers

    Apr 16, 2026Seifeldin Abdellatif, Ahmad AlmasriVisual Token PruningSelf-Supervised Vision Transformers

  40. What Is the Minimum Architecture for Prolepsis? Early Irrevocable Commitment Across Tasks in Small Transformers

    Apr 16, 2026Éric JacopinTransformer ArchitecturesSmall Models

  41. Improving Sparse Autoencoder with Dynamic Attention

    Apr 16, 2026Dongsheng Wang, Jinsen Zhang, Dawei Su +1Improving Sparse AutoencodersDynamic Sparse Attention

  42. Expressivity of Transformers: A Tropical Geometry Perspective

    Apr 16, 2026Ye Su, Yong LiuTransformer AttentionMulti-Head Self-Attention

  43. FreqTrack: Frequency Learning based Vision Transformer for RGB-Event Object Tracking

    Apr 16, 2026Jinlin You, Muyu Li, Xudong ZhaoRgbtMulti-Object Tracking

  44. Attention-Weighted Value Projection for KV-Cache Compression

    Apr 13, 2026Samuel SalfatiKey-Value Cache QuantizationExtreme Low-Bit Quantization

  45. Relational Preference Encoding in Looped Transformer Internal States

    Apr 10, 2026Jan KirinPairwise PreferencesTransformer Encoder

  46. LSRM: High-Fidelity Object-Centric Reconstruction via Scaled Context Windows

    Apr 6, 2026Zhengqin Li, Cheng Zhang, Jakob Engel +1Feed-Forward 3D ReconstructionSurface Reconstruction

  47. HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attention

    Mar 30, 2026Yufei Xu, Fanxu Meng, Fan Jiang +11Dynamic Sparse AttentionEfficient Long-Context Inference

  48. ExFusion: Efficient Transformer Training via Multi-Experts Fusion

    Mar 30, 2026Jiacheng Ruan, Daize Dong, Xiaoye Qu +5Mixture-of-Experts ModelsTransformer Architectures

  49. GaussianGPT: Towards Autoregressive 3D Gaussian Scene Generation

    Mar 27, 2026Nicolas von Lützow, Barbara Rössle, Katharina Schmid +13D Scenes3D Object Understanding

  50. When Perplexity Lies: Generation-Focused Distillation of Hybrid Sequence Models

    Mar 27, 2026Juan Gabriel Kostelec, Qinghai GuoGenerative Pretrained TransformersGenerative Quality

  51. On the Ability of Transformers to Verify Plans

    Mar 20, 2026Yash Sarrof, Yupei Du, Katharina Stein +3Classical PlanningTransformer Architectures

  52. Dual Path Attribution: Efficient Attribution for SwiGLU-Transformers through Layer-Wise Target Propagation

    Mar 20, 2026Lasse Marten Jantsch, Dong-Jae Koh, Seonghyeon Lee +1Transformer AttentionModel Activations

  53. Learning When to Attend: Conditional Memory Access for Long-Context LLMs

    Mar 18, 2026Sakshi Choudhary, Aditya Chattopadhyay, Luca Zancato +4Efficient Long-Context InferenceQuadratic Attention

  54. The Phasor Transformer: Resolving Attention Bottlenecks on the Unit Circle

    Mar 18, 2026Dibakar SigdelTransformer AttentionUnified Transformer

  55. Human-like Object Grouping in Self-supervised Vision Transformers

    Mar 14, 2026Hossein Adeli, Seoyoung Ahn, Andrew Luo +3Self-SupervisedObject-Centric Learning

  56. AccelAes: Accelerating Diffusion Transformers for Training-Free Aesthetic-Enhanced Image Generation

    Mar 13, 2026Xuanhua Yin, Chuanzhi Xu, Haoxian Zhou +2Text-To-ImageImage Generation

  57. HiAP: A Multi-Granular Stochastic Auto-Pruning Framework for Vision Transformers

    Mar 12, 2026Andy Li, Aiden Durrant, Milan Markovic +1Self-Supervised Vision TransformersVision Transformer

  58. Low-latency Event-based Object Detection with Spatially-Sparse Linear Attention

    Mar 6, 2026Haiqing Hao, Zhipeng Sui, Rong Zou +4Object DetectionSpatial Attention

  59. Feature-level Interaction Explanations in Multimodal Transformers

    Mar 4, 2026Yeji Kim, Housam Khalifa Bashier Babiker, Mi-Young Kim +1Multimodal TransformersExplainability

  60. Retrievit: In-context Retrieval Capabilities of Transformers, State Space Models, and Hybrid Architectures

    Mar 3, 2026Georgios Pantazopoulos, Malvina Nikandrou, Ioannis Konstas +1Transformer ArchitecturesPositional Encoding