Transformers

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

42 new papers

A weekly snapshot of new work published in Transformers.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Transformers.

Inside this field

Focused directions

1,209 papers

Latest in Transformers

Browse all topics →
CardsList

  1. MoECa: Aligning Feature Reuse with Expert Decomposition in Diffusion Transformers

    Jun 14, 2026Maoliang Li, Haojing Chen, Jiayu Chen +4Mixture-Of-Expert InferenceMultimodal Diffusion Transformers

  2. Controlled Dynamics Attractor Transformer

    Jun 13, 2026Cheng Zhang, Minnan Luo, Zesheng Yang +3Heterogeneous Graph TransformersAttractors

  3. Beyond Layer Importance in Layer-wise Sparsity: An Inter-Layer Perturbation-Absorption Perspective

    Jun 13, 2026Tao Jing, Ningxin Wu, Chen Kang +3SparsityLarge Language Model Compression

  4. Exploring a Layer-Wise Design Space for KV Cache Eviction

    Jun 13, 2026Chao Fei, Kaihua Liang, Hanzhi Hu +4Key-Value Cache CompressionCache

  5. Transformers Learn the Mestre-Nagao Heuristic

    Jun 13, 2026Pranav Venkata KondaTransformer EncoderTransformer Architectures

  6. Beyond Importance: Interchange-Sobol Sensitivity Reveals Task-Specific Content Channels in Transformer Components

    Jun 12, 2026Yifeng Guo, Jin-Hong Du, Xiang ChenTransformer ArchitecturesImportance

  7. RATS! Patches Talk Through Registers: Emergent Parts in Register Attention Transformers

    Jun 12, 2026Timing Yang, Predrag Neskovic, Jansen Seheult +4Self-Supervised Vision TransformersTransformer Attention

  8. Semidirect Fourier Delta Attention: Phase-Controlled Delta Memory with Constructive Chunk-WY Kernels

    Jun 12, 2026Tiantian ZhangQuadratic AttentionRecurrent State

  9. Zero-shot generalization of transformer neural operators to larger domains

    Jun 12, 2026Armand de Villeroché, Sibo Cheng, Vincent Le Guen +5Neural OperatorsTransformer Architectures

  10. A theoretical model for task routing in mixture-of-expert transformers

    Jun 12, 2026Vinoth Nandakumar, Yongli Xiang, Yunzhi Yao +2Mixture-of-Experts ModelsTransformer Layers

  11. How Linear Is a Transformer Feed-Forward Block? Per-Block Linear Recoverability Is Learned, Not Architectural

    Jun 12, 2026Stuart WhippFeed-Forward NetworkTransformer Architectures

  12. Zero-Shot Neural Priors for Generalizable Cross-Subject and Cross-Task EEG Decoding

    Jun 12, 2026Baimam Boukar Jean Jacques, Brandone Fonya, Nchofon Tagha Ghogomu +2Electroencephalography Spatial Super-ResolutionInter-Subject Variability

  13. Binary Black Hole Parameter Estimation with Hybrid CNN-Transformer Neural Networks

    Jun 11, 2026Panagiotis N. Sakellariou, Spiros V. Georgakopoulos, Sotiris Tasoulis +1Gravitational WavesBinary Black Hole Mergers

  14. Explaining RhythmFormer: A Systematic XAI Analysis of Periodic Sparse Attention for Remote Photoplethysmography

    Jun 11, 2026Louis Chen, Torbjörn E. M. NordlingRemote PhotoplethysmographyPhysiological Data

  15. GeoHAT: Geometry-Adaptive Hybrid Action Transformer for Mobile Manipulation

    Jun 11, 2026Xiangyu Zhu, Renjun Wu, Luzhou Ge +2GeoqueryTransformer Architectures

  16. MiniMax Sparse Attention

    Jun 11, 2026Xunhao Lai, Weiqi Xu, Yufeng Yang +14Dynamic Sparse AttentionLLM Inference Efficiency

  17. A green solvent screening tool for emerging materials via uncertainty aware, transformer enhanced transfer learning

    Jun 11, 2026Ioannis Kouroudis, Simon Ternes, Zhaosu Gu +5Previous Machine-Learning-Based Implicit SolventData-Driven Materials Discovery

  18. YOLO-AMC: An Improved YOLO Architecture with Attention Mechanisms for Building Crack Detection

    Jun 11, 2026Ching-Yu Tsai, Chia-Min Lin, Chih-Hsiang Yang +2Yolov8-Nano ModelGlobal Attention Mechanism

  19. The Signs Were Always There: Training-Free Concept Detection and Steering in Raw Transformer Dimensions

    Jun 10, 2026Varun Reddy NalagatlaTransformer ArchitecturesMultimodal Transformers

  20. On Subquadratic Architectures: From Applications to Principles

    Jun 10, 2026Anamaria-Roberta Hartl, Levente Zólyomi, David Stap +6Sequence ModelingNeural Architectures

  21. SpikeDecoder: Realizing the GPT Architecture with Spiking Neural Networks

    Jun 10, 2026Claas Beger, Florian Walter, Alois KnollSpiking Neural NetworksTransformer Encoder

  22. Vision Transformers for Face Recognition Need More Registers

    Jun 10, 2026Tahar Chettaoui, Guray Ozgur, Eduarda Caldeira +2Vision TransformerTransformer Architectures

  23. Architecture-Aware Reinforcement Learning Makes Sliding-Window Attention Competitive in Math Reasoning

    Jun 10, 2026Kai Liu, Peijie Dong, Xinchen Xie +5Sliding-Window AttentionQuadratic Attention

  24. Precision-Aware Illumination-Disentangled Vision Transformer for Spacecraft 6D Pose Estimation

    Jun 10, 2026Zongwu Xie, Yifan Yang, Yonglong Zhang +3Category-Level Object Pose EstimationVision Sensors

  25. Bridging the sim2real gap in the table tennis robot with a transformer-based ball states predictor

    Jun 9, 2026Yin Bi, Christian Conti, Bilan Yang +3Table Tennis RobotZero-Shot Sim-To-Real Transfer

  26. SPADE: Split-and-Delay Embeddings for Autoregressive High-Granularity Calorimeter Simulation

    Jun 9, 2026Joschka Birk, Frank Gaede, Anna Hallin +3Autoregressive TransformersAutoregressive Model

  27. Do Transformers Actually Help Intrusion Detection? A Temporal Sequence Evaluation on CIC-IDS2017

    Jun 9, 2026Zach Moczkodan, Hany RagabIntrusion DetectionHybrid Transformer

  28. PENet+: A Lightweight Residual Transformer Framework for Efficient Image Steganalysis

    Jun 9, 2026Jincheol AN, Dongsu Kim, Haneol Jang +1SteganographyTransformer Residual Streams

  29. Transformer Based Model for Spatiotemporal Feature Learning in EEG Emotion Recognition

    Jun 9, 2026Xinglong Cui, Dian GuTransformer AttentionSpatiotemporal

  30. Dynamic Linear Attention

    Jun 9, 2026Xin Wang, Hui Shen, Boyuan Zheng +7Quadratic AttentionEfficient Long-Context Inference

  31. Efficient RWKV-based Representation Learning for 3D Point Clouds

    Jun 9, 2026Yun Liu, Xuefeng Yan, Liangliang Nan +5Geometry-Aware Foundation ModelRepresentation Learning

  32. PF-Trans: Physics-Embedded Frequency-Aware Transformer for Spectral Reconstruction

    Jun 9, 2026Yuzhe Gui, Tianzhu Liu, Yanfeng Gu +1Spectral ReconstructionFourier

  33. ViP-VL: Vietnamese Self-supervised Speech Pretraining Model with Vector-Quantization Learning

    Jun 9, 2026Khanh Le, Kiet Anh Hoang, Bao Nguyen +5Calibration-Free Low-Bit Vector QuantizationTransformer Architectures

  34. DUET -- Dual User Embedding Transformers for Offsite Conversion Prediction

    Jun 8, 2026Reazul Hasan Russel, Mingwei Tang, Rostam Shirani +18Click-Through Rate PredictionLate-Stage Ranker

  35. Making Time Editable in Video Diffusion Transformers

    Jun 8, 2026Konstantin Kuklev, Viacheslav Vasilev, Alexander Kunitsyn +2Pre-Trained Video Diffusion ModelsTemporal Dynamics

  36. Tight Sample Complexity of Transformers

    Jun 8, 2026Chenxiao Yang, Nathan Srebro, Zhiyuan LiTransformer ArchitecturesSample Complexity

  37. Integrating gene regulatory priors into Transformer attention with scTransformer for interpretable scRNA-seq analysis

    Jun 8, 2026Mikele Milia, Louis Fabrice Tshimanga, Henning Mueller +2Single-Cell Rna SequencingTransformer Attention

  38. From Rigid to Dynamic: Entropy-Guided Adaptive Inference for Long-Context LLMs

    Jun 8, 2026Zhanchao Xu, Haoyang Li, Qingfa Xiao +4LLM Inference EfficiencyDynamic Sparse Attention

  39. Trajectory Geometry of Transformer Representations Across Layers

    Jun 8, 2026Vishal Pandey, Gopal Singh, Yacine MahdidTransformer ArchitecturesMechanistic Interpretability

  40. Hasse Diagrams for Attention: A Partial Order Framework for Designing Transformer Masks

    Jun 8, 2026Chentao Li, Han GuoTransformer AttentionCausal Attention

  41. FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention

    Jun 8, 2026Yan Wang, Qifan Zhang, Jiachen Yu +12Efficient Long-Context InferenceDynamic Sparse Attention

  42. A systematic investigation of molecular encoding methods for drug property predictions across neural network and Transformer encoder-based model

    Jun 8, 2026Sheng-Ya Chen, Shan-Ju YehMolecular Property PredictionMechanistic Interpretability

  43. GeoMetric: Injecting Metric Geographic Structure into Worldwide Image Geo-Localization

    Jun 8, 2026Junchao Cui, Xuanzi Ma, Wenqi Shi +3Image GeolocalizationGlobal Attention Mechanism

  44. Q-Delta: Beyond Key-Value Associative State Evolution

    Jun 7, 2026Sumin Park, Seojin Kim, Noseong ParkQuadratic AttentionRecurrent State

  45. TeamHerald@CHIPSAL 2026: Hate Speech Detection and Sentiment Analysis of Nepali Memes using Transformer-based Architectures and Ensemble Learning

    Jun 7, 2026Ashish Acharya, Anish Khatiwada, Rohit Khadka +1Hate SpeechSentiment Analysis

  46. Understanding the Parameter Space Geometry of Transformers Encoding Boolean Functions

    Jun 7, 2026Blanka Köver, Alexandra Butoi, Anej Svete +2Transformer ArchitecturesActivation Functions

  47. Self-Supervised Vision Transformers for CBCT-Based Detection of Temporomandibular Joint Osteoarthritis

    Jun 6, 2026Shradhdha Trivedi, Vrundan Sojitra, Mariela PadillaComputed Tomography DatasetCone-Beam Computed Tomography

  48. Chiaroscuro Attention: Spending Compute in the Dark

    Jun 6, 2026Prateek Kumar SikdarTransformer AttentionRepresentational Capacity

  49. Set-Based Transformer for Atmospheric Compensation in Standoff LWIR Hyperspectral Imaging

    Jun 6, 2026Fabian Perez, Nicolas Quintero, Jeferson Acevedo +1Hyperspectral ImagingInfrared

  50. Ablation-Reversible Heads Don't Transfer: A Stress Test for Mechanistic Role Claims in Transformers

    Jun 6, 2026Philip QuirkeTransformer AttentionTransformer Architectures

  51. TextEconomizer: Enhancing Lossy Text Compression with Denoising Transformers and Entropy Coding

    Jun 6, 2026Mahbub E Sobhani, Anika Tasnim Rodela, Chowdhury Mofizur Rahman +2Entropy CodingTransformer Architectures

  52. AttentionCap: Transformer Based Capacitance Matrix Learning Toward Full-Chip Extraction

    Jun 6, 2026Jiechen Huang, Hector R. Rodriguez, Dingcheng Yang +3Global Attention MechanismTransformer Architectures

  53. RAPID: Layer-Wise Redundancy-Aware Pruning and Importance-Driven Token Merging for Efficient ViT

    Jun 6, 2026Kyumin Choi, Ikbeom JangVisual Token PruningSelf-Supervised Vision Transformers

  54. MOLOT System Card: Malicious Operational Logic Observation Transformer

    Jun 5, 2026Daniil Lopatkin, Maksim Mitrofanov, Stanislav Rakovsky +1Malware DetectionProgram Analysis

  55. Sparsely gated tiny linear experts

    Jun 5, 2026Simon SchugFeed-Forward NetworkTransformer Layers

  56. Attention at the Theoretical Minimum: A Mathematics of Arrays Framework for Memory-Optimal Transformer Kernels

    Jun 5, 2026Lenore Mullin, Gaetan HainsTransformer ArchitecturesMathematics

  57. Towards Tight Bounds for Streaming Attention

    Jun 5, 2026Justin Y. Chen, Ying Feng, Piotr Indyk +3Key-Value Cache CompressionTop-K

  58. How Much Dense Attention is Necessary? Oracle-Guided Sparse Prefill for Full/GQA Layers in Hybrid Long-Context Models

    Jun 5, 2026Hongxing Wang, Harenome Razanajato, Zhen Zhang +2Dynamic Sparse AttentionEfficient Long-Context Inference

  59. TrioPose: Native Triple-Stream Diffusion Transformers for Pose-Guided Text-to-Image Generation

    Jun 5, 2026Dian Gu, Zhengyi YangMultimodal Diffusion TransformersText-To-Image

  60. When is 3D Worth It? A Resource-Performance Frontier for CNNs and Transformers in Lung CT

    Jun 5, 2026Md Enamul Hoq, Sharafat Hossain, Imraul Emmaka +4Computed Tomography DatasetNon-Small Cell Lung Cancer