Transformers

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

42 new papers

A weekly snapshot of new work published in Transformers.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Transformers.

Inside this field

Focused directions

1,208 papers

Latest in Transformers

Browse all topics →
CardsList

  1. RaBitQCache: Rotated Binary Quantization for KVCache in Long Context LLM Inference

    Jun 30, 2026Wenhao Li, Jinhao Dong, Hailin Zhang +3LLM Inference EfficiencyDynamic Sparse Attention

  2. Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation

    Jun 30, 2026Jiachun Li, David Simchi-LeviBayesian Experimental DesignHeterogeneous Treatment Effects

  3. Revealing Safety-Critical Scenarios for UTM via Transformer

    Jun 30, 2026Huaze Tang, Bill Zeng, Chao Wang +3Safety-Critical ScenariosTransformer Architectures

  4. Dual Sparse Aggregation Transformer for Multispectral Object Detection

    Jun 30, 2026Wencong Wu, Xiuwei Zhang, Hanlin Yin +2Feature Pyramid NetworksHyperspectral Imaging

  5. Quality-Aware Modulation for Diffusion Transformers

    Jun 29, 2026Luke Budny, Yuhong Guo, Kevin CheungMultimodal Diffusion TransformersText-To-Image Diffusion Models

  6. When transformers learn "impossible" languages, what do they learn?

    Jun 29, 2026Ram Janarthan, Coleman Haley, Sharon GoldwaterTraining Language ModelsLinguistics

  7. Gradient Smoothing: Coupling Layer-wise Updates for Improved Optimization

    Jun 29, 2026Haoming Meng, Anton Sugolov, Vardan PapyanGradient FlowTransformer Layers

  8. Probing-Guided Layer Selection from Self-Supervised Speech Models for Generalizable Audio Deepfake Detection

    Jun 29, 2026Marjan Beheshti, Majid Rostami, Bo ChenAudio Deepfake DetectionSelf-Supervised Speech Models

  9. Reweighting Framewise Attention in Video Transformers for Facial Expression Understanding

    Jun 29, 2026Seongro Yoon, Donghyeon Cho, Jinsun Park +1Facial Expression RecognitionSelf-Supervised Vision Transformers

  10. Morphing into Hybrid Attention Models

    Jun 29, 2026Disen Lan, Jianbin Zheng, Yuxi Ren +5Hybrid TransformerTransformer Attention

  11. Hierarchical Global Attention (HGA)

    Jun 29, 2026Woernle Frank, Fedosov Vladimir, Grinenko ArtemiyEfficient Long-Context InferenceTransformer Architectures

  12. Transformer Architectures as Complete Bayes Processes: A Formal Proof in the Measure-Theoretic Kernel Framework

    Jun 29, 2026Haobo YangTransformer ArchitecturesBayesian

  13. Predict, Reuse, and Repair: Accelerating Dynamic Sparse Attention for Long-Context LLM Decoding

    Jun 29, 2026Tianyu Wang, Gourav Rattihalli, Aditya Dhakal +4Dynamic Sparse AttentionEfficient Long-Context Inference

  14. UniGP: Taming Diffusion Transformer for Prior-Preserved Unified Generation and Perception

    Jun 29, 2026Qin Guo, Hao Luo, Dongxu Yue +4Multimodal Diffusion TransformersDense Prediction

  15. A Point Cloud Transformer for Remote Monitoring and Automated Assessment of Physical Rehabilitation Exercises

    Jun 29, 2026Kazi Rafat, Md. Ismail Hossain, M M Lutfe Elahi +4RehabilitationTransformer Architectures

  16. DNA Language Models: An Assessment of Pre-Training for Fine-Tuning Tasks

    Jun 29, 2026Romain Karpinsky, Julien Mozziconacci, Mickaël DelceyGenerative Pretrained TransformersGenomics

  17. MATCH: Modulating Attention via In-Context Retrieval for Long-Context Transformers

    Jun 29, 2026Linrui Ma, Chun Hei Lo, Xinyu Wang +12Efficient Long-Context InferenceDynamic Sparse Attention

  18. IG-Lens: Exact Additive Probability Attribution Across Transformer Layers via Telescoping Integrated Gradients

    Jun 29, 2026Duc Anh NguyenTransformer LayersTransformer Encoder

  19. Exploring the Cryptographic Limits of Transformer Networks

    Jun 28, 2026Stefan Domunco, Andis Draguns, Philip Torr +2Transformer ArchitecturesDiffie-Hellman

  20. SAFE-DiT: Semantics-Aware Fast-path Execution for High-Resolution Diffusion Transformers

    Jun 28, 2026Xuanhua Yin, Yuxuan Jia, Chuanzhi Xu +1Multimodal Diffusion TransformersSelf-Supervised Vision Transformers

  21. Adaptive Financial Transformer with Regime-Gated Attention for Stock Return Prediction

    Jun 28, 2026Dishan SarkarTransformer AttentionTransformer Architectures

  22. Generalization Analysis of Transformers in Distribution Regression

    Jun 28, 2026Peilin Liu, Ding-Xuan ZhouTransformer ArchitecturesGeneralization Bounds

  23. How Token Influence Decays with Distance: A Green-Function View of Trained Language Models

    Jun 28, 2026Matthias Brändel, Stephan Köhler, Oliver RheinbachAutoregressive TransformersNext-Token Prediction

  24. Do Diabetic Foot Ulcer Segmentation Models Generalize? A Cross-Dataset Benchmark of CNN and Transformer Architectures

    Jun 27, 2026Abderrahmane BenfatahCross-Dataset BenchmarkConvolutional Neural Networks

  25. CCRC: A Change-Aware Captioning and Reasoning Chain for Image Change Captioning and Segmentation

    Jun 27, 2026Jinhong Hu, Xiaoping Wang, Shuyin Huang +3Remote Sensing Image Change CaptioningChange Detection

  26. Depth-Staggered Fibonacci Spacing for Sparse Attention: Static Schedules Beat Learned Dilation and Extrapolate Where Dense Attention Fails

    Jun 26, 2026Chad A. CappsDynamic Sparse AttentionSoftmax Attention

  27. CMSL: Constructive Multi-Sequence Learning for Recommendation Systems

    Jun 26, 2026Zikun Cui, Renzhi Wu, Junjie Yang +10Multimodal RecommendationLarge-Scale Recommendation Systems

  28. From Black-Box to Clinical Insight: A Multi-Stage Explainable Framework for Speech-Based Cognitive Impairment Detection

    Jun 26, 2026Yasaman Haghbin, Sina Rashidi, Ali Zolnour +6Mild Cognitive ImpairmentCognitive Diagnosis

  29. Long-Term Prediction of Local and Global Human Motion with Occlusion Recovery

    Jun 26, 2026Qiaoyue Yang, Sven Heutger, Christopher Niemann +3Human Motion PredictionHuman Motion Generation

  30. Flexformer: Flexible Linear Transformer with Learnable Attention Kernel

    Jun 26, 2026Haoran Zhang, Feng ZhouQuadratic AttentionMixformer

  31. Mitigating Position Bias in Transformers via Layer-Specific Positional Embedding Scaling

    Jun 26, 2026Changze Lv, Zhenghua Wang, Yiran Ding +9Rotary Position EmbeddingTransformer Architectures

  32. AI-Generated Image Recognition via Fusion of CNNs and Vision Transformers

    Jun 26, 2026Xuan-Bach Mai, Hoang-Minh Nguyen-Huu, Quoc-Nghia Nguyen +3Synthetic Image DetectionAuthenticity

  33. The Context-Ready Transformer

    Jun 25, 2026Mahesh GodavartiTransformer LayersTransformer Architectures

  34. Developmental approach reveals the statistical learning of Neural Language Models: Transformers generalize from the most abstract statistical patterns

    Jun 25, 2026Wang Bojun, Holly Jenkins, Elizabeth WonnacottTransformer Architectures

  35. Prism Transformer: Progressive Head Schedules for Hierarchical Attention Processing

    Jun 25, 2026Shubham AggarwalTransformer ArchitecturesPrism

  36. CARVE: Content-Aware Recurrent with Value Efficiency for Chunk-Parallel Linear Attention

    Jun 25, 2026Sayak DuttaRecurrent ModelQuadratic Attention

  37. Transformer-Based Classification of Bacterial Raman Spectra with LOOCV

    Jun 25, 2026Jamile Mohammad Jafari, Thomas BocklitzRamanTransformer Architectures

  38. Information-Aware KV Cache Compression for Long Reasoning

    Jun 25, 2026Jushi Kai, Zhuiri Xiao, Alexandra Birch +1Key-Value Cache CompressionKey-Value

  39. PersistentKV: Page-Aware Decode Scheduling for Long-Context LLM Serving on Commodity GPUs

    Jun 25, 2026Muhammad AhmedGraphics Processing Unit MemoryQuadratic Attention

  40. Generating Special Triangulations with Transformers

    Jun 25, 2026Charles Arnal, Jacky H. T. Yip, François Charton +1TriangulationsPolytopes

  41. CascadeFormer: Depth-Tapered Transformers Motivated by Gradient Fan-in Asymmetry

    Jun 25, 2026Huzama Ahmad, Cao Viet Hai Nam, Se-Young YunTransformer ArchitecturesMixformer

  42. Epiphany-Aware KV Cache Eviction Without the Attention Matrix

    Jun 25, 2026Steven Kolawole, Virginia SmithKey-Value Cache EvictionKey-Value Cache

  43. Enhancing Layer Interaction Using Key-Correlated Layer Attention

    Jun 24, 2026Jianlong Xiong, ChuanBo Xie, Le Yu +2Global Attention Mechanism

  44. Lifelong In-Context Learning with Transformers Requires Parametric Forms of Attention

    Jun 24, 2026Luke McDermott, Robert W. Heath, Rahul ParhiIn-Context LearningTransformer Architectures

  45. Communicability-Inspired Positional Encoding (CIPE)

    Jun 24, 2026Yipeng Zhang, Zhongtian Sun, Pietro Liò +1Heterogeneous Graph TransformersPositional Encoding

  46. An iterative energy-based multimodal transformer for joint retrieval of wheat soil moisture, leaf area index, and plant height from Sentinel-1 and Sentinel-2 time series

    Jun 23, 2026Shubham Kumar Singh, Peilei Fan, Suraj A. Yadav +2Soil Moisture EstimationSentinel-2

  47. AeroCast: Probabilistic 3D Trajectory Prediction for Non-Cooperative Aerial Obstacles via Transformer-MDN Architecture

    Jun 23, 2026Syed Izzat Ullah, Jose BacaAerotrackTransformer Architectures

  48. Emergent Capabilities Arise Randomly from Learning Sparse Attention Patterns

    Jun 23, 2026Vatsal Baherwani, Zixi Chen, Shikai Qiu +2Transformer AttentionDynamic Sparse Attention

  49. DDStereo: Efficient Dual Decoder Transformers for Stereo 3D Road Anomaly Detection

    Jun 23, 2026Shiyi Mu, Zichong Gu, Zhiqi Ai +23D Object DetectionAutonomous Driving Perception

  50. VSANet: View-aware Sparse Attention Network for Light Field Image Denoising

    Jun 23, 2026Gargi Panda, Soumitra Kundu, Saumik Bhattacharya +1Progressive Attention FusionImage Denoising

  51. CrossPool: Efficient Multi-LLM Serving for Cold MoE Models through KV-Cache and Weight Disaggregation

    Jun 23, 2026Zhuoren Ye, Tianyu Wo, Dinghao Xue +4Kv-Cache BlocksLarge Language Model Serving

  52. VistaRef: Boosting Visual Spatial Orientation Awareness for Pointing-to-Object Detection

    Jun 23, 2026Ling Li, Zhizhen Cai, Xinkun Wu +4HandObject Detection

  53. Why Do Accumulated Transformations Extrapolate?

    Jun 23, 2026Mahesh GodavartiRotary Position EmbeddingSoftmax Attention

  54. An LLM-based Two-Stage Transformer Framework for Cross-Domain Bearing Fault Diagnosis with Limited Data

    Jun 23, 2026Jinghan Wang, Feng Cheng, Wentao Wu +3Fault DiagnosisPredictive Maintenance

  55. Parallel Manifold Steering: Efficient Adaptation of Large Associative Memories via Residual Energy Shaping

    Jun 23, 2026Kanishk AwadhiyaTransformer Residual StreamsDense Associative Memory

  56. SURGELLM: Rethinking Multi-Task Evaluation through Task-Aware Feature Gating with Class-Balanced Normalization

    Jun 23, 2026Noor Islam S. Mohammad, Ulug BayazitNatural Language ProcessingMulti--Task Learning

  57. RoPE-Aware Bit Allocation for KV-Cache Quantization

    Jun 23, 2026Fengfeng Liang, Yuechen Zhang, Jiaya JiaKey-Value Cache QuantizationQuadratic Attention

  58. Prediction of Viscoelastic Droplet Impact Dynamics Using a Vision Transformer-Based Approach

    Jun 22, 2026Diego A. de Aguiar, Cassio M. Oishi6-Dof Viscoelastic MechanismSelf-Supervised Vision Transformers

  59. Tapered Language Models

    Jun 22, 2026Reza Bayat, Ali Behrouz, Aaron CourvilleTransformer Architectures

  60. Scaling Linear Mode Connectivity and Merging to Billion Parameter Pretrained Transformers

    Jun 22, 2026Tianyi Li, Zhiqiang ShenGenerative Pretrained TransformersTransformer Architectures