Transformers

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

42 new papers

A weekly snapshot of new work published in Transformers.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Transformers.

Inside this field

Focused directions

1,209 papers

Latest in Transformers

Browse all topics →
CardsList

  1. The Geometry of Last-Layer Model Stealing

    Jun 5, 2026Snigdha Chandan KhilarTransformer LayersTransformer Architectures

  2. The Effect of Training Task Diversity on In-Context Learning through the Lens of Low-Dimensional Subspaces

    Jun 5, 2026Soo Min Kwon, Alec S. Xu, Can Yaras +3In-Context LearningUnified Transformer

  3. Multiscale POD of Transformer Attention Fields: Scale-Selective Analysis via Morlet Scalogram

    Jun 4, 2026Athanasios ZerisTransformer AttentionMultimodal Transformers

  4. You Only Index Once: Cross-Layer Sparse Attention with Shared Routing

    Jun 4, 2026Yutao Sun, Yanqi Zhang, Li Dong +2Dynamic Sparse AttentionEfficient Long-Context Inference

  5. Vortex: Efficient and Programmable Sparse Attention Serving for AI Agents

    Jun 4, 2026Zhuoming Chen, Xinrui Zhong, Qilong Feng +5Dynamic Sparse AttentionLarge Language Model Serving

  6. RedKnot: Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention

    Jun 4, 2026Yang Liu, ZhaoKai Luo, HuaYi Jin +5Key-Value CacheLarge Language Model Serving

  7. Where does Absolute Position come from in decoder-only Transformers?

    Jun 4, 2026Valeria Ruscio, Umberto Nanni, Fabrizio SilvestriTransformer EncoderRotary Position Embedding

  8. When Good Enough Is Optimal: Multiplication-Only Matrix Inversion Approximation for Quantized Gated DeltaNet

    Jun 4, 2026Luoming Zhang, Yuwei Ren, Kui Zhang +7Matrix MultiplicationQuadratic Attention

  9. ATT-CR: Adaptive Triangular Transformer for Cloud Removal

    Jun 4, 2026Yang Wu, Ye Deng, Pengna Li +4Hybrid TransformerRemoval

  10. Mechanistic Insights into Functional Sparsity in Multimodal LLMs via CoRe Heads

    Jun 4, 2026Ruoxi Sun, Quantong Qiu, Juntao Li +3Multi-Modal Large Language ModelsCross-Modal Attention

  11. Dominant-Layer ZO: A Single Layer Dominates Zeroth-Order Fine-Tuning of LLMs

    Jun 3, 2026Wanhao Yu, Ziyan Wang, Zheng Wang +7Continual Fine-TuningTransformer Layers

  12. Graph Set Transformer

    Jun 3, 2026Jose E. Escrig Molina, Baoquan Chen, Daniel ProbstHeterogeneous Graph TransformersTransformer Architectures

  13. Fast & Faithful Function Vectors

    Jun 3, 2026Minh An Pham, Anton Segeler, Thomas Wiegand +4Quadratic AttentionActivation Steering

  14. In-Context Graphical Inference

    Jun 3, 2026Zehua Cheng, Wei Dai, Jiahao SunProbabilistic InferenceBayesian

  15. TaDA: Calibrated Probe Gating for Task-Domain LoRA Merging

    Jun 3, 2026Huy Quoc To, Fuyi Li, Guangyan Huang +1Low-Rank Adaptation AdaptersLow-Rank Adaptation

  16. Signed Dual Attention: Capturing Signed Dependencies in Time Series Forecasting

    Jun 3, 2026Balthazar Courvoisier, Tristan CazenaveTransformer AttentionTime Series

  17. An Empirical Audit of Input Encoders for Multi-Channel Signal Transformers

    Jun 3, 2026Ossi LehtinenTransformer EncoderEncoders

  18. Test-Time Compute Scaling for ASR with Depth-Conditioned Looped Transformers

    Jun 3, 2026Yacouba Kaloga, Shashi Kumar, Shakeel A. Sheikh +3Automatic Speech Recognition SystemsTransformer Architectures

  19. SparDA: Sparse Decoupled Attention for Efficient Long-Context LLM Inference

    Jun 3, 2026Yaosheng Fu, Guangxuan Xiao, Xin Dong +2Dynamic Sparse AttentionLLM Inference Efficiency

  20. ChessMimic: Per-Rating Transformer Models for Human Move, Clock, and Outcome Prediction in Online Blitz Chess

    Jun 3, 2026Thomas JohnsonChessClocks

  21. Low-Rank Decay for Grokking in Scale-Invariant Transformers: A Spectral-Geometric View

    Jun 3, 2026Mingyu LiLow-Rank FactorizationRegularization Techniques

  22. MeshTok: Efficient Multi-Scale Tokenization for Scalable PDE Transformers

    Jun 3, 2026Yanshun Zhao, Xiaoyu Peng, Jiamin Jiang +2Neural Partial Differential Equation SolversTokenization

  23. Generalizable Multi-Task Learning for Wireless Networks Using Prompt Decision Transformers

    Jun 3, 2026Fatih Temiz, Shavbo Salehi, Melike Erol-KantarciWireless NetworksOpen Radio Access Networks

  24. Parameter-Efficient Fine-Tuning with Learnable Rank

    Jun 3, 2026Arpit Garg, Simon Lucey, Hemanth SaratchandranParameter-Efficient Fine-TuningLow-Rank Adaptation

  25. P-Cast Precision in FP8 Attention: Sink-Induced Collapse and the Optimality of S=2^8

    Jun 2, 2026Reed LauFull-Precision PerformanceFp8

  26. Value-Aware Stochastic KV Cache Eviction for Reasoning Models

    Jun 2, 2026Ting-Yun Chang, Harvey Yiyun Fu, Deqing Fu +3Key-Value Cache EvictionEviction

  27. Dynamic Short Convolutions Improve Transformers

    Jun 2, 2026Oliver Sieberling, Bharat Runwal, Rameswar Panda +1Transformer ArchitecturesConvolutional

  28. UtVAA: Ultra-tiny Vision Transformer with Affix Attention for Mobile Image Classification

    Jun 2, 2026Romiyal George, Sathiyamohan Nishankar, Selvarajah Thuseethan +1Self-Supervised Vision TransformersMobilenetv2

  29. TrAction: Action Recognition with Sparse Trajectories

    Jun 2, 2026Jan F. Meier, Felix B. Mueller, Alexander Ecker +1Open-Vocabulary Action RecognitionInteraction Trajectories

  30. PrimeSVT: An Automated Memory-aware Pruning Framework with Prioritized Compression Policy for Spiking Vision Transformers

    Jun 2, 2026Rachmad Vidya Wicaksana Putra, Achyuta Muthuvelan, Alberto Marchisio +1Self-Supervised Vision TransformersContext Compression

  31. Causal Evidence of Stack Representations in Modeling Counter Languages Using Transformers

    Jun 2, 2026Nishit SinghTransformer ArchitecturesNext-Token Prediction

  32. BA-T: An Iterative Transformer for Two-View Bundle Adjustment

    Jun 2, 2026Ganlin Zhang, Weirong Chen, Daniel Cremers +13D ReconstructionCross-Attention

  33. PSViT: A Methodology for Structurally Pruning Spiking Vision Transformers

    Jun 2, 2026Rachmad Vidya Wicaksana Putra, Achyuta Muthuvelan, Alberto Marchisio +1Self-Supervised Vision TransformersSparse Deep Neural Network Accelerators

  34. A Training-Efficient Transformer-Based Anti-Spoofing Network for Logical Access in ASVspoof 5

    Jun 2, 2026Sidan Yin, Bo ZhaoSpeaker Verification PerformanceSpoofed Speech

  35. Multi-Segment Attention: Enabling Efficient KV-Cache Management for Faster Large Language Model Serving

    Jun 1, 2026Chunan Shi, Yilei Chen, Yilin Chen +2CacheLLM Inference Efficiency

  36. Data-Driven Forecasting of three-Component Seismograms Using Transformer Architectures

    Jun 1, 2026Waleed Esmail, Stuart Russell, Jana Klinge +2Full Waveform InversionAutoregressive Transformers

  37. Do Transformers Need Three Projections? Systematic Study of QKV Variants

    Jun 1, 2026Ali Kayyam, Anusha Madan Gopal, M Anthony LewisKey-Value Cache QuantizationTransformer Architectures

  38. Evaluating Transformer and LSTM Frameworks for Prediction in Ungauged Basins

    Jun 1, 2026Taye Akinrele, James Halgren, Noorbakhsh Amiri Golilarz +2Transformer EncoderBidirectional Long Short-Term Memory

  39. Do Value Vectors in Deep Layers Need Context from the Residual Stream?

    Jun 1, 2026Muyu He, Yuchen Liu, Qingya Huang +1Transformer ArchitecturesEfficient Long-Context Inference

  40. Edge Prediction for Roof Wireframe Reconstruction with Transformers

    Jun 1, 2026Gustav Hanning, Ludvig Dillén, Jonathan Astermark +2Dense Semantic SegmentationTransformer Architectures

  41. Forget Attention: Importance-Aware Attention Is All You Need

    Jun 1, 2026Suhyeong Shin, Yeongwook YangSoftmax AttentionTransformer Architectures

  42. Riemannian Gradient Descent for Low-Rank Architectures

    Jun 1, 2026Nicholas KnightLow-Rank FactorizationRiemannian Optimization

  43. LayerRoute: Input-Conditioned Adaptive Layer Skipping via LoRA Fine-Tuning for Agentic Language Models

    Jun 1, 2026Prateek Kumar SikdarLarge Language Model BackbonesTransformer Layers

  44. VG2\text{VG}^2GT: Voxel-Gaussian Splatting Visual Geometry Grounded Transformer

    Jun 1, 2026Yibin Zhao, Yihan Pan, Jun Nan +3Feed-Forward 3D Gaussian ReconstructionGaussian Splatting

  45. Rethinking the Role of Positional Encoding: Sliding-Window Transformers without PE Remain Turing Complete

    Jun 1, 2026Qian Li, Xinyu Mao, Shang-Hua TengPositional EncodingTransformer Architectures

  46. Move the Query, Not the Cache: Characterizing Cross-Instance Latent Attention Redistribution Across GPU Fabrics

    May 31, 2026Bole Ma, Jan Eitzinger, Harald Köstler +1Dynamic Sparse AttentionKv-Cache Blocks

  47. CART: Context-Anchored Recurrent Transformer -- A Parameter-Efficient Architecture with Learned Stability

    May 31, 2026Chad A. CappsRecurrent ModelTransformer Architectures

  48. Interpolation between Convolution and Attention via K-Nearest Neighbors

    May 31, 2026Mingi KangConvolutionalSoftmax Attention

  49. Leaf Spectral Reflectance Prediction Using Multi-Head Attention Neural Networks

    May 31, 2026Parastoo Farajpoor, Alireza Pourreza, Mohammadreza Narimani +2Spectral ReflectanceCanopy

  50. Blurry Window Attention

    May 31, 2026Axel Laborieux, Christos Sourmpis, Juan Gabriel Kostelec +1Softmax AttentionKernel Method

  51. FAiT: Frequency-Aware Inverted Transformer for Multivariate Time Series Forecasting

    May 31, 2026Peng He, Yao Liu, Yanglei Gan +3Multivariate Time Series ForecastingTime Series

  52. Don't Read Everything: A Curvature-Conditioned Query for Linear Attention

    May 31, 2026Dong Le, Thong Nguyen, Cong-Duy Nguyen +1Quadratic AttentionEfficient Long-Context Inference

  53. Emergent Ordinal Geometry in Transformers Trained on Local Comparisons

    May 31, 2026Nishit SinghTransformer ArchitecturesDistance

  54. Beyond Sinusoids: A Morlet Wavelet Framework for Transformer Positional Encoding

    May 31, 2026Athanasios ZerisPositional EncodingRotary Position Embedding

  55. TriLens: Per-Layer Logit-Lens Entropy for White-Box Hallucination Detection

    May 31, 2026Bohan Yang, Yijun Gong, Zhi Zhang +3Hallucination DetectionLens

  56. Efficient Temporal Modeling for Mobile Sleep Staging via Lightweight Random Attention

    May 31, 2026Guisong Liu, Pengfei Wei, Jainsong Zhang +1SleepSpatial Attention

  57. Refit the Probe: Single-Direction Ablation Is Not a Necessity Test

    May 30, 2026Yuhang JiangTransformer LayersSequence Modeling

  58. Mitigating Hallucinations in Large Language Models Via Decoder Layer Skipping

    May 30, 2026Hanze Li, Jinhao You, Yichen Guo +3Large Language Model HallucinationHallucination Mitigation

  59. GIRL-DETR: Gradient-Isolated Reinforcement Learning for Video Moment Retrieval

    May 30, 2026Shihang Zhang, Mingjin Kuai, Ye Wei +2Transformer EncoderTime-Step-Wise Learnable Surrogate Gradient Function

  60. Prior-Guided Multi-Omic Transformers for Single-Cell Gene Regulatory Network Inference

    May 30, 2026Tianyang Xu, Tianci Liu, Niraj Rayamajhi +4Gene-Regulatory NetworksTransformer Architectures