Transformer Encoder

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Transformer Encoder.

Inside this field

Focused directions

228 papers

Latest in Transformer Encoder

Open your feed →
CardsList

  1. Weakening Neurons: An Input-Output Functionality in Transformers with Outsize Influence

    Sep 16, 2026Sebastian Gerstner, Hilal AlQuabeh, Kentaro Inui +1Multi-Layer Transformers

  2. QiT: Quantum-Inspired Transformer for Visual Recognition Task

    Sep 15, 2026Badri N. Patro, Vijay AgneeswaranSelf-Supervised Vision TransformersTransformer Variant

  3. Temporal Recurrence Favors Fewer Layers

    Sep 14, 2026Ivan Anokhin, Johan Obando-Ceron, Irina Rish +1Recurrent ModelMultilayer Perceptrons

  4. Sparse Weight and Edge Circuit Discovery in Transformer-based Acoustic Models

    Sep 11, 2026Jiankun Wei, Ewan Dunbar, Gerald PennTransformer EncoderSelf-Supervised Audio Representations

  5. Length Generalization for Transformers via Compression

    Sep 8, 2026Georg Zetzsche, Hongjian Jiang, Andy Yang +4Transformer VariantGeneralization Bounds

  6. A Transformer-Based Delta Expression Encoder for Psilocybin Transcriptional Response: Architecture, Representations, and Biological Validation

    Sep 8, 2026Sai JayakumarTranscriptomicsTransformer Encoder

  7. One-Layer Transformer Provably Learns Multiclass One-Nearest Neighbor in Context

    Sep 1, 2026Skanda Athreya, Yutong WangMulti-Layer TransformersMulticlass Classification

  8. From Language to Behavior: Scaling Sequence Transformers for Industrial Recommendation Ranking with Rec-Native Designs

    Sep 1, 2026Jie Chen, Xiangqian Yu, Yanchao Lian +9Transformer VariantRecommendation

  9. Denoising Diffusion Generative Models Secretly Calculate Attentions

    Sep 1, 2026Farzan Haddadi, Leila Monfared, Ebrahim Rezaii +3Denoising Diffusion Probabilistic ModelAutoencoder

  10. Toppling the Hierarchy in Byte-level Language Modeling

    Aug 31, 2026Lukas Edman, Alexander FraserProtein Language ModelMulti-Layer Transformers

  11. Late Transformer Layers Recode Syntax Canonically: Evidence from Greek Scrambling and Cross-Layer Generalisation

    Aug 31, 2026Christos Nikolaos Zacharopoulos, Revekka Kyriakoglou, Chara Tsoukala +1Multi-Layer TransformersSyntactic Structure

  12. Stride-k Subsampling: Train-Free Audio Token Reduction for Whisper

    Aug 31, 2026Chanhee Cho, Junhyuk Choi, Bugeun KimAudio TokensSubsampling

  13. AQ3D: Adaptive Query Transformer for 3D Instance Segmentation

    Aug 31, 2026Keno Moenck, Thorsten Schüppstuhl3D Instance Segmentation3D

  14. Transformer-Based Flow Shop Scheduling Using MILP-Generated Training Data

    Aug 30, 2026Roderich WallrathFlexible Job-Shop SchedulingAdaptive Scheduling

  15. Algebraic Decomposition Theory for Transformer Length Generalization

    Aug 13, 2026Andy Yang, Blerta Veseli, Corentin Barloy +5Transformer VariantLength

  16. Virtual Temperature Sensors in Power Transformers Using Neural Ordinary Differential Equations

    Aug 13, 2026Berk Hadzhamolla, Alexander Johannes Stasik, Signe Riemer-SørensenHeat DiffusionNeural Ordinary Differential Equations

  17. LabelFusion-TS: Fusing Large Language Models, Transformer Encoders, and Financial Time Series for Monetary-Policy Stance Classification

    Aug 12, 2026Michael Schlee, Fabian Lukassen, Christoph WeisserFinancial Sentiment AnalysisTime Series Foundation Models

  18. G0.5: One Autoregressive Stream for Robot Reasoning and Action

    Aug 12, 2026Yicheng Liu, Zibin Dong, Baijun Ye +24Diffusion-Based Vision-Language-ActionsVision-Language-Action

  19. ODE-Based Transformer Decoders for Iterative Sign Language Translation

    Aug 11, 2026Tuğçe Kızıltepe, Hacer Yalim KelesSign Language TranslationTransformer Encoder

  20. Why Post-Norm Transformers Collapse: Attention Amplification and Gradient Repair Failure

    Aug 10, 2026Xingjian Wang, Qingyu Han, Xiaodong Luo +1Batch NormalizationTransformer Encoder

  21. Deployable Per-Instance Multi-Layer Activation Steering for Large Language Models

    Aug 9, 2026Muhammad Faishal Adly Nelwan, Alfan Farizki WicaksonoLinear Activation SteeringModel Activations

  22. Why Knowing Both Hops Is Not Enough: Understanding Two-Hop Generalization in Language Models

    Aug 7, 2026Zili Zhang, Yilin Wang, Heng Wang +2Multi-Layer TransformersLarge Multimodal Models

  23. ZIPBrain: Can EEG Foundation Models Be Faster, Locally Deployable, but Accurate?

    Aug 7, 2026Lingwei Li, Yirong Kan, Peng Chen +3Electroencephalography Foundation ModelsElectroencephalogram

  24. Is Self-Pretraining really useful to improve diagnosis in medical Time Series?

    Aug 6, 2026Omar Coser, Antonio Orvieto, Paolo Soda +1Time-Series ClassificationTime Series

  25. Separating quantum circuits from classical LLMs

    Aug 4, 2026Srinivasan Arunachalam, Arkopal Dutt, Hari Krovi +1Quantum CircuitsLarge Language Model Quantization

  26. LoopMTP: A looped transformer guided by latent multi-token prediction

    Aug 4, 2026Behzad Shomali, Markus Frey, David Berghaus +2Multi-Layer TransformersNext-Token Prediction

  27. CMuon: Accelerating and Stabilizing Diffusion Transformer Training via Chunked Momentum Orthogonalization

    Aug 3, 2026Chuyan Chen, Peng Sun, Kun YuanDiffusion TransformersMuon

  28. Transformer Geometry Observatory TGO-III: Semantic Geometry Observatory

    Aug 3, 2026Kaustubh Kapil, Kishor P. UplaVisual Geometry Grounded TransformerVision Transformer

  29. Divergent large language model predictions from convergent representations in ambiguous word pairs

    Aug 3, 2026K. Jack Scott, Narun Pat, Veronica LiesaputraTransformer EncoderProtein Language Model

  30. Dual-Flow Transformers: Decoupling the Primary Prefill Path from Additional Decode Computation

    Jul 31, 2026Liming Liu, Mingze Wang, Tuo ZhaoPrefillLLM Inference Efficiency

  31. HealthCAT: An Interpretable Encoder-only Transformer Framework for Health Indicator Prediction and Temporal Interpretation of Wearable Sensor Data

    Jul 30, 2026Xiaotong Yu, Joshua Y. Kim, HaeJin Lee +1Wearable DevicesTransformer Encoder

  32. MDTransformer: A Hardware-Software Co-Design of Mode-Division Photonic Transformer Accelerator with Inverse-Designed Coherent Crossbar

    Jul 28, 2026Solomon Micheal Serunjogi, Rachmad Vidya Wicaksana Putra, Ayat Taha +2PhotonicsUnified Transformer

  33. Penelope: Localized Latent Recurrence for Efficient Structured Reasoning

    Jul 28, 2026Yutong Chen, Shouqian Shi, Xinran Liu +5Latent ReasoningTransformer Encoder

  34. Recursive transformers for semiconductor thermo-mechanical reliability

    Jul 28, 2026Kart-leong LimUnified TransformerSurrogate Modeling

  35. At-the-Roofline Sparse Tensor Contractions on Vector Processors for Transformer Inference

    Jul 28, 2026Bowen Wang, Chi Zhang, Diyou Shen +3Tensor ProgramsVectorization

  36. Physics Transformer: Tailoring Transformer for General PDE Prediction

    Jul 27, 2026Guoze Sun, Rui Zhang, Jiankai Tang +4Neural Partial Differential Equation SolversPartial Differential Equations

  37. Grounding latent algorithm routing in transformer reasoning

    Jul 27, 2026Xiangbo Zhang, Xiaoxu MaTransformer EncoderDynamic Routing

  38. Sensitivity Analysis of GRU, LSTM and Transformer Encoder in Classification of Automated Driving Systems

    Jul 25, 2026Bidhya Shrestha, Christos PapadopoulosAutonomous DrivingAutomatic Identification System

  39. Hidden Boundary Motion in Transformer Optimization: Function-Space Orthogonalization of Affine Weight and Bias Updates

    Jul 24, 2026Zhang Gongyue, Sheng Yixuan, Liu donghan +3Transformer VariantAdam

  40. Indexing: the Beginning and the End

    Jul 24, 2026Alexander Kozachinskiy, Vicente Opazo, Felipe UrrutiaNeural ArchitecturesMulti-Layer Transformers

  41. Optimizing Transformer Neural Network for Real-Time Outlier Detection on FPGAs

    Jul 24, 2026Ilia Sobakinskikh, Paul Alexander BilokonTransformer VariantTime Series

  42. RED-PIM: Reducing Data Movement for Transformers using Processing-in-Memory

    Jul 23, 2026Zahra Yousefijamarani, Alaa AlameldeenTransformer VariantIn-Memory Computing

  43. KroQuant: Kronecker-Structured Block Transforms for Efficient Post-Training Quantization of Diffusion Transformers

    Jul 23, 2026Yann Bouquet, Alireza Khodamoradi, Kristof Denolf +1Post-Training QuantizationQuantization

  44. Agree on the Model, Verify the Inference: GKR Protocols for HND-Based Transformer Inference

    Jul 23, 2026Xiaolong Liang, Juanjuan Li, Rui Qin +1Transformer VariantTheorem

  45. On the Computational Complexity of Structural Generalization

    Jul 21, 2026Zichao WeiComputableGeneralization Bounds

  46. Enhancing Transformer-based Routing by Encoding Distance via Relative Positional Encoding

    Jul 21, 2026Leyre Encío, Daniel Fuertes, Carlos R. del-Blanco +1Multiplex Graph TransformersPositional Encoding

  47. An Explainable FFT-Based Spatial-Frequency Fusion Framework for Deepfake Detection

    Jul 19, 2026Pamela Kirui, Cho Hyuk, Qingzhong Liu +1Deepfake DetectionSpatio-Temporal Fusion

  48. Visible-Light Imaging Diagnosis of Neutral Particle Emission Tomography in the Tokamak Divertor: An Efficient Transformer-based Surrogate Model

    Jul 19, 2026Xiao Wang, Hao Si, Qiang Chen +8Plasma Shape ControlSelf-Supervised Vision Transformers

  49. The Geometry of Semantic Space: A Continuous Geometric Framework for the Transformer Architecture

    Jul 19, 2026Zhihua LiangTransformer VariantDiffusion Transformers

  50. Physics-Guided Masked Multi-Task Network for Edge-Friendly Battery Health Diagnostics from Sto-chastically Fragmented Charging Profiles

    Jul 19, 2026Shuhao Chen, Tianyu Shi, Chengyi TuLithium-Ion BatteriesUseful Life Prediction

  51. Dynamic Loss Balancing for Joint SOH and RUL Prediction of Lithium-Ion Batteries via a Rotary SOH-Injected Prior Battery Transformer

    Jul 19, 2026Shuhao Chen, Tianyu Shi, Yiwen Huang +1Lithium-Ion BatteriesUseful Life Prediction

  52. Frontier Language Models Struggle to Copy: Text Can Be Better Viewed in 2D

    Jul 17, 2026Haodong Wen, Yiran Zhang, Yingfa Chen +1Protein Language ModelPositional Encoding

  53. A Benchmark for Electrical Load Forecasting Across Grid Levels: Time-Series Transformers Outperform Established Methods

    Jul 17, 2026Matthias Hertel, Sebastian Pütz, Jonathan Kolar +3Energy ForecastingTime Series Foundation Models

  54. T^2MLR: Transformer with Temporal Middle-Layer Recurrence

    Jul 16, 2026Ziyang Cai, Xingyu Zhu, Yihe Dong +2Multi-Layer TransformersLatent Reasoning

  55. RepTran: Search-Based Repair of Transformer Models

    Jul 13, 2026Yuta Ishimoto, Paolo Arcaini, Fuyuki Ishikawa +3Transformer VariantDiffusion Transformers

  56. Incremental Transformer for Surrogate-Based Inverse Design of Geopolymer Mixtures

    Jul 12, 2026Giansalvo Cirrincione, Filippo GrassiaGenerative Inverse DesignBlended Cement Concrete

  57. Do Transformer Temporal Heads and Post-Pooling Motion Gates Help CorrNet-based CSLR? An Empirical Study

    Jul 10, 2026Lisi Wang, Zhidong Xiao, Jianjun PengCnn-LstmTransformer Encoder

  58. At-Grok Is Not Converged:A Measurement-Validity Audit for Grokking Representation Metrics

    Jul 7, 2026Truong Xuan KhanhGrokkingModular Addition

  59. Escaping the Procrustean Bed: Groupwise Orthogonal Connectors for Audio-Language Models

    Jul 7, 2026Ho-Lam Chung, Ke-Han Lu, Yi-Cheng Lin +3Speech EncoderSpeaker Identity

  60. Structure-Specific Representational Priors Causally Control the Grokking Delay

    Jul 5, 2026Gunner Levi HoweGrokkingRepresentation Learning