Energy-Based Transformer

Ask about this topic

Momentum

4 papers in the last four weeks, up 33% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 45

All topics
CardsList

  1. A Physics-Guided Transformer Framework for Electromigration Analysis in Multi-Segment Interconnects

    Oct 5, 2026Pavlos Stoikos, Anuj Pathania, George FlorosEnergy-Based TransformerPhysics-Guided Machine Learning

    Dept. of Electrical and Computer Engineering University of Thessaly, Greece · University of Amsterdam Amsterdam, The Netherlands · Dept. of Electronic and Electrical Engineering, Trinity College Dublin, Ireland

  2. TomoTransformer: Towards a Foundation Model for CT Reconstruction

    Sep 29, 2026AmirEhsan Khorashadizadeh, Benjamín BéjarCone-Beam Computed TomographySparse-View

    Swiss Data Science Center (SDSC) in Paul Scherrer Institute (PSI), Villigen, Switzerland.

  3. Storing Is Not Remembering: LSTM-UT and Bounded Gated Memory for Looped Transformers

    Sep 17, 2026Aras Kavuncu, Muhammad Burhan HafezTransformer ArchitecturesNeural Cellular Automata

    University of Southampton, Electronics and Computer Science

  4. Vision And Text Transformer For Predicting Answerability On Visual Question Answering

    Sep 15, 2026Tung Le, Huy Tien Nguyen, Le Minh NguyenSelf-Supervised Vision TransformersTransformer Architectures

    Japan Advanced Institute of Science and Technology, Ishikawa, Japan. · Faculty of Information Technology, University of Science, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam +1

  5. DA-DLM: Explicitly Modeling Token Dependencies in Diffusion Language Models

    Sep 14, 2026Pengyu Ji, Zichen Zhang, Xiang Hu +1Diffusion Language ModelsAutoregressive Transformers

    ‡School of Information Science and Technology, ShanghaiTech University · §Shanghai Engineering Research Center of Intelligent Vision and Imaging · ¶Tencent

  6. Denoising Diffusion Generative Models Secretly Calculate Attentions

    Sep 1, 2026Farzan Haddadi, Leila Monfared, Ebrahim Rezaii +3Denoising Diffusion Probabilistic ModelImage Generation

    School of Electrical Engineering, Iran University of Science & Technology, Tehran, IRAN · Independent researcher

  7. Non-Parametric Spatiotemporal Trajectory Prediction via State-Conditioned Transition Sampling

    Aug 14, 2026Michael Fore, Akshay Jain, Justin Downes +2Spatiotemporal PredictionEnergy-Based Transformer

    Amazon Web Services

  8. Virtual Temperature Sensors in Power Transformers Using Neural Ordinary Differential Equations

    Aug 13, 2026Berk Hadzhamolla, Alexander Johannes Stasik, Signe Riemer-SørensenEnergy-Based TransformerNeural Ordinary Differential Equations

    University of Oslo, Oslo, 0316, Norway · 2,3 SINTEF AS, Department of Mathematics and Cybernetics, Oslo, Norway · Department of Data Science, Norwegian University of Life Sciences, ˚As, Norway

  9. Full-bandwidth transformer

    Aug 9, 2026Xi Wang, Ziyang Cai, Zheng Zhan +5Autoregressive TransformersTransformer Architectures

    Johns Hopkins University · Princeton University · Microsoft

  10. Decode-Branch Transformers: Decoupling the Primary Prefill Path from Additional Decode Computation

    Jul 31, 2026Liming Liu, Mingze Wang, Tuo ZhaoPrefillInference Cost

    Georgia Institute of Technology · Peking University

  11. MDTransformer: A Hardware-Software Co-Design of Mode-Division Photonic Transformer Accelerator with Inverse-Designed Coherent Crossbar

    Jul 28, 2026Solomon Micheal Serunjogi, Rachmad Vidya Wicaksana Putra, Ayat Taha +2Photonic Transformer AcceleratorsEnergy-Based Transformer

    Photonic Research Lab (PRL), Division of Engineering, New York University (NYU) Abu Dhabi, United Arab Emirates · eBRAIN Lab, Division of Engineering, New York University (NYU) Abu Dhabi, United Arab Emirates

  12. Recursive transformers for semiconductor thermo-mechanical reliability

    Jul 28, 2026Kart-leong LimEnergy-Based TransformerSurrogate Models

    Institute of Microelectronics (IME), Agency for Science, Technology and Research (A*STAR), Singapore

  13. Physics Transformer: Tailoring Transformer for General PDE Prediction

    Jul 27, 2026Guoze Sun, Rui Zhang, Jiankai Tang +4Neural Partial Differential Equation SolversPartial Differential Equations

    Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China · School of Mechanics and Engineering Science, Peking University, Beijing, China

  14. On the Computational Complexity of Structural Generalization

    Jul 21, 2026Zichao WeiCompositional GeneralizationLearnability

    Department of Language Science and Technology Saarland University Saarbrücken, Germany

  15. Bio-SFT: Asymmetric Cortical Guidance and Retinal Adaptation for Robust HDR Reconstruction

    Jul 20, 2026Tingyu Cheng, Ting Zhang, Chongyi Li +2High Dynamic RangeRetinal Imaging

    Fujian Key Laboratory of Intelligent Processing and Wireless Transmission of Media Information, College of Physics and Information Engineering, Fuzhou University, Fujian 350108, China · School of Computer Science, Nankai University, Tianjin 300071, China · School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China

  16. Visible-Light Imaging Diagnosis of Neutral Particle Emission Tomography in the Tokamak Divertor: An Efficient Transformer-based Surrogate Model

    Jul 19, 2026Xiao Wang, Hao Si, Qiang Chen +8Energy-Based TransformerDiffusion Transformers

    School of Computer Science and Technology, Anhui University, Hefei 230601, China. · Institute of Plasma Physics, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China. · School of Artificial Intelligence, Anhui University, Hefei 230601, China.

  17. Physics-Guided Masked Multi-Task Network for Edge-Friendly Battery Health Diagnostics from Sto-chastically Fragmented Charging Profiles

    Jul 19, 2026Shuhao Chen, Tianyu Shi, Chengyi TuLithium-Ion BatteriesUseful Life Prediction

    School of Economics and Management, Zhejiang Sci-Tech University, Hangzhou, 310018, China.

  18. Dynamic Loss Balancing for Joint SOH and RUL Prediction of Lithium-Ion Batteries via a Rotary SOH-Injected Prior Battery Transformer

    Jul 19, 2026Shuhao Chen, Tianyu Shi, Yiwen Huang +1Lithium-Ion BatteriesUseful Life Prediction

    School of Economics and Management, Zhejiang Sci-Tech University, Hangzhou, 310018, China.

  19. Graph-Regularized Deep Learning for EEG-Based Emotion Recognition with Psychologically-Grounded Label Structure

    Jul 8, 2026Dongyang Kuang, Zizheng Ma, Yushan Zhang +1Emotion RecognitionEmotion

    School of Mathematics (Zhuhai), Sun Yat-sen University, Guangzhou, China

  20. At-Grok Is Not Converged:A Measurement-Validity Audit for Grokking Representation Metrics

    Jul 7, 2026Truong Xuan KhanhGrokkingEnergy-Based Transformer

    H&K Research Studio / Clevix LLC, Hanoi, Vietnam

  21. LBTCap: A Lightweight Bilateral Transformer for Real-Time Remote Sensing Image Change Captioning

    Jul 3, 2026Licheng Zhang, Siew-Kei Lam, Naveed AkhtarRemote SensingImage Difference Captioning

    School of Computing and Information Systems, The University of Melbourne, Melbourne, 3010, Victoria, Australia · Nanyang Technological University, Singapore · The University of Melbourne, Melbourne, 3010, Victoria, Australia

  22. PixelU: A U-Shaped Transformer for Efficient End-to-End Pixel Diffusion

    Jun 26, 2026Zipeng Guo, Lichen Ma, Yu He +4Diffusion TransformersPixels

    Xi’an Jiaotong University

  23. An iterative energy-based multimodal transformer for joint retrieval of wheat soil moisture, leaf area index, and plant height from Sentinel-1 and Sentinel-2 time series

    Jun 23, 2026Shubham Kumar Singh, Peilei Fan, Suraj A. Yadav +2SoilNormalized Difference Vegetation Index

    Department of Urban and Environmental Policy and Program, Tufts University, 503 Boston Ave., Medford, MA 02155 · Electrical and Computer Engineering Department, Mississippi State University, Mississippi, USA, 39762 · Department of Physics, Indian Institute of Technology (BHU), Varanasi, India 221005 +1

  24. Energy-Based Transformers as Predictors of Reading Difficulty

    Jun 22, 2026Jakub Dotlacil, Ece TakmazEnergy-Based TransformerReadability

    Utrecht University

  25. Scalable Physics-Inspired Transformers for Spin Glasses

    Jun 22, 2026Lu Zhong, Wenli Duan, Jing Liu +2Spatial Photonic Ising MachinesBoltzmann Distributions

    Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu 611731, China · School of Physical Science and Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China · Institute of Theoretical Physics, Chinese Academy of Sciences, Beijing 100190, China +4

  26. Targeted Recovery of Weight-Space Mechanisms From Neural Networks

    Jun 19, 2026Antoine Vigouroux, Lee SharkeyModel WeightsNeural Network

    1MATS · 2Goodfire

  27. FiLM-Coordinated Dual-Branch Transformer for Global-Local Dependency Modeling in Language Modeling

    Jun 19, 2026Zhiqiang Zhou, Xu Ling, Junliang DaiLanguage ModelingEnergy-Based Transformer

  28. LoopCoder-v2: Only Loop Once for Efficient Test-Time Computation Scaling

    Jun 16, 2026Jian Yang, Shawn Guo, Wei Zhang +16Code OptimizationLoop

    1Beihang University · 2IQuest Research · 4Renming University of China +1

  29. The Effect of Training Task Diversity on In-Context Learning through the Lens of Low-Dimensional Subspaces

    Jun 5, 2026Soo Min Kwon, Alec S. Xu, Can Yaras +3In-Context LearningEquivariant Learning

    Department of Electrical and Computer Engineering, University of Michigan · Department of Statistics, University of California, Davis

  30. LiAuto-GeoX: Efficient Grounded Driving Transformer

    Jun 4, 2026Jiawei Lian, Haoyi Sun, Yang Wu +83D ReconstructionAutonomous Driving

    Nanjing University of Science and Technology · Li Auto Inc. · Northwestern Polytechnical University +1

  31. Fixed Universal Transformers

    May 29, 2026Jingwen Liu, Alexandr Andoni, Daniel HsuTransformer ArchitecturesEnergy-Based Transformer

    Columbia University New York, NY

  32. Customer Churn Prediction on Structured Data Using FT-Transformer and Stacking Ensembles

    May 26, 2026Joyjit Roy, Samaresh Kumar Singh, Laxmi ShawChurn PredictionRoc-Auc

    Independent Researcher, Austin, TX, USA · Independent Researcher, Leander, TX · Texas A & M University-Victoria, Victoria, TX

  33. SoftCap: Soft-Budget Control for Diffusion Transformer Acceleration

    May 26, 2026Yuhang Zhang, Junxiang Qiu, Huixia Ben +3Diffusion TransformersEnergy-Based Transformer

    Hefei University of Technology · University of Science and Technology of China · Anhui University of Science and Technology +1

  34. World Machine: Towards Generative World Modeling for Time-Series

    May 21, 2026Elton Cardoso do Nascimento, Alexandre da Silva Simões, Esther Luna Colombini +2Many WorldsEnergy-Based Transformer

    Universidade Estadual de Campinas (UNICAMP), {FEEC, IC} · Universidade Estadual Paulista (UNESP), Instituto de Ciˆencia e Tecnologia, Sorocaba

  35. DT-Transformer: A Foundation Model for Disease Trajectory Prediction on a Real-world Health System

    May 14, 2026Yunying Zhu, Andrew R Weckstein, Kueiyu Joshua Lin +1Readmission PredictionElectronic Health Records

    Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA · Harvard T.H. Chan School of Public Health, Harvard University, Boston, MA, USA · Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University, MA, USA +2

  36. Z-Order Transformer for Feed-Forward Gaussian Splatting

    May 13, 2026Can Wang, Lei Liu, Wei Jiang +1Feed-Forward 3D Gaussian SplattingFeed-Forward 3D Gaussian

    The University of Hong Kong · Futurewei Technologies Inc.

  37. Complex-Valued Phase-Coherent Transformer

    May 11, 2026Leona HiokiTransformer AttentionEnergy-Based Transformer

  38. The Transformer as a Polar State Estimator

    May 10, 2026Peter RacioppoEnergy-Based TransformerTransformer Architectures

    Independent Researcher Los Angeles, CA

  39. CATO: Charted Attention for Neural PDE Operators

    May 9, 2026Chun-Wun Cheng, Sifan Wang, Carola-Bibiane Schönlieb +1Neural OperatorsNeural Partial Differential Equation Solvers

    DAMTP University of Cambridge · Institute for Foundations of Data Science Yale University · Yau Mathematical Sciences Center Tsinghua University

  40. Carbon-Taxed Transformers: A Green Compression Pipeline for Overgrown Language Models

    Apr 28, 2026Ajmain Inqiad Alam, Palash Roy, Chanchal K. Roy +2Large Language Model CompressionData Compression Methods

    University of Saskatchewan, Canada

  41. Universal Transformers Need Memory: Depth-State Trade-offs in Adaptive Recursive Reasoning

    Apr 23, 2026Grigory SapunovBounded-MemoryTransformer Architectures

    Intento

  42. Hyperloop Transformers

    Apr 23, 2026Abbas Zeitoun, Lucas Torroba-Hennigen, Yoon KimTransformer ArchitecturesEnergy-Based Transformer

    Massachusetts Institute of Technology

  43. The Phasor Transformer: Resolving Attention Bottlenecks on the Unit Circle

    Mar 18, 2026Dibakar SigdelTransformer AttentionEnergy-Based Transformer

    Mindverse Computing LLC, WA 98087

  44. Garbage Attention in Large Language Models: BOS Sink Heads and Sink-aware Pruning

    Jan 11, 2026Jaewon Sok, Jewon Yeom, Seonghyeon Park +2Large Language Model CompressionUnstructured Pruning

    Department of Rural Systems Engineering, Seoul National University · Graduate School of Data Science, Seoul National University · Department of Aerospace Engineering, Seoul National University

  45. Left-Branching Transformers Excel at Right-Branching Languages: Data Shapes Word Order Preferences in Language Models

    Date pendingVarvara Arzt, Allan Hanbury, Terra BlevinsLanguage ModelingLanguage Pairs

    Faculty of Informatics, TU Wien · Khoury College of Computer Sciences, Northeastern University