Generative Pretrained Transformers

Ask about this topic

Momentum

4 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 42

All topics
CardsList

  1. Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes It

    Sep 29, 2026Zehao Jin, Ruixuan Deng, Junran WangTransformer ArchitecturesGenerative Pretrained Transformers

    Georgia Institute of Technology

  2. ZonoGPT: Towards An Abstract Domain for Verifying Large GPT Models

    Sep 28, 2026Hai Duong, Thanh Le, ThanhVu NguyenGenerative Pretrained TransformersTransformer Architectures

    Department of Computer Science George Mason University Fairfax, VA, USA · Unaffiliated Yokosuka, Japan

  3. Pretraining Transformers with Quantized Softmax in Attention

    Sep 27, 2026Shangzhen Zhu, Muyan Hu, Tomasz KozlowskiTransformer AttentionGumbel-Softmax Relaxation

    University of Illinois Urbana-Champaign

  4. Harm Laundering in GPT Models: Evidence That Gender Discrimination Is Transformed Rather Than Reduced Across Safety-Trained Generations

    Sep 17, 2026Sarah Wyer, Sue Black, Noura Al MoubayedToxicityLarge Language Model Safety

    Durham University

  5. Counterfactual Analysis via Large Language Models

    Aug 5, 2026Zonghao YangLarge Language Model ForecastingGenerative Pretrained Transformers

    Stevens Institute of Technology

  6. Learning What to Remember: Test-Time Training via Context Distillation

    Aug 3, 2026Zixuan Wang, Xingyu Dang, Rui-Jie Zhu +4Efficient Long-Context InferenceTest-Time Training

    Princeton University · UC Santa Cruz · Carnegie Mellon University +1

  7. PolymerGPT: Multi-property Optimization with a Decoder-Based GPT Model for Generative Polymer Design

    Aug 2, 2026Charlie Pyle, Adarsh Gadari, C. Adrian Figg +3Polymer Property PredictionGenerative Pretrained Transformers

    Texas A&M University · University of Pittsburgh · Department of Chemistry, Virginia Tech +2

  8. Training nGPT

    Aug 2, 2026Ilya Loshchilov, Boris GinsburgGenerative Pretrained TransformersTransformer Architectures

    NVIDIA

  9. Verifier-Guided Model Discovery for Physical Dynamical Systems with Pretrained Symbolic Transformers

    Aug 1, 2026Farbod Faraji, Francesco BelardinelliVortexDynamical Systems

    Department of Computing, Huxley Building, Imperial College London 180 Queen’s Gate, South Kensington, London SW7 2RH, United Kingdom

  10. StellarTTS: Sparse Temporal Embedding for Low-Latency and Robust Speech Synthesis

    Jul 22, 2026Kaicheng Luo, Xuefei Gong, Yutao Sun +6Autoregressive Text-To-SpeechSpeech Synthesis

    Honor Device Co., Ltd., China · Auditory Cognition and Computational Acoustics Lab MoE Key Lab of Artificial Intelligence, AI Institute School of Computer Science, Shanghai Jiao Tong University, Shanghai, China

  11. Learning Standard Model structure from LHC data with Riemannian flow matching

    Jul 17, 2026Midori Kato, Kevin A. Urquía-Calderón, Inar Timiryasov +1Riemannian Flow MatchingGenerative Pretrained Transformers

    Niels Bohr Institute, University of Copenhagen, Jagtvej 155A, DK-2200, Copenhagen, Denmark

  12. Ad Headline Generation using Self-Critical Masked Language Model

    Jul 7, 2026Yashal Shakti Kanungo, Sumit Negi, Aruna RajanGenerative Pretrained Transformers

    Amazon

  13. DNA Language Models: An Assessment of Pre-Training for Fine-Tuning Tasks

    Jun 29, 2026Romain Karpinsky, Julien Mozziconacci, Mickaël DelceyGenerative Pretrained TransformersTransformer Architectures

    CNRS, LORIA · Nancy, 54000, France · CNRS +1

  14. Muown Implicitly Performs Angular Step-size Decay

    Jun 22, 2026Florian Hübler, Kai Lion, Antonio Orvieto +1MuonAdaptive Optimizers

    Department of Computer Science · ETH Zurich, Switzerland · ELLIS Institute Tübingen, MPI-IS +1

  15. Scaling Linear Mode Connectivity and Merging to Billion Parameter Pretrained Transformers

    Jun 22, 2026Tianyi Li, Zhiqiang ShenGenerative Pretrained TransformersTransformer Architectures

    MBZUAI

  16. RT-DocLayout: Real-Time End-to-End Document Layout Analysis with Reading Order in the Wild

    Jun 22, 2026Cheng Cui, Tingquan Gao, Xueqing Wang +11Document ParsingOrder Matters

    PaddlePaddle Team, Baidu Inc.

  17. A Comparative Study of Pretrained Transformer Models for Quranic ASR: Speech Representations, Label Formats, and Dataset Composition

    Jun 18, 2026Nabil Mosharraf Hossain, Riasat Islam, Unaizah ObaidellahDiscrete Speech RepresentationsGenerative Pretrained Transformers

    Greentech Apps Foundation, United Kingdom · Greentech Apps Foundation, United Kingdom. · Queen Mary University of London, United Kingdom +3

  18. HLS-GPT: A Generative Pretrained Transformer (GPT) for Continental-Scale NASA Harmonized Landsat and Sentinel-2 (HLS) Reflectance Reconstruction Across All Bands on Arbitrary Dates

    Jun 16, 2026Junjie Li, Hankui K. Zhang, David P. RoySentinel-2Remote Sensing

    Geospatial Sciences Center of Excellence, Department of Geography and Geospatial Sciences, South Dakota State University, Brookings, SD 57007, USA · Department of Geography, Environment, and Spatial Sciences, & Center for Global Change and Earth Observations, Michigan State University, MI 48824, USA

  19. Toward Controllable Catalyst Inverse Design via Large-Scale Autoregressive Pretraining

    Jun 16, 2026Dong Hyeon Mok, Jonggeol Na, Seoin BackHeterogeneous CatalysisGenerative Pretrained Transformers

    Department of Chemical and Biomolecular Engineering, Institute of Emergent Materials, Sogang University, Seoul 04107, Republic of Korea · Department of Chemical Engineering and Materials Science, Ewha Womans University, Seoul 03760, Republic of Korea · Department of Chemical Engineering, Graduate Program in System Health Science and Engineering, Ewha Womans University, Seoul, 03760, Republic of Korea +4

  20. The Reservoir Attention Network: Cross-Pass State in Pretrained Transformers via Content-Addressable Reservoir Injection

    Jun 14, 2026Emma LeonhartTransformer ArchitecturesGenerative Pretrained Transformers

  21. Simplifying the Modeling of Arbitrary Conditionals in Natural Language

    Jun 12, 2026Yinhan Lu, Eric Elmoznino, Léo Gagnon +3Generative Pretrained TransformersHigh-Fidelity Conditional Generation

    Mila — Quebec AI Institute · McGill University · Université de Montréal

  22. SpikeDecoder: Realizing the GPT Architecture with Spiking Neural Networks

    Jun 10, 2026Claas Beger, Florian Walter, Alois KnollSpiking Neural NetworksTransformer Encoder

    Chair of Robotics, Artificial Intelligence and Real-time Systems Technical University of Munich, Germany

  23. Humanoid-GPT: Scaling Data and Structure for Zero-Shot Motion Tracking

    Jun 2, 2026Zekun Qi, Xuchuan Chen, Dairu Liu +10Generative Pretrained TransformersSequential Scaling

    Tsinghua University · Galbot Inc. · Shanghai Jiao Tong University +2

  24. LLUMI: Improving LLM Writing Assistance for Mental Health Support with Online Community Feedback

    May 28, 2026Jiwon Kim, Maya Ajit, Sherry Gong +4Mental HealthAi-Assisted Writing

    University of Illinois Urbana-Champaign · Indiana University Indianapolis

  25. Spectral Probe-Circuits: A Three-Step Recipe for Identifying Attention-Head Circuits in Pretrained Transformers

    May 22, 2026Yongzhong XuTransformer AttentionTransformer Architectures

  26. Argo: Efficient Importance Labeling for Enterprise Email Systems

    May 20, 2026Siddhant Ray, Ganesh Ananthanarayanan, Kevin Chian +5Inference CostImportance

    University of Chicago · Microsoft · University of Chicago / TensorMesh

  27. LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series

    May 19, 2026Alexis Roger, Prateek Humane, Zhenghan Tai +4Large Language Model PretrainingGenerative Pretrained Transformers

    Université de Montréal Mila - Quebec AI Institute

  28. MiniGPT: Rebuilding GPT from First Principles

    May 17, 2026Jibin JosephAutoregressive Language ModelsGenerative Pretrained Transformers

    Department of Computer Science The University of Texas at Austin Austin, TX 78712

  29. From Sparsity to Simplicity: Enabling Simpler Sequential Replacements via Sparse Attention Distillation

    May 15, 2026Yuxin Ren, Maxwell D Collins, Miao Hu +1Transformer AttentionAttention Layers

    University of Arizona, Tucson AZ, USA · TetraMem, Inc., San Jose CA, USA

  30. Protocol-Aware Tokenization and Architecture Co-Design for Wireless Packet Foundation Models

    May 14, 2026Swadhin Pradhan, Shazal Irshad, Jerome HenryCapture ProtocolWireless Foundation Models

    Cisco Systems

  31. Masked Generative Transformer Is What You Need for Image Editing

    May 11, 2026Wei Chow, Linfeng Li, Xian Sun +14Diffusion-Based Image EditingImage Editing

    1ByteDance · 2National University of Singapore · 3Duke University +2

  32. Learning Less Is More: Premature Upper-Layer Attention Specialization Hurts Language Model Pretraining

    May 11, 2026Jinchang Zhu, Jindong Li, Yuwen Hao +3Large Language Model PretrainingPretraining

    The Hong Kong University of Science and Technology (Guangzhou)

  33. Priming: Hybrid State Space Models From Pre-trained Transformers

    May 8, 2026Aditya Chattopadhyay, Elvis Nunez, Prannay Kaul +6Generative Pretrained TransformersPriming

    AWS Agentic AI

  34. Spreadsheet Modeling Experiments Using GPTs on Small Problem Statements and the Wall Task

    Apr 28, 2026Thomas A. Grossman, Yuan Chen, Sopiko DatuashviliSpreadsheetsReproducibility

    University of San Francisco 2130 Fulton St. San Francisco, CA 94117 USA

  35. The signal is the ceiling: Measurement limits of LLM-predicted experience ratings from open-ended survey text

    Apr 21, 2026Andrew Hong, Jason Potteiger, Luis E. ZapataSurveyCeiling

    Dimension Labs

  36. RoLegalGEC: Legal Domain Grammatical Error Detection and Correction Dataset for Romanian

    Apr 21, 2026Mircea Timpuriu, Mihaela-Claudia Cercel, Dumitru-Clementin CercelGrammatical Error CorrectionLegal Domain

    National University of Science and Technology POLITEHNICA Bucharest, Bucharest, Romania · Paris 1 Panthéon-Sorbonne University, Paris, France · University of Bucharest, Bucharest, Romania

  37. Freeze, Diffuse, Decode: Task-Aware Adaptation of Transformer Embeddings for Antimicrobial Peptide Design

    Nov 28, 2025Pankhil Gawade, Adam Izdebski, Myriam Lizotte +4Generative Pretrained TransformersBehavioral Embeddings

    Institute of AI for Health, Helmholtz Munich · School of Computation, Information and Technology, Technical University of Munich · Mila - Quebec AI institute; Univ. de Montreal +3

  38. RheOFormer: A generative transformer model for simulation of complex fluids and flows

    Oct 1, 2025Maedeh Saberi, Amir Barati Farimani, Safa JamaliFluid DynamicsSurrogate Models

    Department of Mechanical and Industrial Engineering, Northeastern University, Boston, MA 02115, USA · Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA

  39. Any-Order GPT as Masked Diffusion Model: Decoupling Formulation and Architecture

    Jun 24, 2025Shuchen Xue, Tianyu Xie, Tianyang Hu +5Diffusion Language ModelsMasked Diffusion Language Models

    University of Chinese Academy of Sciences · Academy of Mathematics and Systems Science, Chinese Academy of Sciences · School of Mathematical Sciences, Peking University +2

  40. MetaTT: A Global Tensor-Train Adapter for Parameter-Efficient Fine-Tuning

    Jun 10, 2025Javier Lopez-Piqueres, Pranav Deshpande, Archan Ray +3Parameter-Efficient Fine-Tuning MethodsModel Fine-Tuning

    Global Technology Applied Research, JPMorgan Chase, New York, NY 10001, USA

  41. Ensemble Learning for Large Language Models in Text and Code Generation: A Survey

    Mar 13, 2025Mari Ashiga, Wei Jie, Fan Wu +5Large Language Model GenerationCode Generation

    School of Computing and Engineering, University of West London, London W5 5RF, United Kingdom · Turing Intelligence Technology Limited, London EC2M 2PF, United Kingdom · School of Computer Science, University of Leeds, Leeds LS2 9JT, United Kingdom

  42. Attention is All You Need Until You Need Retention

    Jan 15, 2025M. Murat YasliogluRetentionRecurrent Model

    Istanbul University, School of Business