Generative Pretrained Transformers

Follow this topic Ask about this topic

Recent momentum

-100%

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-07

3 new papers

A weekly snapshot of new work published in Generative Pretrained Transformers.

36 papers

Latest in Generative Pretrained Transformers

Open your feed →

Counterfactual Analysis via Large Language Models
Aug 5, 2026Zonghao YangCounterfactual Reasoning Large Language Models Fail
Learning What to Remember: Test-Time Training via Context Distillation
Aug 3, 2026Zixuan Wang, Xingyu Dang, Rui-Jie Zhu +4Efficient Long-Context Inference Efficient Continual Learning
PolymerGPT: Multi-property Optimization with a Decoder-Based GPT Model for Generative Polymer Design
Aug 2, 2026Charlie Pyle, Adarsh Gadari, C. Adrian Figg +3Polymer Computer-Aided Design
Verifier-Guided Model Discovery for Physical Dynamical Systems with Pretrained Symbolic Transformers
Aug 1, 2026Farbod Faraji, Francesco BelardinelliSymbolic Regression Nonlinear Dynamics
Ad Headline Generation using Self-Critical Masked Language Model
Jul 7, 2026Yashal Shakti Kanungo, Sumit Negi, Aruna RajanText Generation Generative Pretrained Transformers
DNA Language Models: An Assessment of Pre-Training for Fine-Tuning Tasks
Jun 29, 2026Romain Karpinsky, Julien Mozziconacci, Mickaël DelceyGenerative Pretrained Transformers Finetuning
Scaling Linear Mode Connectivity and Merging to Billion Parameter Pretrained Transformers
Jun 22, 2026Tianyi Li, Zhiqiang ShenGenerative Pretrained Transformers Model Weights
RT-DocLayout: Real-Time End-to-End Document Layout Analysis with Reading Order in the Wild
Jun 22, 2026Cheng Cui, Tingquan Gao, Xueqing Wang +11End-To-End Document Parsing Rt-Detr
A Comparative Study of Pretrained Transformer Models for Quranic ASR: Speech Representations, Label Formats, and Dataset Composition
Jun 18, 2026Nabil Mosharraf Hossain, Riasat Islam, Unaizah ObaidellahAutomatic Speech Recognition Model Generative Pretrained Transformers
HLS-GPT: A Generative Pretrained Transformer (GPT) for Continental-Scale NASA Harmonized Landsat and Sentinel-2 (HLS) Reflectance Reconstruction Across All Bands on Arbitrary Dates
Jun 16, 2026Junjie Li, Hankui K. Zhang, David P. RoySentinel-2 Remote Sensing
Toward Controllable Catalyst Inverse Design via Large-Scale Autoregressive Pretraining
Jun 16, 2026Dong Hyeon Mok, Jonggeol Na, Seoin BackHeterogeneous Catalysis Adsorption Energies
The Reservoir Attention Network: Cross-Pass State in Pretrained Transformers via Content-Addressable Reservoir Injection
Jun 14, 2026Emma LeonhartTransformer Attention Generative Pretrained Transformers
Simplifying the Modeling of Arbitrary Conditionals in Natural Language
Jun 12, 2026Yinhan Lu, Eric Elmoznino, Léo Gagnon +3Generative Pretrained Transformers Next-Token Prediction
SpikeDecoder: Realizing the GPT Architecture with Spiking Neural Networks
Jun 10, 2026Claas Beger, Florian Walter, Alois KnollSpiking Neural Networks Generative Pretrained Transformers
Humanoid-GPT: Scaling Data and Structure for Zero-Shot Motion Tracking
Jun 2, 2026Zekun Qi, Xuchuan Chen, Dairu Liu +10State-Of-The-Art Zero-Shot Performance Robot Imitation Learning
LLUMI: Improving LLM Writing Assistance for Mental Health Support with Online Community Feedback
May 28, 2026Jiwon Kim, Maya Ajit, Sherry Gong +4Mental Health Sensitive Data
Grammatically-Guided Sparse Attention for Efficient and Interpretable Transformers
May 23, 2026Spandan PratyushDynamic Sparse Attention Generative Pretrained Transformers
Spectral Probe-Circuits: A Three-Step Recipe for Identifying Attention-Head Circuits in Pretrained Transformers
May 22, 2026Yongzhong XuGenerative Pretrained Transformers Circuits
Argo: Efficient Importance Labeling for Enterprise Email Systems
May 20, 2026Siddhant Ray, Ganesh Ananthanarayanan, Kevin Chian +5Spam Inference Cost
LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series
May 19, 2026Alexis Roger, Prateek Humane, Zhenghan Tai +4Generative Pretrained Transformers Time Series
MiniGPT: Rebuilding GPT from First Principles
May 17, 2026Jibin JosephSparsegpt Generative Pretrained Transformers
From Sparsity to Simplicity: Enabling Simpler Sequential Replacements via Sparse Attention Distillation
May 15, 2026Yuxin Ren, Maxwell D Collins, Miao Hu +1Dynamic Sparse Attention Dataset Distillation
GiLT: Augmenting Transformer Language Models with Dependency Graphs
May 15, 2026Tianyu Huang, Yida Zhao, Chuyan Zhou +1Generative Pretrained Transformers Protein Language Model
Protocol-Aware Tokenization and Architecture Co-Design for Wireless Packet Foundation Models
May 14, 2026Swadhin Pradhan, Shazal Irshad, Jerome HenryCapture Protocol Wireless Foundation Models
Learning Less Is More: Premature Upper-Layer Attention Specialization Hurts Language Model Pretraining
May 11, 2026Jinchang Zhu, Jindong Li, Yuwen Hao +3Generative Pretrained Transformers Large Language Model Decoding
Teaching LLMs to See Graphs: Unifying Text and Structural Reasoning
May 11, 2026Dario VajdaGraph-Based Retrieval Augmented Generation Large Language Model Backbones
Priming: Hybrid State Space Models From Pre-trained Transformers
May 8, 2026Aditya Chattopadhyay, Elvis Nunez, Prannay Kaul +6Generative Pretrained Transformers Priming
CircuitFormer: A Circuit Language Model for Analog Topology Design from Natural Language Prompt
May 7, 2026Md Touhidul Islam, Sujan Kumar Saha, Farimah Farahmandi +1Analog Circuits Circuits
Spreadsheet Modeling Experiments Using GPTs on Small Problem Statements and the Wall Task
Apr 28, 2026Thomas A. Grossman, Yuan Chen, Sopiko DatuashviliSpreadsheet Benchmarks Reproducibility
The signal is the ceiling: Measurement limits of LLM-predicted experience ratings from open-ended survey text
Apr 21, 2026Andrew Hong, Jason Potteiger, Luis E. ZapataSurvey Ceiling
RoLegalGEC: Legal Domain Grammatical Error Detection and Correction Dataset for Romanian
Apr 21, 2026Mircea Timpuriu, Mihaela-Claudia Cercel, Dumitru-Clementin CercelGrammatical Error Correction Legal Domain
Freeze, Diffuse, Decode: Task-Aware Adaptation of Transformer Embeddings for Antimicrobial Peptide Design
Nov 28, 2025Pankhil Gawade, Adam Izdebski, Myriam Lizotte +4Antimicrobial Peptide Generative Pretrained Transformers
Any-Order GPT as Masked Diffusion Model: Decoupling Formulation and Architecture
Jun 24, 2025Shuchen Xue, Tianyu Xie, Tianyang Hu +5Diffusion Language Models Masked Diffusion Language Models
MetaTT: A Global Tensor-Train Adapter for Parameter-Efficient Fine-Tuning
Jun 10, 2025Javier Lopez-Piqueres, Pranav Deshpande, Archan Ray +3Parameter-Efficient Fine-Tuning Methods Generative Pretrained Transformers
Ensemble Learning for Large Language Models in Text and Code Generation: A Survey
Mar 13, 2025Mari Ashiga, Wei Jie, Fan Wu +5Large Language Model Generation AI Code Generation
A Survey of Transformer-based Language Models with Focus on Efficiency
May 15, 2024Wazib Ansar, Saptarsi Goswami, Amlan ChakrabartiGenerative Pretrained Transformers Large Language Model Scale

Aug 5, 2026 · cs.AIJ/K move · Enter open · S save

Counterfactual Analysis via Large Language Models

Zonghao Yang

Stevens Institute of Technology

Counterfactual analysis aims to predict potential outcomes under hypothetical scenarios, offering valuable insights for decision-making. This paper investigates the application of large language models (LLMs), specifically the GPT-3.5 model, for counterfactual analysis. We focus on the online lending context, where the counterfactual return on investment (ROI) is crucial for evaluating different interest rate schemes. We begin by assessing the predictive performance of GPT and comparing it with advanced machine learning algorithms. The results show that prompt engineering can significantly enhance GPT's predictions, with the R-squared increasing from 1.97% to 2.84%, closely approaching the 3.48% achieved by gradient-boosted regression. Subsequently, we utilize GPT to generate counterfactual ROIs under a set of alternative interest rates. GPT exhibits logical coherence and causal reasoning in its responses. The findings underscore the potential of LLMs as effective tools for counterfactual analysis in online lending, suggesting broader applications for LLMs in various predictive and decision-making contexts.

Counterfactual Reasoning Large Language Models Fail Predictive Performance Generative Pretrained Transformers

Open paper Ask about it PDF