Decoder-Only Language Models

Latest papers 67

All topics
CardsList
  1. Project Greenhouse: Progress Toward Fully Open and Sovereign Agentic Search

    Oct 8, 2026Jimmy Lin, Sahel Sharifymoghaddam, Lingwei Gu +1LLM RerankingLearning to Rank

  2. Sequential Functional Structured Tucker Compression for Large Language Model Attentions

    Sep 30, 2026Jiangfeng Chen, Xinyu Wang, Tianshuo Yan +4LLM CompressionDecoder-Only Language Models

  3. Is Weight Tying Still Beneficial for Decoder-Only LLMs in Private Settings Under DP-SGD?

    Sep 30, 2026Razan El Mais, Ali Chehab, Ibrahim Issa +1Privacy-Preserving Language ModelsDifferentially Private Stochastic Gradient Descent

  4. Predictive Geometry of Hidden Trajectories in Transformers

    Sep 29, 2026Timur Mudarisov, Mikhail Burtsev, Tatiana Petrova +1Decoder-Only Language ModelsRepresentation Geometry in Language Models

  5. Repurposing Pre-trained LLMs as High Fidelity Continuous Text Autoencoders

    Sep 23, 2026Arkanath Pathak, Unnat Jain, Alexander C. BergDecoder-Only Language ModelsAutoencoders

  6. Breaking the Token Ceiling: Distilling Smaller, Stronger Byte Models

    Sep 14, 2026Kalyani Marathe, Artidoro Pagnoni, Tomasz Limisiewicz +4Decoder-Only Language ModelsLanguage Model Scaling Laws

  7. How Perturbations Propagate: A Multi-Level Analysis of Robustness in Large Language Models

    Sep 3, 2026Dun Li Chan, Emily Liu, Niyathi Allu +1Decoder-Only Language ModelsLanguage Model Robustness

  8. Generative vs. Encoder Models for Multilingual NER: A Comprehensive Empirical Study on Naamapadam

    Aug 30, 2026Jakkala Mahesh, Jatavath Shravan Kumar, Komalla Shivani +1Named Entity RecognitionMultilingual Language Model Evaluation

  9. Ask Self, Ask Others: Relation Is All You Need

    Aug 20, 2026Yuting Ge, Pengju Yang, Mingkai NieSelf-AttentionDecoder-Only Language Models

  10. LoKiFormer: Locality-aware Attention with Decoupled Knowledge Memory for Efficient Large Language Model Pretraining

    Aug 12, 2026Qiuwu Chen, Zimo Liu, Yuchen Li +8Language Model PretrainingMemory-Augmented Language Models

  11. Why Post-Norm Transformers Collapse: Attention Amplification and Gradient Repair Failure

    Aug 10, 2026Xingjian Wang, Qingyu Han, Xiaodong Luo +1TransformerSelf-Attention

  12. Is SwiGLU's Open Positive Tail Necessary? Evidence from Closed-Tail Gating with MemGLU

    Aug 7, 2026Yuting Ge, Pengju Yang, Mingkai NieLanguage Model PretrainingDecoder-Only Language Models

  13. When Attention Goes Blind: Numerical Failure in ALiBi Positional Encodings

    Aug 4, 2026Christopher Schröder, Lukas Gienapp, Ferdinand Schlatt +2Language Model PretrainingLong-Context Retrieval

  14. Separating quantum circuits from classical LLMs

    Aug 4, 2026Srinivasan Arunachalam, Arkopal Dutt, Hari Krovi +1Decoder-Only Language ModelsDiffusion Language Models

  15. Divergent large language model predictions from convergent representations in ambiguous word pairs

    Aug 3, 2026K. Jack Scott, Narun Pat, Veronica LiesaputraTransformer InterpretabilityDecoder-Only Language Models

  16. MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities

    Jul 28, 2026Mingqiao Ye, Zhaochong An, Zhitong Gao +11Unified Multimodal ModelsDecoder-Only Language Models

  17. Convolution for Large Language Models

    Jul 20, 2026Yuchuan Tian, Yingte Shu, Wei He +7Decoder-Only Language ModelsConvolutional Neural Networks

  18. A Controlled Study of Attention-Only Transformers

    Jul 20, 2026Henry Ndubuaku, Karen Mosoyan, Jakub Mroz +5Transformer FFNsTransformer

  19. PARTREP: Learning What to Repeat for Decoder-only LLMs

    Jul 2, 2026Andikawati P Widjaja, Yongjun Kim, Hyounghun Kim +1Decoder-Only Language ModelsLLM Prompting