Autoregressive Language Modeling

Ask about this topic

Momentum

9 papers in the last four weeks, up 50% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 81

All topics
CardsList

  1. The Truth Lies Somewhere in the Middle (of the Generated Tokens)

    May 11, 2026Sophie L. Wang, Phillip Isola, Brian CheungLLM InterpretabilityAutoregressive Language Modeling

    1MIT.

  2. Bias by Necessity: Impossibility Theorems for Sequential Processing with Convergent AI and Human Validation

    May 9, 2026Jikun Wu, Dongxin Guo, Siu-Ming YiuComputational Cognitive ModelingAutoregressive Language Modeling

    1Stellaris AI Limited, Hong Kong, China · 3The University of Hong Kong, Hong Kong, China

  3. Structured Recurrent Mixers for Massively Parallelized Sequence Generation

    May 9, 2026Benjamin L. BadgerRecurrent Neural NetworksAutoregressive Language Modeling

    IBM

  4. Encoder-Decoder Transformers: Logical Characterizations and Periodicity

    May 8, 2026Veeti Ahvonen, Damian Heiman, Antti Kuusisto +2Transformer ExpressivityTransformer

    Mathematics Research Centre, Tampere University, Tampere, Finland

  5. Limits of Reliability and Scaling in Language Models

    May 8, 2026Subhabrata MajumdarLanguage Model Scaling LawsAutoregressive Language Modeling

    Indian Institute of Management Bangalore Bengaluru, India

  6. Matrix-Decoupled Concentration for Autoregressive Sequences: Dimension-Free Guarantees for Sparse Long-Context Rewards

    May 7, 2026Pei-Sen LiLong-Context Language ModelingAutoregressive Language Modeling

    School of Mathematics and Statistics, Beijing Institute of Technology, Beijing 100872, China

  7. Contextual Memory-Enhanced Source Coding for Low-SNR Communications

    May 6, 2026Ziqiong Wang, Rongpeng LiMemory-Augmented Language ModelsAutoregressive Language Modeling

    College of Information Science and Electronic Engineering, Zhejiang University

  8. Perturbation is All You Need for Extrapolating Language Models

    May 5, 2026Zetai Cen, Jin Zhu, Xinwei Shen +1OOD GeneralizationAutoregressive Language Modeling

    School of Mathematics, University of Bristol · School of Mathematics, University of Birmingham · Department of Statistics, University of Washington +1

  9. Memory as a Markov Matrix: Sample Efficient Knowledge Expansion via Token-to-Dictionary Mapping

    May 5, 2026Kaustubh Pethkar, Ziyang Xiong, Zuofeng Shang +1Continual Learning for LLMsMemorization in Language Models

    New Jersey Institute of Technology · University of California, Berkeley

  10. Characterizing the Expressivity of Local Attention in Transformers

    May 1, 2026Jiaoda Li, Ryan CotterellTransformer ExpressivityTransformer

  11. Caracal: Causal Architecture via Spectral Mixing

    Apr 30, 2026Bingzheng Gan, Tianyi Zhang, Yusu Li +4Long-Context Language ModelingAutoregressive Language Modeling

    Huawei Technologies Co., Ltd., China.

  12. Representational Curvature Modulates Behavioral Uncertainty in Large Language Models

    Apr 27, 2026Jack King, Evelina Fedorenko, Eghbal A. HosseiniNeural Representation GeometryAutoregressive Language Modeling

    Brain and Cognitive Sciences Massachusetts Institute of Technology Cambridge, MA 02139 · Google DeepMind

  13. The Recurrent Transformer: Greater Effective Depth and Efficient Decoding

    Apr 23, 2026Costin-Andrei Oncescu, Depen Morwani, Samy Jelassi +3Efficient Transformer InferenceSelf-Attention

    Harvard University

  14. StarTSE: Towards Streaming Target Speaker Extraction via Chunk-wise Interleaved Splicing of Autoregressive Language Model

    Apr 21, 2026Shuhai Peng, Hui Lu, Jinjiang Liu +8Target Speaker ExtractionSpeech Processing

    Shenzhen International Graduate School, Tsinghua University · The Chinese University of Hong Kong · SenseTime Research

  15. Generic Triple-Latent Compression with Gated Associative Retrieval

    Apr 17, 2026Liu XiaoMemory-Augmented Language ModelsAutoregressive Language Modeling

  16. Hallucination as Trajectory Commitment: Causal Evidence for Asymmetric Attractor Dynamics in Transformer Generation

    Apr 16, 2026G. Aytug AkarlarTransformer InterpretabilityHallucination in Language Models

    Chimera Research Initiative Istanbul, Türkiye · Chimera Research Initiative is an independent research effort exploring causal and neuro-symbolic approaches to AI systems.

  17. Invertible Query-Key Coupling Composes with Attention Mechanisms

    Apr 2, 2026Barak Gahtan, Alex M. BronsteinInvertible Neural NetworksSelf-Attention

    Technion – Israel Institute of Technology · ISTA – Institute of Science and Technology Austria

  18. Mechanism Shift During Post-training from Autoregressive to Masked Diffusion Language Models

    Jan 21, 2026Injin Kong, Hyoungjoon Lee, Yohan JoMasked Diffusion ModelsAutoregressive Diffusion

    Graduate School of Data Science, Seoul National University · Department of Biosystems & Biomaterials Science and Engineering, Seoul National University

  19. Selective Rotary Position Embedding

    Nov 21, 2025Sajad Movahedi, Timur Carstensen, Arshia Afzal +3Softmax AttentionRotary Positional Embeddings

    ELLIS Institute Tübingen1, LIONS, EPFL2, University of Freiburg3, Max-Planck-Institute for Intelligent Systems4, Prior Labs5

  20. Robust Filter Attention: Self-Attention as Precision-Weighted State Estimation

    Sep 4, 2025Peter RacioppoSelf-AttentionLong-Context Language Modeling

    Independent Researcher, Los Angeles, CA, USA.

  21. Any-Order GPT as Masked Diffusion Model: Decoupling Formulation and Architecture

    Jun 24, 2025Shuchen Xue, Tianyu Xie, Tianyang Hu +5Masked Diffusion ModelsDecoder-Only Language Models

    University of Chinese Academy of Sciences · Academy of Mathematics and Systems Science, Chinese Academy of Sciences · School of Mathematical Sciences, Peking University +2