Overview

Language modeling studies how systems learn from sequences, most often by predicting the next token or reconstructing masked tokens, enabling text generation and language understanding. Common approaches include autoregressive Transformers, masked language models, and alternatives such as recurrent or state-space networks, mixture-of-experts models, and diffusion methods. Researchers improve models through data and tokenizer design, continued pretraining, instruction tuning, retrieval, distillation, and compression, evaluating them with perplexity, linguistic tests, and task benchmarks. A central tension is how to make models more efficient and reliable while ensuring benchmark success reflects robust reasoning, generalization, and competence across languages and domains.

Latest papers

CardsList

549 papers41 papers in the last four weeks, down 49% on the four weeks before. 0.3% of all new papers.

  1. The Depth of a Prediction: Where Language Models Resolve a Word Predicts Human Reading Time

    Nov 19, 2026Donato Riccio, Fabrizio MaturoLanguage ModelingLLM Interpretability

  2. Joint Language Modeling for In-the-Wild Speech-Scene Generation with Distinct Token Representations

    Nov 19, 2026Semin Kim, Ju Yeon Kang, Sunghwan Ahn +2Audio GenerationSpeech Language Models

  3. Polarity Composition Hypothesis: Causal Evidence from Negation in NLI

    Nov 17, 2026Nhi Hoai Doan, Tatsuya Hiraoka, Munachiso Samuel Nwadike +1Natural Language InferenceLanguage Modeling

  4. What's in a Tree ? Hierarchical nn-gram models as controlled approximators of language

    Nov 17, 2026Aritra Das, Francesco Cagnetta, Darshil Doshi +3Language ModelingAutoregressive Language Modeling

  5. Architectures and KV Replay for Depth-Recurrent Language Models

    Nov 14, 2026Lucia Licakova, Niklas Schmidinger, Lukas Hauzenberger +2Recurrent TransformersKV-Cache Management

  6. LECAT: Measuring LLM Category Learning from Pretraining Data

    Nov 13, 2026Jimin Mun, Maarten Sap, Yanai ElazarLanguage Model PretrainingLanguage Modeling

  7. Dancing Tokens on the Vocabulary Network from LLM Generation

    Nov 13, 2026Yuan-Chao HuLLM InterpretabilityRepresentation Geometry in Language Models

  8. Rectified Looped Transformer

    Nov 12, 2026Musheng Li, Jiawei Zhang, Ziyuan Liu +1Recurrent TransformersLanguage Modeling

  9. Better Boundary Prediction Does Not Guarantee Better Structural Representation

    Nov 11, 2026Michael Zhou, Markus Frohmann, Junghyun Min +1Language Modeling

  10. Differential Backpack Language Models: Sense Inhibition via Signed Contextualization Weights

    Nov 11, 2026Konstantinos Skianis, Giannis Nikolentzos, Polykarpos Meladianos +1Language ModelingLLM Interpretability

  11. VWEN: A Curated Small Language Model for Vietnamese Paraphrasing

    Nov 11, 2026Sang Quang NguyenLanguage Model Post-TrainingText Generation

  12. Adapting Small Language Models to Romanized Nepali: Tokenizer Fertility, Reference Likelihood, and Generation Metrics

    Nov 9, 2026Pujan Pant, Sagar Newpane, Hari Sitaula +1Language ModelingSmall Language Models

  13. onóna: Twin Embeddings with Tied Unembeddings

    Nov 7, 2026Cheng Luo, Xinnan Dai, Zhongzhu Zhou +4Language Modeling

  14. Compact Greek Language Models from Scratch: Pretraining 500M- and 1B-Parameter Decoder-Only Transformers

    Nov 6, 2026Konstantinos Perifanos, Anne Laure Poite, Giannis Tsilikas +3Language ModelingLanguage Model Pretraining

  15. Learning of Tree Automata applied to Neural Language Acceptors

    Nov 6, 2026Juan Pedro da Silva, Sergio YovineAutomata LearningLanguage Modeling

  16. Retrofitting Unbounded Context onto Stock Language Models

    Nov 6, 2026Mark OskinLong-Context Language Model InferenceLanguage Modeling

  17. AssameseLM-E2B: An Open-Weight Assamese Language Model for Efficient Consumer-Grade Deployment

    Nov 4, 2026Pragyan PhukanLanguage ModelingLow-Resource Language Processing

  18. Going Deeper With Memories

    Nov 3, 2026Afshin KhadangiLanguage ModelingByte-Level Language Model

  19. ViAdminShift: A Vietnamese Benchmark for Event-Level Knowledge Updates Built from the 2025 Administrative Reform

    Nov 2, 2026Phan Binh Nguyen Lam, Nguyen Thai AnhFactual Knowledge in Language ModelsLanguage Modeling

  20. GatedV: Self-Gated Values for Transformer Language Models

    Nov 1, 2026Li-Ming Yu, Hongyu ZhouTransformer AttentionGated Attention

  21. Asynchronous Flow Map Language Models

    Oct 30, 2026Chanhyuk Lee, Wonjung Kim, Patrick Pynadath +5Language ModelingOne-Step Generative Modeling

  22. Homebrewed SLMs: Autoregressive and Masked Diffusion Language Models Built at Home for Portuguese

    Oct 29, 2026Pedro Carvalho BromLanguage ModelingSmall Language Models

  23. Dual DeltaNet: Value-Side Memory Reorganization for Linear Attention

    Oct 28, 2026Yixiao Qian, Yaxin Zhang, Song Chen +3Linear AttentionDelta-Rule Linear Attention

  24. StateLoop: Persistent State for Looped Transformers

    Oct 26, 2026Feichen Song, Boyi Zeng, He Li +7Persistent Memory for Language ModelsLanguage Modeling

  25. Synthetic Child-Adult Interactions Are Not Developmentally Plausible

    Oct 26, 2026Bastian Bunzeck, Francesca PadovaniLanguage ModelingSynthetic Data Evaluation