Language Modeling

Ask about this topic

Overview

Language modeling studies how systems learn from sequences, most often by predicting the next token or reconstructing masked tokens, enabling text generation and language understanding. Common approaches include autoregressive Transformers, masked language models, and alternatives such as recurrent or state-space networks, mixture-of-experts models, and diffusion methods. Researchers improve models through data and tokenizer design, continued pretraining, instruction tuning, retrieval, distillation, and compression, evaluating them with perplexity, linguistic tests, and task benchmarks. A central tension is how to make models more efficient and reliable while ensuring benchmark success reflects robust reasoning, generalization, and competence across languages and domains.

Latest papers

CardsList

525 papers73 papers in the last four weeks, up 22% on the four weeks before. 0.5% of all new papers.

  1. Where Should LoRA Go? Component-Type Placement in Hybrid Language Models

    Apr 24, 2026Hector Borobia, Elies Seguí-Mas, Guillermina Tormo-CarbóLow-Rank AdaptationRecurrent Transformers

    VRAIN – Valencian Research Institute for Artificial Intelligence, Universitat Politècnica de València, Valencia, Spain · Department of Economics and Social Sciences, Universitat Politècnica de València, Valencia, Spain · Department of Business Organisation, Universitat Politècnica de València, Valencia, Spain

  2. Promoting Simple Agents: Ensemble Methods for Event-Log Prediction

    Apr 23, 2026Benedikt Bollig, Matthias Függer, Thomas Nowak +1Next-Action PredictionEnsemble Learning

    Université Paris-Saclay, CNRS, ENS Paris-Saclay, LMF, Gif-sur-Yvette, France · Institut Universitaire de France, Paris, France · Direction Générale de l’Armement, Paris, France

  3. Reasoning Primitives in Hybrid and Non-Hybrid LLMs: Do Architectural Differences Yield Advantages in State-Tracking and Recall?

    Apr 23, 2026Shivam Rawat, Lucie Flek, Florian Mai +1CoT ReasoningLLM Memory

    Lamarr Institute for Machine Learning and Artificial Intelligence · Rheinische Friedrich-Wilhelms-Universität Bonn

  4. Working Memory Constraints Scaffold Learning in Transformers under Data Scarcity

    Apr 22, 2026Pranava Madhyastha, Dagmar AdamcovaTransformer AttentionLanguage Modeling

    △City, University of London · ∞The Alan Turing Institute · ⊕Grounded Machines

  5. Verbal tics in frontier language models: A critical review of current releases, research evidence, and public discussion

    Apr 21, 2026Shuai Wu, Xue Li, Zhijun Wang +4LLM EvaluationLLM Sycophancy

  6. OLLM: Options-based Large Language Models

    Apr 21, 2026Shashank Sharma, Janina Hoffmann, Vinay NamboodiriLLM AlignmentRL for Language Model Reasoning

    Department of Computer Science University of Bath · Department of Psychology University of Bath

  7. Dual Alignment Between Language Model Layers and Human Sentence Processing

    Apr 20, 2026Tatsuki Kuribayashi, Alex Warstadt, Yohei Oseki +1Language ModelingComputational Psycholinguistics

    MBZUAI · Tohoku University · UC San Diego +2

  8. JudgeMeNot: Personalizing Large Language Models to Emulate Judicial Reasoning in Hebrew

    Apr 20, 2026Itay Razumenko, Arnon Sturm, Nir GrinbergLegal NLPLLM Personalization

    Computer and Information Science Ben-Gurion University of the Negev Beer-Sheva, Israel

  9. Heterogeneity in Formal Linguistic Competence of Language Models: Is Data the Real Bottleneck?

    Apr 20, 2026H S V N S Kowndinya Renduchintala, Sumit BhatiaLanguage Model PretrainingLanguage Modeling

    Media and Data Science Research Adobe Inc., India

  10. How Tokenization Limits Phonological Knowledge Representation in Language Models and How to Improve Them

    Apr 18, 2026Disen Liao, Freda ShiLLM Fine-TuningLanguage Modeling

    University of Waterloo · Vector Institute

  11. Optimizing Korean-Centric LLMs via Token Pruning

    Apr 17, 2026Hoyeol Kim, Hyeonwoo KimMultilingual Language Model EvaluationLLM Pruning

    College of Computing · Independent Researcher

  12. RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration

    Apr 17, 2026Fabian Ridder, Laurin Lessel, Malte SchillingHallucination in Language ModelsRAG Robustness

    Computer Science Department, University of Münster, Münster, Germany

  13. DALM: A Domain-Algebraic Language Model via Three-Phase Structured Generation

    Apr 17, 2026Chao LiConstrained Generative ModelingKnowledge Conflicts in Language Models

  14. Brain Score Tracks Shared Properties of Languages: Evidence from Many Natural Languages and Structured Sequences

    Apr 16, 2026Jingnong Qu, Ashvin Ranjan, Shane Steinert-ThrelkeldMultilingual Language Model EvaluationLanguage Modeling

    University of Washington

  15. Enhancing Mental Health Counseling Support in Bangladesh using Culturally-Grounded Knowledge

    Apr 16, 2026Md Arid Hasan, Azhagu Meena SP, Aditya Khan +6Retrieval-Augmented GenerationMental Health Counseling

    University of Toronto, Canada · mPower Social Enterprises Ltd, Bangladesh · National Institute of Mental Health, Bangladesh +2

  16. Filling in the Mechanisms: How do LMs Learn Filler-Gap Dependencies under Developmental Constraints?

    Apr 15, 2026Atrey Desai, Sathvik NairSyntactic Generalization in Language ModelsLanguage Modeling

    University of Maryland

  17. Hierarchical vs. Flat Iteration in Shared-Weight Transformers

    Apr 15, 2026Sang-Il HanTransformerRecurrent Transformers

    Korea University of Technology and Education · Biomonster Co.

  18. Three-Phase Transformer

    Apr 15, 2026Mohammad R. Abu AyyashTransformerLanguage Modeling

    Brains Build Research, Ramallah, Palestine

  19. BiSpikCLM: A Spiking Language Model integrating Softmax-Free Spiking Attention and Spike-Aware Alignment Distillation

    Apr 14, 2026Sihang Guo, Chenlin Zhou, Jiaqi Wang +3Spiking Language ModelsSpiking Neural Networks

    Harbin Institute of Technology, Shenzhen, China · School of Electronic and Computer Engineering, Shenzhen Graduate School, Peking University, Shenzhen, China · Peng Cheng Laboratory, Shenzhen, China

  20. Agentic Control in Variational Language Models

    Apr 14, 2026Yves RuffenachLanguage Model-Based ControlLanguage Modeling

    Conservatoire National des Arts et Métiers, Strasbourg, France

  21. Mining Large Language Models for Low-Resource Language Data: Comparing Elicitation Strategies for Hausa and Fongbe

    Apr 14, 2026Mahounan Pericles Adjovi, Roald Eiselen, Prasenjit MitraLow-Resource Language ProcessingLLM Prompting

    Carnegie Mellon University Africa, Kigali, Rwanda · Centre for Text Technology, North-West University, Potchefstroom, South Africa

  22. SCRIPT: A Subcharacter Compositional Representation Injection Module for Korean Pre-Trained Language Models

    Apr 14, 2026SungHo Kim, Juhyeong Park, Eda Atalay +1Language ModelingCompositional Representation Learning

    Department of Artificial Intelligence, Korea University, Seoul, South Korea · Department of Computer Science and Engineering, Korea University, Seoul, South Korea

  23. LASA: Language-Agnostic Semantic Alignment at the Semantic Bottleneck for LLM Safety

    Apr 13, 2026Junxiao Yang, Haoran Liu, Jinzhe Tu +9LLM Safety AlignmentCross-Lingual Representation Alignment

    The Conversational AI (CoAI) group, DCST, Tsinghua University · Alibaba Group · Tsinghua University

  24. SHARE: Social-Humanities AI for Research and Education

    Apr 13, 2026João Gonçalves, Sonia de Jager, Petr Knoth +2Language Modeling

    Erasmus University Rotterdam, NL · Open University, UK

  25. Advancing Polish Language Modeling through Tokenizer Optimization in the Bielik v3 7B and 11B Series

    Apr 12, 2026Krzysztof Ociepa, Łukasz Flis, Remigiusz Kinas +2Language ModelingSubword Tokenization

    SpeakLeash · Azurro · ACK Cyfronet AGH +2

  26. From GPT-3 to GPT-5: Mapping their capabilities, scope, limitations, and consequences

    Apr 11, 2026Hina Afridi, Habib Ullah, Sultan Daud Khan +1Language ModelingGenerative AI Evaluation

    MediaFutures, Department of Information Science and Media Studies, University of Bergen, Bergen, Norway. · Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway. · Faculty of Computing and Information Technology, Sohar University, Sohar, Oman. +2

  27. The Rise and Fall of GG in AGI

    Apr 10, 2026David C. KrakauerArtificial General IntelligenceGenerative AI Evaluation

    Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM. 87501. USA

  28. You Can't Fight in Here! This is BBS!

    Apr 10, 2026Richard Futrell, Kyle MahowaldLanguage Modeling

    University of California Irvine, USA · The University of Texas at Austin, USA

  29. EvoLen: Evolution-Guided Tokenization for DNA Language Model

    Apr 9, 2026Nan Huang, Xiaoxiao Zhou, Junxia Cui +4Language ModelingSequence Modeling

    University of California, San Diego, La Jolla, CA 92093, USA · Washington University in St. Louis, St. Louis, MO 63130, USA

  30. EXAONE 4.5 Technical Report

    Apr 9, 2026Eunbi Choi, Kibong Choi, Sehyun Chun +55Vision-Language ModelsLarge Vision-Language Models