Overview

Language modeling studies how systems learn from sequences, most often by predicting the next token or reconstructing masked tokens, enabling text generation and language understanding. Common approaches include autoregressive Transformers, masked language models, and alternatives such as recurrent or state-space networks, mixture-of-experts models, and diffusion methods. Researchers improve models through data and tokenizer design, continued pretraining, instruction tuning, retrieval, distillation, and compression, evaluating them with perplexity, linguistic tests, and task benchmarks. A central tension is how to make models more efficient and reliable while ensuring benchmark success reflects robust reasoning, generalization, and competence across languages and domains.

Latest papers

CardsList

549 papers41 papers in the last four weeks, down 49% on the four weeks before. 0.3% of all new papers.

  1. Wave Is All You Need: A Quantum Language Model Based on Fully Connected Quantum Boltzmann Machine

    Oct 25, 2026Jun LiuLanguage ModelingQuantum NLP

  2. CoLink: Global Compression and Knowledge Linking for Multi-Turn Dialogue Modeling

    Oct 25, 2026Lian Yan, Zhenbo Xia, Haotian Wang +4Language ModelingLLM Context Compression

  3. Rectified Language Flow

    Oct 24, 2026Baiyu Su, Runlong Liao, Lizhang Chen +1Language ModelingNon-Autoregressive Text Generation

  4. Cola2: Co-Adaptive Hierarchical Language Modeling with Continuous Semantic Compression and Balanced Discrete Generation

    Oct 22, 2026Hongcan Guo, Yian Zhao, Feng Wang +4Language ModelingDiffusion Language Model Decoding

  5. Mixing Matters: Evidence Position Bias Across Sequence Mixers in Long-Context Question Answering

    Oct 22, 2026Aman Behera, Namit Solanki, Mehul AnandState Space Language ModelsLong-Context Language Modeling

  6. ULME: Predictive Coupling for Mutually Reinforcing Embedding and Language Modeling

    Oct 22, 2026Yang Liu, Zhiqiang Yuan, Shu-Dong Huang +5Text EmbeddingsText Embedding Models

  7. GoiBERT: Compact Vietnamese Language Models via Two-Stage Knowledge Distillation

    Oct 21, 2026Quang Anh Kiet Tran, Nam Anh Tran, Manh Hung Trinh +3Small Language ModelsLanguage Modeling

  8. Large Flow Language Models

    Oct 20, 2026Arnav Chavan, Chanhyuk Lee, Udbhav Bamba +4Language ModelingFlow-Based Generative Modeling

  9. BCT: Bayes-Consistent Tempering for Flow Language Models

    Oct 20, 2026Daisuke Oba, Hiroki Furuta, Naoaki OkazakiLanguage ModelingFlow-Based Generative Modeling

  10. Scaling Synthetic Pre-training Data and Evaluation for Indic LLMs

    Oct 19, 2026Abhinav Khattar, Aishwarya Chhabra, Ameya Velingker +24Language ModelingLanguage Model Pretraining

  11. Modal Sense Disambiguation in Language Models: The Role of Context, Training Domain, and Input Frequency

    Oct 19, 2026Ezgi Başar, Yevgen Matusevych, Annemarie van Dooren +1Word Sense DisambiguationLanguage Modeling

  12. Weighted State Souping for Efficient Long-Context Retrieval

    Oct 19, 2026William L. Tong, Aryo Lotfi, Emmanuel Abbe +3State Space Language ModelsLong-Context Retrieval

  13. Aligning Cultural Behaviors in Arabic Language Models via Preferences

    Oct 18, 2026Junhong Liang, Daniil Orel, Saeed Almheiri +11Arabic Language ModelsCultural Alignment

  14. Prosodic Structure as a Critical Component in Phonemic Language Modelling at BabyLM Scale

    Oct 16, 2026Bill K. S. Ku, Janet B. PierrehumbertLanguage ModelingMasked Language Modeling

  15. Chinese Decision: A Family of Language Models for Single-Pass Probabilistic Decision-Making

    Oct 16, 2026Qiang Yan, Jiafeng GuoLanguage ModelingProbabilistic Modeling

  16. FederatedBytes: Tokenizer-Free, Byte-Level Small Language Models for Efficient Multimodal Federated Learning

    Oct 16, 2026Ali Sarabadani, Massimo Piccardi, Mehran Abolhasan +3Byte-Level Language ModelMultimodal Federated Learning

  17. Intern-Decision: Efficient Multimodal Decision Making with Autoregressive Masked Language Modeling

    Oct 15, 2026Tong Zhu, Zhe Xu, Zefeng He +7LLM Decision-MakingEfficient Multimodal Inference

  18. Wolof-Adapted Industry-Ready Open-Weight Language Models

    Oct 14, 2026Gwénolé Lecorvé, Delphine Charlet, Pierre Adam +15Language ModelingOpen-Weight Language Models

  19. Manacá-1B-Instruct-2: Reproducible Safety Alignment for a Native Brazilian-Portuguese LLM. Negative DPO Results, a Controlled Composition Scan, and a Silent Failure Mode of the LLM Judge

    Oct 14, 2026Bruno Leonardo Santos Menezes, Carlos Leonardo Souza Cardoso, Fabio André Machado PortoPortuguese Language ModelsLLM Safety Alignment

  20. The COUNT: Counting N-grams Unlocks Scaling on Host Memory and Training-Free Adaptation

    Oct 14, 2026Ian Magnusson, Noah A. Smith, Pang Wei KohLanguage ModelingLanguage Model Scaling

  21. KoAGE: Do Large Language Models Know, Learn, and Coin Korean Neologisms?

    Oct 14, 2026SungHo Kim, Eojin Jeon, SangKeun LeeLanguage ModelingLanguage Model Generation Evaluation