Knowledge Augmentation for Language Models

Overview

The field studies how language models can acquire and use information beyond what is already stored in their parameters, especially for changing facts, specialized domains, and multi-step tasks. Common approaches retrieve text or knowledge-graph evidence at inference time, maintain external memories, or inject knowledge through continued pretraining, supervised tuning, adapters, and synthetic data; agents may also decide when to search or use tools. Work increasingly tests not just whether evidence is available but whether models select and integrate it, using knowledge-intensive question answering, long-context, coding, medical, legal, and reasoning benchmarks. A central tension is choosing between retrieval and internalization: added context can be noisy or ignored, while weight updates can be costly and erode general abilities, so the best method depends on model scale, evidence quality, update needs, and compute.

Latest papers

CardsList

187 papers12 papers in the last four weeks, down 20% on the four weeks before. 0.1% of all new papers.

  1. Knowledge and Reasoning Augmented Learning for Language Model Assisted Antimicrobial Decision Support

    Nov 20, 2025Zhe Li, Yehan Qiu, Yuntian Huang +15Clinical Decision SupportKnowledge Augmentation for Language Models

  2. Zoom In Disparities in Healthcare LLM Q&A

    Oct 20, 2025Ipek Baris Schlicht, Burcu Sayin, Zhixue Zhao +5Multilingual Language Model EvaluationKnowledge Augmentation for Language Models

  3. Instruction Retrieval at Inference Time for Small Language Models

    Oct 15, 2025Kenan Alkiek, David Jurgens, Vinod VydiswaranKnowledge Augmentation for Language ModelsSmall Language Models

  4. Demasking Fine-Tuning for Knowledge Injection in Autoregressive and Diffusion LLMs

    Oct 10, 2025Xu Pan, Ely Hahami, Jingxuan Fan +2Fine-Tuning Language ModelsKnowledge Augmentation for Language Models

  5. All Entities are Not Created Equal: Examining the Long Tail for Ultra-Fine Entity Typing

    Oct 22, 2024Advait Deshmukh, Ashwin Umadi, Dananjay Srinivas +1Language Model PretrainingLong-Tail Learning

  6. FlashBack: Efficient Retrieval-Augmented Language Modeling for Fast Inference

    May 7, 2024Runheng Liu, Xingchen Xiao, Heyan Huang +2Knowledge Augmentation for Language ModelsLLM Inference Acceleration