Knowledge Augmentation for Language Models

Follow this topic Ask about this topic

The field studies how language models can acquire and use information beyond what is already stored in their parameters, especially for changing facts, specialized domains, and multi-step tasks. Common approaches retrieve text or knowledge-graph evidence at inference time, maintain external memories, or inject knowledge through continued pretraining, supervised tuning, adapters, and synthetic data; agents may also decide when to search or use tools. Work increasingly tests not just whether evidence is available but whether models select and integrate it, using knowledge-intensive question answering, long-context, coding, medical, legal, and reasoning benchmarks. A central tension is choosing between retrieval and internalization: added context can be noisy or ignored, while weight updates can be costly and erode general abilities, so the best method depends on model scale, evidence quality, update needs, and compute.

Latest papers

180 papers18 papers in the last four weeks, up 80% on the four weeks before. 0.1% of all new papers.

Encode Once, Answer Many: Internalizing Context via Parametric Flow
Oct 28, 2026Xinhao Zhong, Mengxuan Wu, Chonghua Liao +22Knowledge Augmentation for Language Models
Tencent · UT Austin · NTU +5
Two Birds with One Stone: Skill-Augmented Generation via Unified Retrieval and Utilization
Oct 28, 2026Zihan Luo, Jiahui Kang, Wei Zhou +3Knowledge Augmentation for Language Models RL for RAG
Chongqing University, Chongqing, China · Huazhong University of Science and Technology, Wuhan, China
Weakness Localization Is Not Enough: State-Aware Recursive Knowledge Improvement for Large Language Models
Oct 27, 2026Runze Jiang, Yan Wang, Boran Jin +2Knowledge Augmentation for Language Models Knowledge Editing
Peking University · Alibaba Group · Delft University of Technology
MARQ: A Modular, LLM-Agnostic Requirement Quality Assessor Infused with Systems Engineering Knowledge
Oct 26, 2026Jannatul Shefa, Alejandro Salado, Paul Wach +1Requirements Engineering Knowledge Augmentation for Language Models
Grado Department of Industrial and Systems Engineering, Virginia Tech, Blacksburg, Virginia, USA · Department of Systems and Industrial Engineering, University of Arizona, Tucson, Arizona, USA
FinLLM-KG: Knowledge Graph-Enhanced LLMs for Financial Sentiment Analysis in Algorithmic Trading
Oct 26, 2026Giorgos Iacovides, Oliver Hannibal, Wuyang Zhou +1Financial Sentiment Analysis Knowledge Augmentation for Language Models
Department of Electrical and Electronic Engineering, Imperial College London, UK
CoLink: Global Compression and Knowledge Linking for Multi-Turn Dialogue Modeling
Oct 25, 2026Lian Yan, Zhenbo Xia, Haotian Wang +4Language Modeling LLM Context Compression
Research Center of Language Technology, Faculty of Computing, Harbin Institute of Technology, Harbin, Heilongjiang 150001, China · Institute for Advanced Algorithms Research, Shanghai, 201306, China · Research Center of Artificial Intelligence of Things, Faculty of Computing, Harbin Institute of Technology, Harbin, Heilongjiang 150001, China, and also with the National Key Laboratory of Smart Farm Technologies and Systems, Harbin, Heilongjiang 150001, China
Are We Done with UMLS Injection into Language Models for Medical Question Answering?
Oct 24, 2026Yiran Zeng, Giacomo Frisoni, Lorenzo Molfetta +1Knowledge Augmentation for Language Models Medical QA
Department of Computer Science and Engineering, University of Bologna, Italy
A Reality Check of Inference-Time Context Augmentation Methods
Oct 22, 2026Haeun Yu, Geert Heyman, Pepa Atanasova +1Knowledge Augmentation for Language Models LLM Reliability
University of Copenhagen, Copenhagen, Denmark · Nokia Bell Labs, Antwerp, Belgium
Every Part Has Its Role: Part-of-Speech Effects in LLM Post-Training
Oct 21, 2026Laya Iyer, Chen ShaniLanguage Model Post-Training Knowledge Augmentation for Language Models
Stanford University · Tel Aviv University
MapSteal: Map-based Autonomous Knowledge Stealing Attack Against Large Language Models
Oct 19, 2026Zhengao Li, Chuxu Zhang, Yushun DongKnowledge Augmentation for Language Models LLM Training
University of Connecticut, Storrs, CT, USA · Florida State University, Tallahassee, FL, USA
Mixture-of-Memories: Low-Overhead Language Modeling with Modular Knowledge Injection
Oct 18, 2026Yifan Lu, Rubin Wei, Ziyun Zou +8Memory-Augmented Language Models Knowledge Augmentation for Language Models
MBZUAI, Abu Dhabi, UAE · LUMIA Lab, School of Artificial Intelligence, Shanghai Jiao Tong University, China · The Chinese University of Hong Kong, Hong Kong +1
Deep Fusion Memory: A Chunk-Level Parametric Memory for Large Language Models
Oct 18, 2026Rubin Wei, Siyuan Sun, Hao Sun +5Memory-Augmented Language Models Knowledge Augmentation for Language Models
LUMIA Lab, School of Artificial Intelligence, Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory
Cache2LoRA: Understanding and Improving Context Internalization
Oct 16, 2026Yi Fang, Gaurav Srivastava, Zhenyu Bi +2LLM Context Compression Knowledge Augmentation for Language Models
Department of Computer Science, Virginia Tech, Blacksburg, VA, USA · Department of Biomedical Data Science, Stanford University, Stanford, CA, USA
Writability and the Pathway: A Controlled Study of Knowledge Injection into Addressed Memory
Oct 15, 2026Wei Gao, Cheng Tian, Duan Yang +1Knowledge Augmentation for Language Models Knowledge Editing
Shibei Qiyuan AI Technology · Xi'an Jiaotong University
Enhancing Legal Reasoning in Pre-trained Language Models via Knowledge Graph-Guided Multi-Task Pre-training
Oct 9, 2026Ibrahim Siddig, Munir GeorgesLegal NLP Language Model Pretraining
AImotion Bavaria, Technische Hochschule Ingolstadt
RAGenome: Scaling Retrieval-Based Genomic Language Models to Long Contexts
Oct 8, 2026Frederikke Isa Marin, Panagiotis Antoniadis, Dionysia Danai Brilli +5Genomics Bioinformatics
University of Copenhagen · Novo Nordisk A/S · Technical University of Denmark
UNREAL: Unifying Retrieval and Long-Context with a Single Model
Oct 6, 2026Edan Kinderman, Elad Hoffer, Yochai Blau +4Retrieval-Augmented Generation Knowledge Augmentation for Language Models
NVIDIA · Technion
From Papers to Mechanisms: An Evidence-Grounded Knowledge Substrate for Scientific Language Models
Oct 5, 2026Qiuhui Chen, Yibo Liu, Tao Dai +3Knowledge Augmentation for Language Models Scientific QA
School of Information Science and Engineering, East China University of Science and Technology · School of Computer Science, Shanghai Jiao Tong University · Tencent
From Knowledge Access to Source Learning: Developing Source-Specific Competence
Oct 1, 2026Lucheng Fu, Kejing Xia, Yiyang Wang +10Continual Learning for LLM Agents Knowledge Augmentation for Language Models
Georgia Institute of Technology · University of Illinois at Urbana-Champaign · University of California, Los Angeles
On the (In)effectiveness of AMR Augmentation for Large Language Models
Sep 30, 2026Hoa Quynh Nhung Nguyen, Jacopo Staiano, Michael SullivanLLM Evaluation Knowledge Augmentation for Language Models
Saarland University · University of Trento
Offline Guidance, Online Reasoning: Reusing LLM Feedback for Small Language Models
Sep 30, 2026Bohan Zhang, Linan Yue, Weibo Gao +3Efficient Language Model Reasoning Small Language Models
Southeast University · Hong Kong Polytechnic University · ZTE Corporation
Draft-KV: Learning Useful Latent Communication Between Language Models
Sep 28, 2026Linquan Wu, Shichang Meng, Tianxiang Jiang +7Knowledge Augmentation for Language Models Multi-LLM Collaboration
City University of Hong Kong · University of Science and Technology of China · University of Electronic Science and Technology of China +3
When Does Structured Knowledge Help Neural Theorem Proving?
Sep 28, 2026Sareh Nabi, Roland Vogl, Marzieh NabiAutomated Theorem Proving Knowledge Augmentation for Language Models
Stanford University
Context Spanning: A Communication Framework for Full-Duplex Speech Models and External LLM Backends
Sep 27, 2026Seonghyeon Go, Yongwoo Kim, Hyeonjin Cha +1Knowledge Augmentation for Language Models Spoken Dialogue Systems
Mindlogic
Semantic Abstraction for Natural Language Inference: a Methodological Framework for Discovering and Compensating Semantic Knowledge and Reasoning Gaps in Large Language Models
Sep 22, 2026David Torres-Moreno, Jorge Hermosillo-ValadezKnowledge Augmentation for Language Models Natural Language Inference
GraphSkillAA: Attribution-Guided Skill-Graph Updating with Targeted Validation and Rollback
Sep 17, 2026Ziqiao Shang, Ling-Yue Ge, Lan-Zhe GuoKnowledge Augmentation for Language Models LLM Agent Skill Learning
National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · School of Intelligence Science and Technology, Nanjing University, Suzhou, China
Where Should a Document Live: Context, Representations, or Parameters?
Sep 15, 2026Nathanaël Carraz Rakotonirina, Momchil Hardalov, Gonzalo Iglesias +1Memory-Augmented Language Models Knowledge Augmentation for Language Models
Amazon AGI
PunGraph: Retrieval-Enhanced Phonetic-Semantic Graph Reasoning for Pun Understanding
Sep 15, 2026Yuchen Su, Zijian Huang, Yaotian Shi +6Figurative Language Understanding Knowledge Augmentation for Language Models
School of Computer Science, University of Auckland, New Zealand · School of Computer and Information Technology, Shanxi University · School of Electronic Engineering and Computer Science, Queen Mary University of London
Repair Before Reinforce: Context-Augmented Knowledge Graph Reasoning for Multi-Hop Question Answering
Sep 10, 2026Tharaka D. Fonseka, Niraj K. JhaMulti-Hop QA Knowledge Augmentation for Language Models
Dept. of Electrical and Computer Engineering, Princeton University
KuaiRP Series Role-playing Models Technical Report
Sep 10, 2026Yipeng Wang, Ziwei Zhang, Jiahui Zhang +2Large Language Model-Based Role-Play Simulation Knowledge Augmentation for Language Models
Kuaishou GameMind Lab

Oct 28, 2026J/K move · Enter open · S save

Encode Once, Answer Many: Internalizing Context via Parametric Flow

Xinhao Zhong, Mengxuan Wu, Chonghua Liao, Linxi Xie+21

Tencent · UT Austin · NTU+5

Knowledge Augmentation for Language Models

Open paper AskPDF arXiv

Large language models are typically adapted to a task either by placing knowledge in the input context, which keeps it external and re-encodes it on every call, or by fine-tuning, which internalizes it but demands a costly round of optimization per task. We aim to internalize the knowledge carried by a context directly into model parameters, without per-task optimization and without keeping the context in the input at inference time. To this end, we propose CLIK, a context-conditioned parameter generator that maps a given context to LoRA parameters for internalizing knowledge in a single forward pass through a Transformer encoder with cross-attention, requiring no test-time gradient updates. Once the generated LoRA is inserted into a frozen base model, the model answers downstream queries from the query alone, having internalized the context into its weights rather than re-reading it. Because the adapter depends only on the context, it is generated once and reused for every query on the same document, so the cost of reading that document is amortized across queries. On document question answering, CLIK substantially improves over the no-context base model and outperforms existing hypernetwork-based generators, while remaining behind a base model that reads the context directly, and it carries over to multi-turn dialogue without any multi-turn-specific training. Analysis further shows that the context-dependent part of the generated adapter concentrates in a compact and specific substructure.