cs.CLOct 10, 2025

Demasking Fine-Tuning for Knowledge Injection in Autoregressive and Diffusion LLMs

Authors: Xu Pan, Ely Hahami, Jingxuan Fan, Ziqian Xie, Haim Sompolinsky

Organizations: Harvard University · Edmond and Lily Safra Center for Brain Sciences, Hebrew University

Abstract

Large language models (LLMs) are often used in environments where facts evolve, yet factual knowledge updates via fine-tuning on unstructured plain text often suffer from 1) reliance on paraphrasing augmentation, without which training does not generalize to downstream QA tasks, and 2) the reversal curse. Recent evidence of the sample efficiency of masked diffusion large language models (dLLMs) and the bidirectional information flow in their demasking training objective suggests that the demasking objective may facilitate knowledge learning from raw knowledge text. We test this hypothesis in controlled knowledge fine-tuning experiments on dLLMs and autoregressive LLMs (arLLMs) and find that while arLLMs rely on paraphrase augmentation to generalize knowledge text into question-answering (QA) capability, dLLMs do not require paraphrases to achieve high QA accuracy. We find that giving arLLMs a demasking objective, where the LLM must reconstruct the original text from a masked copy given in context, improves the efficacy of knowledge injection, removing the need for paraphrases and mitigating the reversal curse, recovering most of the gap between arLLMs and dLLMs. We also provide preliminary evidence of the broader applicability of using the demasking objective in arLLM fine-tuning on a large-scale knowledge-intensive dataset.

Figures & tables

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Extracting Training Data from Diffusion Language Models via Infilling

    May 22, 2026Yihan Wang, N. AsokanData LeakageMasked Diffusion Models

  2. KItCAT: Knowledge Injection via Input Corruption for Auto-regressive Training

    Aug 31, 2026Meghanadh Pulivarthi, Kushagra Bhushan, Vineet Kumar +5Language Model PretrainingKnowledge Augmentation for Language Models

  3. MoRFI: Monotonic Sparse Autoencoder Feature Identification

    Apr 29, 2026Dimitris Dimakopoulos, Shay B. Cohen, Ioannis KonstasLLM Hallucination MitigationLLM Interpretability