NanoKnow: How to Know What Your Language Model Knows

Lingwei Gu; Nour Jedidi; Jimmy Lin

cs.CLFeb 23, 2026

NanoKnow: How to Know What Your Language Model Knows

Authors: Lingwei Gu, Nour Jedidi, Jimmy Lin

Organizations: University of Waterloo Waterloo, ON, Canada

Abstract

How do large language models (LLMs) know what they know? Answering this question has been difficult because pre-training data is often a "black box" - unknown or inaccessible. The recent release of nanochat - a family of small LLMs with fully open pre-training data - addresses this as it provides a transparent view into where a model's parametric knowledge comes from. Towards the goal of understanding how knowledge is encoded by LLMs, we release NanoKnow, a benchmark dataset that partitions questions from Natural Questions and SQuAD into splits based on whether their answers are present in nanochat's pre-training corpus. Using these splits, we can now properly disentangle the sources of knowledge that LLMs rely on when producing an output. To demonstrate NanoKnow's utility, we conduct experiments using eight nanochat checkpoints. Our findings show: (1) closed-book accuracy is strongly influenced by answer frequency in the pre-training data, (2) providing external evidence can mitigate this frequency dependence, (3) even with external evidence, models are more accurate when answers were seen during pre-training, demonstrating that parametric and external knowledge are complementary, and (4) non-relevant information is harmful, with accuracy decreasing based on both the position and the number of non-relevant contexts. We release all NanoKnow artifacts at https://github.com/castorini/NanoKnow.

Figures & tables

Figure 1. An example of NanoKnow on a question-answer pair. If any passage is deemed to answer the question after the string match and LLM verify steps, we label the question as “supported”. Otherwise, the question is considered “unsupported”.

Type	Question	Answer	Evidence
Supported	When was the last time anyone was on the moon?	December 1972	“No one has walked on the Moon since December 1972 .”
	What is the main artery that takes blood from the heart to the body?	The aorta	“Arteries begin with the aorta , the large artery leaving the heart.”
Unsupported (Failed LLM Verify)	Who won last year’s NCAA women’s basketball?	South Carolina	“The Wildcats were ranked 8th in the nation before a loss to South Carolina on October 4.”
	Love Yourself by Justin Bieber is about who?	Rihanna	“Other Icelandic artists have followed in his footsteps, collaborating with international acts like Rihanna and Ed Sheeran.”
Unsupported (No String Match)	Who sang I ran all the way home?	The Impalas	–
	Who plays Gram on The Young and the Restless?	Max Shippee	–

Table 1. Examples of supported and unsupported NQ questions. Failed LLM Verify examples contain the gold answer string in FineWeb-Edu but do not directly answer the question.

	NQ	SQuAD
Total QA Pairs	3,610	10,570
String Match only	73.9%	78.9%
String Match $\rightarrow$ LLM Verify (Supported)	66.2%	70.9%

Table 2. Projection rates for NQ and SQuAD on FineWeb-Edu. The reported percentage represents how many questions are supported after String Match only and after String Match followed by LLM Verify.

Figure 2. Distribution of NanoKnow’s supported questions by answer frequency in FineWeb-Edu for NQ and SQuAD.

Figure 3. Influence of pre-training data answer frequency on nanochat’s accuracy. Solid lines show the closed-book prompt setup; dashed lines show w/ FineWeb-Edu context.

			NQ				SQuAD
			Closed-Book		w/ FineWeb Context		Closed-Book		w/ FineWeb Context		w/ Original Context
	Model Checkpoint	Model Size	EM	LLM-Judge	EM	LLM-Judge	EM	LLM-Judge	EM	LLM-Judge	EM	LLM-Judge
1	sampathchanda/nanochat-d20	561M	0.004	0.008	0.022	0.018	0.004	0.002	0.021	0.019	0.086	0.081
2	shu127/nanochat-d20		0.003	0.005	0.021	0.016	0.003	0.011	0.016	0.013	0.042	0.040
3	pankajmathur/nanochat-d20		0.003	0.014	0.028	0.023	0.005	0.008	0.022	0.020	0.056	0.052
4	karpathy/nanochat-d32	1.9B	0.196	0.224	0.468	0.476	0.114	0.173	0.465	0.540	0.672	0.736

Table 3. Comparing closed-book versus open-book QA over the NQ and SQuAD supported splits of NanoKnow.

		Supported		Unsupported
Model Checkpoint	Model Size	EM	LLM-Judge	EM	LLM-Judge
sampathchanda/nanochat-d20	561M	0.086	0.081	0.069	0.068
shu127/nanochat-d20		0.042	0.040	0.032	0.031
pankajmathur/nanochat-d20		0.056	0.052	0.051	0.054
karpathy/nanochat-d32	1.9B	0.672	0.736	0.554	0.688
Antigma/nanochat-d32	1.9B	0.686	0.740	0.553	0.680

Table 4. QA accuracy for supported versus unsupported questions on SQuAD (w/ Original Context).

		Far: [A, D, Q]				Mid: [D, A, D, Q]				Near: [D, A, Q]
		NQ		SQuAD		NQ		SQuAD		NQ		SQuAD
	Setting	EM	LLM-Judge	EM	LLM-Judge	EM	LLM-Judge	EM	LLM-Judge	EM	LLM-Judge	EM	LLM-Judge
1	Closed-Book	0.198	0.226	0.122	0.169	0.198	0.226	0.122	0.169	0.198	0.226	0.122	0.169
2	Distractor only	0.152	0.194	0.091	0.154	0.152	0.194	0.091	0.154	0.152	0.194	0.091	0.154
3	Answer only	0.516	0.492	0.483	0.551	0.516	0.492	0.483	0.551	0.516	0.492	0.483	0.551
4	Answer + 1 Distractor	0.452	0.447	0.411	0.478	N/A				0.456	0.457	0.428	0.501

Table 5. Influence of distractors on nanochat’s ( Antigma/nanochat-d32 ) effectiveness on supported questions. A denotes the answer document, D denotes the distractor document, and Q denotes the question.

Factual Knowledge in Language Models

Factual knowledge in language models concerns what facts they store, how reliably they retrieve them, and how that knowledge can be tested or changed.

See all 193 papers

May 26, 2026

Beyond Questions: Evaluating the Stated Knowledge of LLMs

Parametric knowledge in large language models (LLMs) is a cornerstone of their success, yet remains poorly understood. Existing knowledge benchmarks typically rely on predefined questions (e.g., "What is the birth date of M.L. King?"), evaluating only knowledge that benchmark designers explicitly choose to query, a problematic availability bias. In this paper, we introduce open knowledge evaluation, a new paradigm for benchmarking the stated knowledge of LLMs. Instead of asking narrow questions, it evaluates models on the knowledge they choose to surface in response to open-ended elicitation prompts (e.g., "Tell me everything you know about M.L. King"). This shifts the focus from predefined answer retrieval toward characterizing the knowledge models naturally express. We instantiate this paradigm with BeQu (Beyond Questions), a benchmark of 10,000 entities paired with reference corpora for statement verification. Using BeQu, we evaluate a broad range of language models and analyze the effects of reasoning effort, model scale, prompt format, and knowledge domain. Data and leaderboard are available on this work's GitHub repository and at ANONYMIZED.

Luca Giordano, Simon Razniewski

ScaDS.AI Dresden/Leipzig & TU Dresden, Germany

LLM Evaluation Factual Knowledge in Language Models

Ask PDF

Oct 2, 2026

LUMOS: Tracing Parametric Knowledge from Training Data to Behavioral Outputs in LLMs

Current analyses of LLMs' parametric knowledge are largely output-centric, drawing conclusions about what a model knows without verifying what it was actually trained on. This leaves fundamental questions, such as whether a correct response reflects genuine generalization or rote memorization, grounded in speculation rather than evidence. To resolve these ambiguities, we introduce LUMOS, a diagnostic framework that traces knowledge along the causal chain from training-data exposure to behavioral output, leveraging OLMo 2 with its fully transparent training corpus. By grounding analysis in verified exposure, we reveal that models internally encode rare facts with high separability (84%) yet fail to express them behaviorally (54%), though this retrieval gap narrows with scale. Furthermore, when models are asked to self-reflect on their own answers, they perform reliably on trained content (83%) but drop to random-baseline levels (49%) on unseen content. This collapse persists even under chain-of-thought prompting, which inflates confidence signals rather than improving calibration. Collectively, these findings demonstrate that incorporating the training-data axis into LLM evaluation transforms speculative diagnoses into verifiable claims, and we advocate that this axis should be a standard component of knowledge assessment in LLMs.

Seoyeon Ye, Gayoung Kim, Jiyoung Hong +2

Ewha Womans University

Factual Knowledge in Language Models LLM Evaluation Training Data Attribution Language Model Self-Assessment

Ask PDF

May 14, 2026

MeMo: Memory as a Model

Large language models (LLMs) achieve strong performance across a wide range of tasks, but remain frozen after pretraining until subsequent updates. Many real-world applications require timely, domain-specific information, motivating the need for efficient mechanisms to incorporate new knowledge. In this paper, we introduce MeMo (Memory as a Model), a modular framework that encodes new knowledge into a dedicated memory model while keeping the LLM parameters unchanged. Compared to existing methods, MeMo offers several advantages: (a) it captures complex cross-document relationships, (b) it is robust to retrieval noise, (c) it avoids catastrophic forgetting in the LLM, (d) it does not require access to the LLM's weights or output logits, enabling plug-and-play integration with both open and proprietary closed-source LLMs, and (e) its retrieval cost is independent of corpus size at inference time. Our experimental results on three benchmarks, BrowseComp-Plus, NarrativeQA, and MuSiQue, show that MeMo achieves strong performance compared to existing methods across diverse settings.

Ryan Wei Heng Quek, Sanghyuk Lee, Alfred Wei Lun Leong +6

Institute of Data Science, National University of Singapore, Singapore · Integrative Sciences and Engineering Programme, NUSGS, Singapore · Agency for Science, Technology, Research (A*STAR), Singapore +6

Retrieval-Augmented Generation Memory-Augmented Language Models Knowledge Augmentation for Language Models Persistent Memory for Language Models LLM Memory

Ask PDF

Abstract

Factual Knowledge in Language Models

Explore similar work

Beyond Questions: Evaluating the Stated Knowledge of LLMs

LUMOS: Tracing Parametric Knowledge from Training Data to Behavioral Outputs in LLMs

MeMo: Memory as a Model