Listening to the Wise Few: Query-Key Alignment Unlocks Latent Correct Answers in Large Language Models

Eduard Tulchinskii; Kristian Kuznetsov; Laida Kushnareva; Anastasia Voznyuk; Andrei Andriiainen; Irina Piontkovskaya; Evgeny Burnaev; Serguei Barannikov

cs.CLOct 3, 2024

Listening to the Wise Few: Query-Key Alignment Unlocks Latent Correct Answers in Large Language Models

Authors: Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva, Anastasia Voznyuk, Andrei Andriiainen, Irina Piontkovskaya, Evgeny Burnaev, Serguei Barannikov

Organizations: Skolkovo Institute of Science and Technology · AI Foundation and Algorithm Lab · Moscow Institute of Physics and Technology · Artificial Intelligence Research Institute (AIRI) · CNRS, Universit´e Paris Cit´e

Abstract

Large language models (LLMs) routinely fail to output the correct option in multiple-choice question answering (MCQA) while encoding the answer internally. We expose this latent knowledge via the Query--Key (QK) score, defined for an attention head as the inner product between the last-token query and the key at the end-of-line token following option $i$ , evaluated before rotary positional embedding is applied. Its argmax identifies a universal class of select-and-copy heads in middle layers that perform option selection through semantic query--key alignment, mechanistically distinct from induction and copy-suppression heads (Olsson et al., 2022): they are invariant to label symbols, and solve a synthetic task with zero surface overlap---properties no positional-copy account explains and that critically require stripping RoPE. Across 24 models from 1.5B to 72B parameters (LLaMA-2/3/3.1/3.3, Qwen-2.5, Gemma, Phi-3.5, DeepSeek-R1-Distill), a single head's QK-score exceeds the model's own zero-shot accuracy by up to $+27.4$ pp on HellaSwag and $+49.8$ pp on HaluDialogue; causal zero-ablation collapses MCQA accuracy to near-random. To remove any dependence on labeled validation data, we introduce an unsupervised HeadScore that ranks heads from unlabeled inputs and recovers the supervised top- $k$ heads on every tested model. Against four positional-debiasing baselines (e.g., PriDe, Wiegrefe, Wang), QK-score is complementary by construction: debiasing re-weights output logits, whereas QK-score reads the model's selection from a middle-layer head before decoding. We release a one-line drop-in HeadScore script and per-model head indices, making every result one-command reproducible across all 24 models and four benchmarks.

Figures & tables

Figure 1 : Our method calculates the Query-Key score between the end-of-line token of an answer option and the last token of the prompt for the designated head, from which we derive the answer.

Panel of Figure 2 — Figure 2 : (Left) Scheme for option-representative token types. (Right) Performance of QK-score and Attention-score (App. E ) for different option-representative tokens on LLaMA3.1-8B base.

Figure 3 : Comparison of different methods for LLaMA3.1-8B. Reported metrics are Accuracy and Permutation Accuracy.

Language	Max Acc	Min Acc
English	1.000	0.995
Italian	1.000	0.995
French	1.000	0.985
Russian	1.000	0.990

Table 1 : Top-20 heads performance on Simple Synthetic Dataset for LLaMA 3.1-8B

		LLaMA…				LLaMA… (chat, instruct)
Method		3-8B	3-70B	3.1-8B	3.1-70B	3-8B	3-70B	3.1-8B	3.1-70B
		MMLU
Baseline	Acc	60.3	75.3	59.1	72.9	60.5	78.2	60.4	73.8
Baseline	PA	50.4	68.8	48.6	65.7	47.7	70.1	50.1	67.8
QK-score	Acc	61.0	74.5	62.5	75.9	63.0	77.9	64.9	79.4
QK-score	PA	51.5	66.0	50.7	68.5	49.3	67.9	55.2	74.4

Table 2 : Comparison of different base models in zero-shot setup on various Q&A datasets. Reported metrics are Accuracy (Acc) and Permutation Accuracy (PA). Best results are highlighted in bold . Detailed results for LLaMA3.1-8B are presented in Table 5 .

Figure 6 : QK-scores after softmax and attentions for the last token on the 0-shot MMLU example on (17, 26) head of LLaMA3.1-8B. The task is “ Question: What singer appeared in the 1992 baseball film ’A League of Their Own’ $\backslash$ nOptions: $\backslash$ nA. Brandy. $\backslash$ nB. Madonna. $\backslash$ nC. Garth Brooks. $\backslash$ nD. Whitney Houston. $\backslash$ nE. I don’t know. $\backslash$ nF. None of the above. $\backslash$ nAnswer: ” with correct option ’B’.

Figure 7 : Zero-ablation of select-and-copy and random heads for LLaMA3.1-8B. The heads are selected by above-random performance of QK-score , and the random ablation with the same amount of random heads as selected heads is added for comparison. Random heads were ablated with 5 different launches, and we report mean and standard deviation of accuracy.

Appendix figures & tables54 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 8 — Figure 8 : Correlation between heads QK-scoring accuracy on questions with 4 (‘A’-‘D’) and 6 (‘A’-‘F’) answer options. Solid red lines mark the accuracy level of $0.25$ , dashed red line – $0.167$ (6 options random choice accuracy). Model LLaMA3.1-8B (base).

Model	Method		MMLU	Cosmos QA	Hellaswag QA	Halu Dialogue
	Baseline	Acc	59.5 $\pm$ 0.6	74.3 $\pm$ 1.2	58.1 $\pm$ 2.2	35.9 $\pm$ 2.1
Qwen2.5	Baseline	PA	49.3 $\pm$ 0.4	67.8 $\pm$ 1.6	48.6 $\pm$ 2.9	25.7 $\pm$ 1.7
-1.5B	QK-score	Acc	57.0 $\pm$ 1.3	72.1 $\pm$ 0.9	58.7 $\pm$ 1.0	38.3 $\pm$ 0.9
	QK-score	PA	45.9 $\pm$ 1.1	64.2 $\pm$ 0.8	48.6 $\pm$ 0.3	27.5 $\pm$ 0.8
	Baseline	Acc	29.2 $\pm$ 1.5	37.3 $\pm$ 1.2	29.4 $\pm$ 0.2	26.5 $\pm$ 1.1
LLaMA2	Baseline	PA	10.6 $\pm$ 0.5	17.3 $\pm$ 0.9	9.1 $\pm$ 0.1	7.2 $\pm$ 1.2

Appendix

Table 3 : Performance of Baseline and QK-score methods on prompts with explicit instructions. 0-shot setup.

Model	Method		MMLU	Cosmos QA	Hellaswag QA	Halu Dialogue
	Baseline	Acc	56.9 $\pm$ 2.9	71.4 $\pm$ 2.7	56.7 $\pm$ 3.7	38.4 $\pm$ 2.5
Qwen2.5	Baseline	PA	45.9 $\pm$ 3.2	64.2 $\pm$ 3.2	47.3 $\pm$ 3.8	27.2 $\pm$ 2.5
-1.5B	QK-score	Acc	55.3 $\pm$ 2.0	70.6 $\pm$ 1.3	56.1 $\pm$ 1.8	38.2 $\pm$ 2.5
	QK-score	PA	43.3 $\pm$ 1.9	61.3 $\pm$ 1.4	45.8 $\pm$ 2.9	23.6 $\pm$ 5.7
	Baseline	Acc	$29.2\pm$ 1.8	29.6 $\pm$ 5.0	27.3 $\pm$ 1.3	23.9 $\pm$ 1.2
LLaMA2	Baseline	PA	10.0 $\pm$ 1.5	11.5 $\pm$ 4.2	8.1 $\pm$ 0.9	5.7 $\pm$ 0.8

Appendix

Table 4 : Performance on prompts asking to emulate an emotion (or roleplay a character) and then answer the question. 0-shot setup

	…-shot prompting
Method		0	1	2	3	4	5
	MMLU
Baseline	Acc	59.0	63.3	63.3	62.6	62.7	63.7
Baseline	PA	48.4	52.6	52.9	52.1	52.0	53.3
PRIDE	Acc	62.4	64.0	63.3	63.0	63.4	64.1
PRIDE	PA	52.9	53.8	54.2	53.7	53.5	54.1

Appendix

Table 5 : Comparison of different methods for LLaMA3.1-8B (base) on various Q&A datasets. Reported metrics are Accuracy (Acc) and Permutation Accuracy (PA). The best results are highlighted in bold .

	…-shot prompting
Method		0	1	2	3	4	5
	MMLU
Baseline	Acc	26.7	39.1	43.1	43.7	44.1	43.8
Baseline	PA	8.9	21.3	26.2	27.4	28.5	28.4
PRIDE	Acc	15.5	36.9	39.8	40.8	41.5	42.7
PRIDE	PA	5.7	20.8	24.2	24.6	25.6	28.9

Appendix

Table 6 : Comparison of different methods for LLaMA2-7B (base) on various Q&A datasets. Reported metrics are Accuracy (Acc) and Permutation Accuracy (PA). The best results are highlighted in bold .

Dataset	Best (Layer, Head)
MMLU	(16, 19) , (17, 24) , (16, 26), (17, 26)
HaluDialogue	(14, 5), (14, 21), (14, 2), (17, 24)
HellaSwag	(14, 5) (17, 24) , (17, 29), (16, 19)
CosmosQA	(17, 24) , (16,19) , (16, 26), (17,29)

Appendix

Table 7 : Common top 1% heads across all

n

-shots based on accuracy for different datasets for LLaMA-3.1-8B

Dataset	Best (Layer, Head)
MMLU	(14, 24) , (15, 4), (17, 0), (14, 20) ,
HaluDialogue	(14, 29), (14, 24) , (14, 26)
HellaSwag	(15, 5), (15, 4), (18, 10), (14, 20)
CosmosQA	(14, 24) , (15, 5), (15, 4), (15, 23), (14, 20)

Appendix

Table 8 : Common top 1% heads across all

n

-shots based on accuracy for different datasets for LLaMA-2-7B

Panel of Figure 9 — Figure 9 : Stable heads for QK-score for 0-shot setup across all tasks for LLaMA-3.1-8B and Qwen2.5-1.5B (Base and Instruct). “ $k$ -th Minimum of Percentiles” means that the head is better than $k$ share of all heads for all tasks.

Panel of Figure 10 — Figure 10 : Stable heads for QK-score in (a) LLaMA2-7B and (b) LLaMA3-8B for 0-shot setup across all tasks. “ $k$ -th Minimum of Percentiles” means that the head is better than $k$ share of all heads for all tasks.

Figure 11 : Accuracy of the best performing heads and several of the most robust heads – (14, 24), (14, 20) in LLaMA-2-7B

Figure 12 : Zero-ablation of select-and-copy and random heads for LLaMA3.1-8B-Instruct

Figure 13 : Zero-ablation of select-and-copy and random heads for DeepSeek-R1-Distill-Qwen-7B

Panel of Figure 14 — Figure 14 : The results for various numbers of options in the Simple Synthetic Dataset (SSD) in a zero-shot setting are shown for LLaMA2-7B (left), LLaMA3-8B (center) and LLaMA3.1-8B (right). Different line colors represent the QK dot products from different heads. “Square” markers indicate heads that perform well across real datasets, while “round” markers represent heads that perform well on the synthetic dataset. Interestingly, more heads from newer versions of the LLaMA model show stability under increasing the number of options.

Panel of Figure 15 — Figure 15 : Performance of the QK-score from the best heads of the LLaMA2-7B for different option symbols, with “uncertainty” options (i.e. “I don’t know” and “None of the above”) presented (upper figure) and not presented (lower figure)

Panel of Figure 16 — Figure 16 : Upper row: performance of the QK-score from the best heads of the LLaMA3-8B for different option symbols, with “uncertainty” options (i.e. “I don’t know” and “None of the above”) presented (upper figure) and not presented (lower figure)

Panel of Figure 19 — Figure 19 : Performance of QK-score across different heads of Qwen 2.5-1.5B-base (upper row) and Qwen 2.5-1.5B-Instruct (lower row) on a multi-language SSD

Panel of Figure 20 — Figure 20 : Left: average top heads scores of LLaMA models across real datasets (first twenty). Right: Top head scores on the Simple Synthetic Dataset (first ten). Dark blue marks the best heads, stable across all real datasets (i.e. heads from top-right corner of the Figures 9 and 10 ). Note that for calculating this score, we did not use the dataset labels.

Panel of Figure 21 — Figure 21 : Left: average top heads scores for Qwen-2.5-1.5B-base across real datasets (first ten). Right: the same for Qwen-2.5-1.5B-instruct. Dark blue marks those heads that are the best across all four real datasets (see Figure 9 ).

Panel of Figure 22 — Figure 22 : Distribution of predictions across options for different methods on MMLU 0-shot and 5-shot setup. $(l,h)$ depicts the distribution for $S^{(l,h)}_{QK}$

Figure 23 : Comparison of different methods for LLaMA2-13B (base) on various Q&A datasets.

Figure 24 : Comparison of different methods for LLaMA2-70B (base) on various Q&A datasets.

Figure 25 : Comparison of different methods for LLaMA3-8B (base) on various Q&A datasets.

Figure 26 : Comparison of different methods for LLaMA3-70B (base) on various Q&A datasets.

Figure 27 : Comparison of different methods for LLaMA3.1-70B (base) on various Q&A datasets.

Figure 28 : Comparison of different methods for LLaMA-30B (base) on various Q&A datasets.

Figure 29 : Comparison of different methods for LLaMA-65B (base) on various Q&A datasets.

Figure 30 : Comparison of different methods for LLaMA2-13B-chat on various Q&A datasets.

Figure 31 : Comparison of different methods for LLaMA2-70B-chat on various Q&A datasets.

Figure 32 : Comparison of different methods for LLaMA3-8B-instruct on various Q&A datasets.

Figure 33 : Comparison of different methods for LLaMA3.1-8B-instruct on various Q&A datasets.

Figure 34 : Comparison of different methods for LLaMA3-70B-instruct on various Q&A datasets.

Figure 35 : Comparison of different methods for LLaMA3.1-70B-instruct on various Q&A datasets.

Figure 36 : Comparison of different methods for LLaMA3.3-70B-instruct on various Q&A datasets.

Figure 37 : Comparison of different methods for DeepSeek-R1 distilled on Qwen2.5-7B on various Q&A datasets.

Figure 38 : Comparison of different methods for DeepSeek-R1 distilled on Qwen2.5-14B on various Q&A datasets.

Figure 39 : Comparison of different methods for Dolly V2-3B on various Q&A datasets.

Figure 40 : Comparison of different methods for Gemma-2B on various Q&A datasets.

Figure 41 : Comparison of different methods for Phi-3.5-mini (instruct tuned) on various Q&A datasets.

Figure 42 : Comparison of different methods for Qwen-1.5B on various Q&A datasets.

Figure 43 : Comparison of different methods for Qwen-1.5B-Instruct on various Q&A datasets.

Figure 44 : Comparison of different methods for Qwen-7B on various Q&A datasets.

Figure 45 : Comparison of different methods for Qwen-7B-Instruct on various Q&A datasets.

Figure 46 : Comparison of different methods for Qwen-14B on various Q&A datasets.

Figure 47 : Comparison of different methods for Qwen-14B-Instruct on various Q&A datasets.

Figure 48 : Comparison of different methods for Qwen-32B on various Q&A datasets.

Figure 49 : Comparison of different methods for Qwen-32B-Instruct on various Q&A datasets.

Figure 50 : Comparison of different methods for Qwen-72B on various Q&A datasets.

Figure 51 : Comparison of different methods for Qwen-72B-Instruct on various Q&A datasets.

		MMLU		CosmosQA		HaluDialogue		HellaSwag
		SFT	LLaMA	SFT	LLaMA	SFT	LLaMA	SFT	LLaMA
0-shot	Baseline	0.493	0.267	0.863	0.311	0.923	0.211	0.352	0.265
0-shot	QK	0.488	0.336	0.840	0.414	0.905	0.371	0.393	0.330
1-shot	Baseline	0.476	0.391	0.836	0.393	0.474	0.309	0.716	0.288
1-shot	QK	0.472	0.407	0.810	0.5	0.858	0.366	0.800	0.371
2-shot	Baseline	0.477	0.431	0.643	0.591	0.823	0.342	0.795	0.306

Appendix

Table 11 : Accuracy on supervised fine-tuned LLaMA2-7B on the same dataset the model was fine-tuned

	MMLU		CosmosQA		HaluDialogue		HellaSwag
	Cloze	QK	Cloze	QK	Cloze	QK	Cloze	QK
0-shot	0.38	0.35	0.49	0.46	0.42	0.40	0.52	0.38
1-shot	0.40	0.39	0.51	0.50	0.45	0.42	0.52	0.35
2-shot	0.39	0.40	0.48	0.51	0.46	0.42	0.53	0.38
3-shot	0.39	0.40	0.48	0.57	0.45	0.37	0.53	0.43
4-shot	0.39	0.39	0.53	0.54	0.46	0.39	0.53	0.43

Appendix

Table 12 : Comparison of cloze prompting and our method with QK-Score. All experiments were ran on 4-optioned examples.

Figure 52 : Attention maps of (14, 24), (14, 20) and (14,4) pairs (Head, Layer) for 0-shot setting for MMLU example: Question: What singer appeared in the 1992 baseball film ’A League of Their Own’? $\backslash$ nOptions: $\backslash$ nA. Brandy. $\backslash$ nB. Madonna. $\backslash$ nC. Garth Brooks. $\backslash$ nD. Whitney Houston. $\backslash$ nE. I don’t know. $\backslash$ nF. None of the above. $\backslash$ nAnswer: . Second plot for each pair corresponds to the same, but scaled to the end-of-text-sequence attention map. Values in annotated cells are corresponding QK-score values. End of each option is denoted with $\backslash$ n symbols. 33th token is the end of A option, 38th token is the end of B option, 46th token - the end of C option, 53th token - the end of D option, 62th token - the end of E option, 70th token - the end of F option. The answer from QK-score of (14, 24) and (14, 4) is B, of (14, 20) is D. The correct answer for this example is B.

Dataset	both $\checkmark$	QK $\checkmark$ / baseline $\times$	QK $\times$ / baseline $\checkmark$	both $\times$
MMLU	61.09	7.99	6.07	24.85
CosmosQA	79.26	8.64	2.64	9.46
HellaSwag	76.69	5.91	5.26	12.14
HaluDialogue	48.06	15.07	6.44	30.43

Appendix

Table 13 : Full correctness table for DeepSeek-R1-Distill-Qwen-7B model. 0-shot, evaluation, QK-scores drawned from single validation-selected head.

\checkmark

means that method yields correct answer,

\times

- incorrect.

Explore similar work

Check The Scoreboard: An Analysis of Scoring Schemes on Multiple-Choice Evaluation
Aug 30, 2026Nishant Balepur, Paiheng Xu, Wei Ai +3Multiple-Choice Questions Scoring
University of Maryland · New York University · Nanyang Technological University
ReasonLab: A Controlled and Auditable Evaluation of Prompting Techniques for Multiple-Choice QA
May 7, 2026Inder Preet, Shuxin Lin, Dhaval PatelMultiple-Choice Questions Large Reasoning Models
IBM, Dublin · IBM, New York
Robustness assessment of large audio language models in multiple-choice evaluation
Oct 6, 2025Fernando López, Santosh Kesiraju, Jordi LuqueLarge Audio Language Models Text-To-Audio
Scientific Research, Telef´onica Innovaci´on Digital, Spain · Brno University of Technology, Czech Republic

Aug 30, 2026, cs.CLJ/K move · Enter open · S save

Check The Scoreboard: An Analysis of Scoring Schemes on Multiple-Choice Evaluation

Nishant Balepur, Paiheng Xu, Wei Ai, Eunsol Choi+2

University of Maryland · New York University · Nanyang Technological University

Multiple-Choice Questions Scoring Scores Large Language Model Benchmarks Responses Natural Language Processing

Open paper AskPDF arXiv

Multiple-choice question answering (MCQA) benchmarks in NLP use number-right scoring (accuracy), but in educational testing, the scoring scheme, the combination of the response mode models follow and the rule for grading responses, is a key design choice that dictates which abilities to reward. We examine how alternatives to number right change what MCQA measures with six education-inspired schemes that assess abilities beyond accuracy: distractor elimination, abstention, confidence calibration, and self-correction. On LLM benchmarks, these schemes: 1) shift rankings of 31 LLMs beyond rephrased number right prompts; 2) better predict the LLMs users prefer in LLM Arena; and 3) reveal distinct model capabilities, like that GPT-5 rarely abstains and readily self-corrects, while weaker open-weight models often abstain and hesitate to eliminate choices. Given the benefits of alternative scoring schemes, we discuss ways to extend them to tasks beyond MCQA.

Abstract

Explore similar work

Check The Scoreboard: An Analysis of Scoring Schemes on Multiple-Choice Evaluation

ReasonLab: A Controlled and Auditable Evaluation of Prompting Techniques for Multiple-Choice QA

Robustness assessment of large audio language models in multiple-choice evaluation