Aligning the Query Space: Greedy Information Projection for Language Model Data Selection

Victor Ye Dong; Kuan-Yun Lee; Jiamei Shuai; Shengfei Liu; Yi Liu; Jian Jiao

cs.LGMar 14, 2026

Aligning the Query Space: Greedy Information Projection for Language Model Data Selection

Authors: Victor Ye Dong, Kuan-Yun Lee, Jiamei Shuai, Shengfei Liu, Yi Liu, Jian Jiao

Organizations: Microsoft

Abstract

Data selection for language models is often framed as balancing example quality and diversity. We argue that both are consequences of a more fundamental principle: selected examples should preserve the downstream query space induced by instructions, task signals, or retrieval needs. We present Greedy Information Projection (GIP), a query-aligned method that requires only candidate embeddings and a score signal from LLM judgments, metadata, or intrinsic geometry. GIP greedily selects examples whose embedding span explains the largest residual component of task/query scores, yielding a fast matching-pursuit selector. A Gaussian projection view connects this update to maximizing mutual information, equivalently minimizing the residual volume of the query subspace left unexplained by selected data; this explains how quality and diversity emerge from one objective. Empirically, GIP matches or surpasses full-data fine-tuning using small instruction and reasoning subsets, while pretraining and RAG passage-selection experiments show that the same residual principle transfers beyond supervised fine-tuning.

Figures & tables

Figure 1: Geometric view of GIP . The selected embeddings $F_{S}$ define $\mathrm{span}(F_{S})$ and the projector $P_{S}$ ; greedy selection increases the query energy captured by $P_{S}Q$ while leaving smaller residual directions $(I-P_{S})Q$ .

Method	Prep.	Selection	Val.	Grad.
GIP (C)	$O(m^{2}d)$	$O(mk)$	No	No
GIP (S)	$O(m)$	$O(mkd)$	No	No
DPP (C)	$O(m^{2}d)$	$O(mk^{2})$	No	No
DPP (S)	$O(m)$	$O(mk^{2}d)$	No	No
LESS	$O(mN)$	$O(mm_{v}d)$	Yes	Yes
DiSF	$O(m)$	$\tilde{O}(mkd^{2})$	Yes	No

Table 1: Complexity and selector requirements. C/S denotes cached/streamed and

m_{v}=m_{\text{val}}

. Val. marks whether a method requires held-out target or validation data, and Grad. marks model-gradient computation.

\tilde{O}

hides polylogarithmic factors, and

l

counts tokens per DSIR candidate.

Model	Data	ModernBERT	Qwen-reasoning
Mistral-7B	20%	49.81	50.27
Mistral-7B	10%	46.63	46.25
Qwen3-32B	20%	87.57	88.02
Qwen3-32B	10%	87.34	87.72

Table 4: Embedding ablation for GIP+SC on GSM8K.

Panel of Figure 2 — Figure 2: RAG and MT-Bench ablations. Left: oracle best-over-sweep mean answer F1 grouped by retrieved-context budget; tunable methods are swept over their tradeoff parameters. Right: GIP+SC at 1% data versus Full on cleaned and non-cleaned Alpaca, judged by GPT-5.

Tokens	Selector	HS	PIQA	ARC-E	ARC-C	WG	OBQA	SciQ	Mean
0.8B	Random	0.298	0.544	0.300	0.198	0.489	0.250	0.335	0.345
0.8B	DSIR	0.290	0.545	0.296	0.218	0.494	0.262	0.352	0.351
0.8B	DiSF	0.299	0.543	0.315	0.218	0.491	0.262	0.340	0.353
0.8B	GIP	0.306	0.540	0.317	0.217	0.496	0.260	0.390	0.361
1.6B	Random	0.317	0.572	0.332	0.205	0.498	0.236	0.475	0.376
1.6B	DSIR	0.326	0.590	0.331	0.211	0.504	0.242	0.513	0.388

Table 5: Per-task results from single 411M-parameter OpenLM pretraining runs at 0.8B and 1.6B tokens. Best results are in bold ; second-best results are underlined . The OpenLM-1B follow-up and scale comparison are shown in Figure 3 ; three-seed averages are reported in Table 31 .

Figure 3: Seven-task pretraining mean for DSIR and GIP in the 411M/0.8B-token, 411M/1.6B-token, and OpenLM-1B/4.1B-token settings. The 4.1B-token runs use the same seed and optimization recipe; this setting changes both model size and token budget and therefore measures their combined scale-up.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset and license	Use in this work
Fine-tuning and evaluation
Alpaca-52k ( Taori et al., 2023 ) (CC BY-NC 4.0)	Instruction-tuning candidate pool
GSM8K ( Cobbe et al., 2021 ) (release repository: MIT)	Mathematical-reasoning selection, fine-tuning, and held-out evaluation.
MT-Bench ( Zheng et al., 2023 ) (release repository: Apache 2.0)	Multi-turn instruction-following evaluation.
BIG-Bench Hard (BBH) ( Suzgun et al., 2023 ) (release repository: MIT)	Exact-match reasoning evaluation.
AlpacaEval 2 ( Dubois et al., 2024 ) (CC BY-NC 4.0)	Length-controlled instruction-following evaluation.

Appendix

Table 7: Datasets used in our experiments, their stated licenses, and their roles. We report a dataset license when the source states one; otherwise, we identify the license of the official release repository. Documents incorporated from third-party sources retain their original terms.

Dataset	Size	GPUs	Epochs	LR	Sched.	Ctx. Len.
Alpaca-52k	52k	8	10	3e-5	Linear	2048
Alpagasus-1k	1k	4	10	3e-5	Linear	2048
CaR-1k	1k	4	10	3e-5	Linear	2048
Random-1k	1k	4	10	3e-5	Linear	2048
GIP-512	512	4	10	3e-5	Linear	2048

Appendix

Table 8: LoRA fine-tuning hyperparameters. Qwen3-32B uses the same LoRA configuration as the other Qwen3 models.

We would like to request your feedback on the performance of the AI assistant in response to the instruction and the given input displayed following, based on the following guideline.

1. Coherence. What to judge: logical flow, internal consistency, clarity.

Score anchors: 0 – Nonsensical or self-contradictory; 1 – Confusing, frequent jumps; 2 – Some lapses but understandable; 3 – Clear and orderly; 4 – Excellent narrative flow and transitions; 5 – Flawless logic, elegant structure, exceptionally smooth.

2. Correctness / Accuracy. What to judge: factual accuracy and fidelity to the prompt.

Score anchors: 0 – Main claim wrong or unsupported; 1 – Many errors or hallucinations; 2 – Minor slips or partially met requirements; 3 – Fully correct, only trivial issues; 4 – Rigorous and well-sourced; 5 – Authoritative, thoroughly sourced, withstands expert scrutiny.

3. Helpfulness. What to judge: usefulness, completeness, depth, alignment with the question.

Score anchors: 0 – Provides no help; 1 – Little usable information; 2 – Partially helpful but key gaps; 3 – Satisfies the question well; 4 – Exceeds expectations, anticipates follow-ups, adds examples; 5 – Exceptional, deep insights, meta-guidance, multiple perspectives.

Appendix

Table 9: Evaluation rubric used to obtain LLM-based scores for the GIP+MA variant.

$k$	GIP / Optimal	Random / Optimal
1	$0.958\pm 0.108$	$0.255\pm 0.304$
2	$0.911\pm 0.120$	$0.320\pm 0.258$
3	$0.877\pm 0.115$	$0.395\pm 0.246$
4	$0.874\pm 0.101$	$0.482\pm 0.214$
5	$0.870\pm 0.095$	$0.574\pm 0.225$
6	$0.889\pm 0.088$	$0.655\pm 0.211$

Appendix

Table 10: Approximation fidelity: ratio of the original (non-linearized) objective value at the method’s selection to the optimal value at each

k

. GIP is consistently near-optimal; random trails. For

k{=}10

both methods select all items.

Noise level $\sigma$	10% (747)	20% (1494)
$10^{-4}$	$95.89\pm 0.32$	$91.72\pm 0.36$
$10^{-3}$	$94.20\pm 0.66$	$87.85\pm 0.39$
$10^{-2}$	$66.32\pm 1.13$	$61.74\pm 0.29$

Appendix

Table 11: Selection IoU under embedding noise.

Data	Pool $m$	Subset	Preproc. / Sel. (s)	Peak RAM (Prep. / Sel.)
GSM8K	7k	10%	7.01 / 0.86	0.3 GB / 0.04 GB
GSM8K	7k	20%	7.01 / 0.87	0.3 GB / 0.04 GB
GSM8K	7k	50%	7.01 / 0.87	0.3 GB / 0.04 GB
Alpaca	52k	10%	214.11 / 25.12	10.8 GB / 0.3 GB
Alpaca	52k	20%	214.11 / 49.67	10.8 GB / 0.3 GB
Alpaca	52k	50%	214.11 / 123.62	10.8 GB / 0.3 GB

Appendix

Table 12: Runtime and memory profiling. Preprocessing is amortized across budgets and score variants.

Figure 4: Oracle best-over-sweep RAG answer F1 across retrieved-context budgets.

Question. Where did the punter for the Dallas Cowboys in the 1980s play college football? Gold answer. Arizona State University.
Method	Chosen text excerpts
GIP $(\lambda=0.8)$ ; prediction: Arizona State University; F1: 1.00	1. “… is a Serbian former professional American football punter in the National Football League for the Dallas Cowboys. He played college football at the University of South Dakota.” 2. “… is an American former professional football player who was a defensive back for 13 seasons with the Dallas Cowboys … He did not play college football at Utah State University …” 3. “… commentator for Cowboys games on Compass Media Networks’ America’s Team Radio Network since the 2011 season. He played college football at Arizona State University.” 4. “White was honored … at a Legends Luncheon hosted by the Arizona State University Alumni Association … The Dallas Cowboys selected him in the third round …” 5. “… developing skills to break into professional football leagues. Filipovic also works with agents that sign and negotiate NFL contracts.”
DPP $(\alpha=2)$ ; prediction: University of Mississippi; F1: 0.33	1. “… is a Serbian former professional American football punter in the National Football League for the Dallas Cowboys. He played college football at the University of South Dakota.” 2. “… is an American former professional football player who was a defensive back for 13 seasons with the Dallas Cowboys … He did not play college football at Utah State University …” 3. “… commentator for Cowboys games on Compass Media Networks’ America’s Team Radio Network since the 2011 season. He played college football at Arizona State University.” 4. “Tubbs played three varsity years at the University of Oklahoma, and the Sooners won all 31 games in that period.” 5. “James Gordon Miller … was a punter in the National Football League … Miller played college football for the University of Mississippi …”

Appendix

Table 15: HotpotQA selected text excerpts. GIP uses the query-only residual setting with

\lambda=0.8

Question. What is the place of birth of Aleksey Greig’s father? Gold answer. Inverkeithing.
Method	Chosen text excerpts
GIP $(\lambda=0.8)$ ; prediction: Inverkeithing; F1: 1.00	1. “… Greig’s grandfather Charles was an emigrant from Scotland. His father Samuil was an admiral in the Russian Imperial Navy.” 2. “Dr Rogerson … ordered him to proceed immediately to Revel … The ceremonial of the admiral’s funeral in the Tallinn Cathedral …” 3. “Vice-Admiral Samuel Greig … (30 November 1735, Inverkeithing, Fife, Scotland - 26 October 1788, Tallinn, Estonia, Russian Empire) …” 4. “Aleksey Samuilovich Greig … born into the noble Greig family, was an admiral of the Imperial Russian Navy.” 5. “Family Samuel Greig married Sarah … He was father to Alexey Greig, admiral of the Imperial Russian Navy …”
Top-k; prediction: St. Petersburg; F1: 0.00	1. “… Greig’s grandfather Charles was an emigrant from Scotland. His father Samuil was an admiral in the Russian Imperial Navy.” 2. “Dr Rogerson … ordered him to proceed immediately to Revel … The ceremonial of the admiral’s funeral in the Tallinn Cathedral …” 3. “Family Samuel Greig married Sarah … He was father to Alexey Greig, admiral of the Imperial Russian Navy …” 4. “In 1816 Greig became Commander of the Black Sea Fleet … he served as Military Governor of Sevastopol and Nikolayev …” 5. “He succeeded Paul Dacre as editor of the Daily Mail in September 2018 …”

Appendix

Table 16: 2WikiMQA selected text excerpts.

Question. Who was the first African American student at the university Robert Khayat was educated at? Gold answer. James Meredith.
Method	Chosen text excerpts
GIP $(\lambda=0.8)$ ; prediction: Robert Robinson Taylor; F1: 0.00	1. “Robert Robinson Taylor … Taylor was the first African-American student enrolled at the Massachusetts Institute of Technology (MIT) …” 2. “Robert Conrad Khayat … was the 15th Chancellor of the University of Mississippi … He attended Moss Point High School and the University of Mississippi.” 3. “Khayat later became a lawyer and taught law at the University of Mississippi School of Law. He was appointed chancellor in 1995.” 4. “Juanita Karpf wrote the book Performing Racial Uplift … Death Hackley collapsed on stage while performing in San Diego in 1921 …” 5. “An African American, she promoted racial pride through her support and promotion of music education for African-Americans.”
DPP $(\alpha=2)$ ; prediction: James Meredith; F1: 1.00	1. “Robert Robinson Taylor … Taylor was the first African-American student enrolled at the Massachusetts Institute of Technology (MIT) …” 2. “Robert Conrad Khayat … was the 15th Chancellor of the University of Mississippi … He attended Moss Point High School and the University of Mississippi.” 3. “Juanita Karpf wrote the book Performing Racial Uplift … Death Hackley collapsed on stage while performing in San Diego in 1921 …” 4. “Meredith’s admission is regarded as a pivotal moment in the history of civil rights in the United States. Many students harassed Meredith during his two semesters on campus …” 5. “… Alpha Phi Alpha fraternity. He died of a heart ailment at his home in Washington, D.C., February 28, 1998 …”

Appendix

Table 17: MuSiQue selected text excerpts.

Question. When will BC leave Boston? Gold answer. August 25.
Method	Chosen text excerpts
GIP $(\lambda=0.8)$ ; prediction: Never mentioned; F1: 0.00	1. “… one of the great under-appreciated aspects of BC sports is ‘For Boston.’ … BCeagles.com put out another player Q&A …” 2. “… BCeagles.com released the bios of the six new freshmen and two transfers for the men’s soccer team. Four of the newbies hail from New York.” 3. “Boston is ideal because the local universities could already supply many of the venues.” 4. “The closer we get to kickoff and the more I read, the more I think BC might surprise people this year.” 5. “Canada will be new there this fall … If Canada became a candidate for BC, I would have hesitation about his time at Indiana.”
xQuAD; prediction: August 25; F1: 1.00	1. “Canada will be new there this fall … If Canada became a candidate for BC, I would have hesitation about his time at Indiana.” 2. “… BCeagles.com released the bios of the six new freshmen and two transfers for the men’s soccer team. Four of the newbies hail from New York.” 3. “… one of the great under-appreciated aspects of BC sports is ‘For Boston.’ … BCeagles.com put out another player Q&A …” 4. “August 31 Barcelona - Game 3 … September 1 Barcelona - Game 4 … The team leaves Boston on August 25.” 5. “Although the Eagle is now synonymous with BC … Father McLaughlin … suggested the eagle due to its majesty, power and freedom.”

Appendix

Table 18: MultiFieldQA selected text excerpts.

Method	Sig. Wins	Sig. Losses	Neutral	Total
GIP+MA (All)	34 (37.0%)	8 (8.7%)	50 (54.3%)	92
GIP+SC (All)	25 (27.2%)	5 (5.4%)	62 (67.4%)	92
GIP+MA vs Random	6 (30.0%)	0 (0.0%)	14 (70.0%)	20
GIP+SC vs Random	4 (20.0%)	1 (5.0%)	15 (75.0%)	20

Appendix

Table 19: Summary of significance tests for the supervised fine-tuning comparisons.

Model	Method	2.5%	20%
Mistral-7B	FisherSFT	33.89 $\pm$ 1.30	46.32 $\pm$ 1.37
	GIP+MA	42.99 $\pm$ 1.36	47.46 $\pm$ 1.37
	GIP+SC	38.89 $\pm$ 1.34	49.81 $\pm$ 1.38
Qwen3-8B	FisherSFT	76.65 $\pm$ 1.16	81.27 $\pm$ 1.07
	GIP+MA	81.58 $\pm$ 1.07	83.24 $\pm$ 1.03
	GIP+SC	80.36 $\pm$ 1.09	82.26 $\pm$ 1.05

Appendix

Table 28: Matched GSM8K accuracy for the settings in which we evaluated FisherSFT. Best results within each model–budget column are in bold ; uncertainty is the binomial standard error over 1,319 test questions.

Scale	Method	HS	PIQA	ARC-E	ARC-C	WG	OBQA	SciQ	Mean $\uparrow$
1.6B	Random	0.319	0.572	0.333	0.204	0.489	0.245	0.467	0.376
1.6B	DSIR	0.321	0.579	0.328	0.203	0.489	0.255	0.479	0.379
1.6B	GIP	0.333	0.579	0.345	0.211	0.490	0.258	0.513	0.390

Appendix

Table 31: Three-seed average pretraining results at 1.6B tokens by downstream task, with each selected subset held fixed across seeds.

Question	Draft a professional email seeking your supervisor’s feedback on the ‘Quarterly Financial Report’ you prepared. Ask specifically about the data analysis, presentation style, and the clarity of conclusions drawn. Keep the email short and to the point.
CaR	Dear Supervisor, I would like to seek your feedback on the ‘Qu quarterly Financial Report’ that I recently prepared. I am particularly interested in receiving your thoughts on the following aspects: data analysis (was it accurate and comprehensive?), presentation style (clear, concise, engaging?), and conclusions drawn (did you agree?). I would greatly appreciate any feedback you can provide. Thank you. Sincerely.
GIP+MA	Dear Supervisor, I would like to seek your feedback on the Quarterly Financial Report I prepared. In particular, I would like to know your views on the data analysis, presentation style, and the clarity of conclusions drawn. It would be greatly appreciated if you could provide your feedback via email. Thank you.
GIP+SC	Dear [Supervisor’s Name], I’m writing to request your feedback on the ‘Quarterly Financial Report’ I have prepared. Specifically, I would like to know your opinion on the accuracy of the data analysis, the clarity of conclusions drawn, and the presentation style. Thank you, [Your Name].
Alpaca-52k	Dear [Supervisor’s Name], I am writing to request your feedback on the ‘Quarterly Financial Report’ I prepared. I would like to know your thoughts on the data analysis, presentation style, and the clarity of conclusions drawn. If you have any suggestions, I am eager to hear them. Sincerely, [Your Name].
Alpagasus-1k	Dear Supervisor, I would like to seek your feedback on the ‘Quital Financial Report’ that I prepared. Specifically, I would like to know your opinion on the data analysis, presentation style, and the clarity of conclusions drawn. I would appreciate any constructive feedback you may have. Thank you.

Appendix

Table 32: Example outputs in the writing category.

Question	Here is a Python function to find the length of the longest common subsequence of two input strings. Can you identify any bug in this function?
CaR	The function looks correct. However, there is a bug in the line where the dp matrix is initialized … this bug will cause the function to return incorrect results. (Incorrectly flags the initialization as buggy.)
GIP+MA	There is no bug in this function. It correctly calculates the length of the longest common subsequence.
GIP+SC	The function is correctly written and should work as intended. However, there is a bug in the code … it should be dp = [[0 for _ in range(n+1)] for _ in range(m+1)] . (Partially correct, but adds a spurious fix.)
Alpaca-52k	The function has a bug in the line where it compares the characters of the two strings … str1[i-1] should be compared with str2[j-1] in reverse order. (Incorrect; follows the misleading prompt.)

Appendix

Table 33: Example outputs in the coding category. (The function in the question is actually correct.)

Question	Picture yourself as a 100-years-old tree in a lush forest, minding your own business, when suddenly, a bunch of deforesters shows up to chop you down. How do you feel when those guys start hacking away at you?
CaR	I’m sorry, I’m not sure how to answer that question. Could you please rephrase it? (Refuses to engage with the roleplay framing.)
GIP+MA	I would feel sad and angry. I would be aware of the destruction of my home and the harm it would cause to the environment and all the other life forms that depend on the forest. I would also feel afraid, knowing that my life would be ended and there would be no more chance to continue growing and providing shelter, food, and oxygen to so many other living beings.
GIP+SC	(After two rounds of self-clarification:) In my opinion, the tree feels a great deal of pain, fear, and even grief when being chopped down by loggers.
Alpaca-52k	I would feel sad and scared, and I would try to escape.

Appendix

Table 34: Example outputs in the roleplay category.

Explore similar work

Jul 20, 2026cs.CL

PPL-Factory: Task-Aware and Budget-Aware Data Selection from Language Modeling to Reasoning

Not all training samples contribute equally to large language model fine-tuning. Selecting informative training samples can reduce the computational cost while preserving downstream performance. Many existing data selection methods rely on indirect heuristics, such as data quality, diversity or reasoning trace length. However, the effectiveness of these fixed criteria is task-dependent and difficult to generalize across diverse downstream tasks. Perplexity-based data selection provides a simple and model-aware solution to estimate the sample difficulty, but existing approaches typically score the entire training sequence and ignore the difference in learning objectives of language modeling and reasoning tasks. In this paper, we propose PPL-Factory, a simple and interpretable data selection framework that combines task-aware perplexity-based scores and data budget-aware selection criteria. Experiments on GSM8K demonstrate that PPL-Factory outperforms other state-of-the-art data selection methods using only

1\%

of the training set. With

10\%

of the data, PPL-Factory exceeds full-data fine-tuning accuracy by 0.9 on GSM8K and 4.8 on MATH. Overall, our results demonstrate that task-aware and budget-aware perplexity-based selection provides an effective and applicable approach for efficient fine-tuning.

Hang Zhang, Warren J. Gross

Department of Electrical and Computer Engineering, McGill University Montreal, QC, Canada

Large Language Model Fine-Tuning Language Modeling Training Data

Ask PDF

Jun 29, 2026cs.LG

Online Data Selection for Instruction Tuning via Gaussian Processes

With Large Language Model (LLM) pre-training and fine-tuning shifting its focus from data volume to data quality, quality data selection has emerged as a critical research topic. Existing online data selection methods for LLM training are typically "batch-constrained", limiting optimization to local utility within random batches. To overcome this, we propose GAIA (Global Adaptive Instruction tuning via GAussian processes), a framework that formulates data valuation as a global estimation process. GAIA employs Gaussian Process regression to model continuous utility manifolds across the semantic space, utilizing an adaptive strategy fusion mechanism to dynamically prioritize high-utility samples. By casting the strategy-posterior update as an instance of the classical fixed-share Hedge framework for tracking the best expert, we inherit a dynamic-regret guarantee that characterizes GAIA's robustness under non-stationary quality scores during training. Empirical evaluations on three datasets demonstrate that GAIA significantly outperforms state-of-the-art baselines like \greats, establishing our method as a scalable and robust solution for efficient instruction tuning.

Jun Wang, Quoc Phong Nguyen, Julien Monteil +1

Amazon

Instruction Tuning Gaussian Process Value

Ask PDF

Sep 17, 2026cs.CL

Form Over Content In Gradient-Based Data Attribution Methods

Data attribution methods using gradient similarity are widely used to analyze and select training data for large language models, but what gradient similarity actually measures is debated. Some interpret it as identifying task-relevant skills, while other work reports that surface form is the main factor. We resolve this debate for supervised fine-tuning examples by varying task and answer format independently. Specifically, we render benchmarks in different answer formats, such that datasets can share a task without a format or a format without a task. We find that gradient alignment follows the answer format, as benchmark pairs sharing an answer format align strongly (disattenuated cosine near 0.4), while same benchmarks rendered with different answer format classes show no alignment (near 0.0). We demonstrate that this ordering holds from the earliest pretraining checkpoints through post-training, and across model scales and families. We then analyze the released selections of LESS, a gradient-based data selection method for instruction tuning, and find that each target's selections over-represent the target's own answer format. Hence, we demonstrate that gradient-based attribution methods track format similarity more than task semantics, meaning that such methods, as well as the semantic interpretation of the gradient, should be tested on data where answer format and task vary independently for greater robustness and reliability.

Sunwoo Kim, Seokwon Jung, Sohyung Kim +2

KAIST

Value Alignment Benchmark Saturation

Ask PDF