Neuron-Aware Data Selection In Instruction Tuning For Large Language Models
Authors: Xin Chen, Junchao Wu, Shu Yang, Runzhe Zhan, Zeyu Wu, Min Yang, Shujian Huang, Lidia S. Chao, Derek F. Wong
Organizations: NLP2CT Lab, Department of Computer and Information Science, University of Macau · Artificial Intelligence Research Institute, Shenzhen University of Advanced Technology · National Key Laboratory for Novel Software Technology, Nanjing University · Provable Responsible AI and Data Analytics Lab, KAUST · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences
Instruction Tuning (IT) has been proven to be an effective approach to unlock the powerful capabilities of large language models (LLMs). Recent studies indicate that excessive IT data can degrade LLMs performance, while carefully selecting a small subset of high-quality IT data can significantly enhance their capabilities. Therefore, identifying the most efficient subset data from the IT dataset to effectively develop either specific or general abilities in LLMs has become a critical challenge. To address this, we propose a novel and efficient framework called NAIT. NAIT evaluates the impact of IT data on LLMs performance by analyzing the similarity of neuron activation patterns between the IT dataset and the target domain capability. Specifically, NAIT captures neuron activation patterns from in-domain datasets of target domain capabilities to construct reusable and transferable neuron activation features. It then evaluates and selects optimal samples based on the similarity between candidate samples and the expected activation features of the target capabilities. Experimental results show that training on the 10% Alpaca-GPT4 IT data subset selected by NAIT consistently outperforms methods that rely on external advanced models or uncertainty-based features across various tasks. Our findings also reveal the transferability of neuron activation features across different capabilities of LLMs. In particular, IT data with more logical reasoning and programmatic features possesses strong general transferability, enabling models to develop stronger capabilities across multiple tasks, while a stable core subset of data is sufficient to consistently activate fundamental model capabilities and universally improve performance across diverse tasks.
Figures & tables
Figure 1: Overall framework of Nait . First, we capture the neuron activation of the LLM using in-domain data that we want the model to learn. Next, we construct an activation feature through a dimensionality reduction method. Finally, we evaluate the feature alignment score between this activation feature and the model’s activation on each candidate dataset to guide data selection.
Method
Feature Source
Externally-Independent
Gradient-Free
Targeted Ability
Cost-Effective
Interpretability
LIMA ( Zhou et al., 2023 )
Human
✔
✔
✗
✗
✗
Instruction Mining ( Cao et al., 2023 )
PPL & Reward scores & …
✔
✔
✗
✗
◯
AlpaGasus ( Chen et al., 2024 )
ChatGPT Score
✗
✔
✗
✗
✗
SelectIT ( Liu et al., 2024b )
Multi-granularity Uncertainty
✔
✔
✗
✗
◯
LESS ( Xia et al., 2024 )
Gradient-based
✔
✗
✔
✗
◯
Nait (Ours)
Neuron Activation
✔
✔
✔
✔
✔
Table 1: Comparison of our method and existing methods. ✔ indicates the presence of a feature, ✗ indicates its absence, and ◯ denotes partial support.
System ID ↓
Method ↓
Factual Knowledge
Mathematical Reasoning
Coding Ability
Multilingual Understanding
General Reasoning
Test →
MMLU
MMLU-Pro
GSM
SVAMP
H-Eval
MBPP
TydiQA
XQuAD
BBH
AVG
Δ ( ↑ )
Full Fine-tuning
01
Alpaca-GPT4 ( Peng et al., 2023 )
46.87
21.89
14.63
39.00
27.87
51.58
39.48
42.99
39.94
36.03
-
02
LIMA ( Zhou et al., 2023 )
45.20
23.04
15.76
37.67
27.75
46.56
44.92
44.72
39.91
36.17
+0.39%
03
01 + AlpaGasus ( Chen et al., 2024 )
43.21
21.96
13.34
36.67
23.94
46.08
44.70
46.84
39.91
35.18
-2.34%
04
01 + Q2Q ( Li et al., 2024 )
46.73
21.50
14.50
35.00
25.19
44.97
44.41
48.44
40.34
35.68
-0.98%
Table 2: Performance comparison of baslines using 10% of the IT data. This table shows the results of various tasks across multiple baselines. Nait (e.g., Nait (MMLU)) refers to the process where in-domain dataset to guide the IT data selection. Random refers to a randomly sampled 10% subset of the IT data. The Bold and Underline represent the best and second performance respectively in each column. Δ ( ↑ ) indicates the performance improvement relative to the ID 01.
Figure 2: Performance at different proportions of the IT dataset. Task performance in Nait (the in-domain dataset), e.g., MMLU in Nait , refers to using in-domain data to guide IT data selection and to assess task outcomes. The comprehensive results correspond to System 12.
Figure 3: Comparison of Nait with random data selection across different in-domain data scales at 30% IT dataset. Task performance in Nait (in-domain dataset), such as MMLU in Nait (MMLU), refers to using the in-domain dataset to guide IT data selection and evaluate task performance. The dashed line indicates the 30% of the IT dataset random selection baseline.
Intensity
MMLU
BBH
H-Eval
TydiQA
H-Eval
Overall
AVG
Δ ( ↑ )
Random
47.14
39.21
14.13
44.16
25.55
34.04
-
High
46.83
40.02
16.53
46.09
26.44
35.18
+3.35%
Low
46.30
28.15
10.61
36.12
20.15
28.27
-17.54%
Table 3: Performance comparison across datasets at 10% IT dataset under different neuron activation levels.
Method
MMLU
BBH
GSM
TydiQA
H-Eval
Overall
AVG
Δ ( ↑ )
LLama-2-13b
Alpaca
53.90
45.00
20.85
44.13
34.84
39.74
-
+Random
52.70
47.59
22.06
44.66
36.79
40.76
+2.57%
+ Nait
54.10
47.04
24.11
48.89
38.53
42.53
+7.02%
Mistral-7b
Table 4: Effectiveness of Nait Across Models. Alpaca and Random are trained on the full dataset and a random of 10% subset, while Nait selects a 10% subset.
Figure 4: Comparison of Nait across different IT dataset. Task performance in Nait refers to using the all in-domain dataset to guide IT data selection and evaluate task performance.
Method
Externally-Independent
Time
Cost
AlpaGasus ( Chen et al., 2024 )
✗
19.07h
$178.02
Q2Q ( Li et al., 2024 )
✗
3.52h
$4.05
LESS ( Liu et al., 2024b )
✓
9.86h
$11.33
SelectIT ( Xia et al., 2024 )
✓
23.20h
$26.68
Nait
✓
1.32h
$1.52
Table 5: Efficiency Comparison of Different Methods on NVIDIA A800 80GB with Batch Size set to 8 . API costs follow the official OpenAI pricing (2.00/millioninputtokens,8/million output tokens for GPT-4.1); GPU costs are estimated based on the Google Cloud pricing ($1.15 per GPU hour for NVIDIA A800 80GB).
Figure 5: Transferability of neural activation features across capabilities. Each column represents the capability to which the neural activation feature is applied. Icons with an outline border indicate the capability’s performance on its own task, which serves as the baseline reference.
Figure 6: Comparison of the direction of activation feature extracted by Nait across different capabilities.
Figure 7: Distribution of 10% IT dataset selected by Nait . The upper bar chart quantifies the intersection sizes among different IT data subsets, while the connected dots below identify the specific capability combinations corresponding to each intersection.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Function
Before Instruction Tuning
After Instruction Tuning
Enhancing instruction-following ability
Limited to text completion
Capable of following instructions
Activating latent abilities
Relies on knowledge from pertaining
Activates abilities not evident in pretraining, such as logical reasoning or task decomposition
Enhancing downstream task performance
General knowledge capabilities
Improved downstream task capabilities
Appendix
Table 6: Comparison of model functions before and after instruction tuning.
Targeted Ability →
Factual Knowledge
Mathematical Reasoning
Coding Ability
Multilingual Understanding
General Reasoning
DownStream Task
MMLU
MMLU-Pro
GSM
SVAMP
HumanEval
BigCodeBench
MBPP
TydiQA
XQuAD
BBH
In-domain Dataset
1,531
-
7,473
-
-
1,140
-
49,400
-
4,556
Alpaca-GPT4 IT Dataset
52,002
Evol-Instruct IT Dataset
70,000
Oral-GPT4 IT Dataset
100,000
Test Dataset
14042
2800
1319
900
164
-
974
900
1,200
1080
Appendix
Table 7: Statistics of the Number of in-domain, IT, and Test datasets Used in Experiment.
Parameter Key
Value
Main Experiment
Ablation Experiment
Learning rate
2.0e-5
Cutoff length
2048
LR scheduler
cosine
bf16
True
Warmup ratio
0.03
Appendix
Table 8: Hyperparameter settings for supervised fine-tuning.
Method ↓
Factual Knowledge
Multilingual Understanding
General Reasoning
Test →
MMLU
MMLU-Pro
TydiQA
XQuAD
BBH
AVG
+MMLU
Embedding
46.29
21.07
42.97
44.73
36.67
38.35
Representation-based
45.57
23.0
46.78
46.85
38.15
40.07
LESS
48.12
24.50
43.42
45.51
38.15
39.94
Nait
47.81
23.61
47.16
49.47
38.52
41.31
Appendix
Table 9: Performance comparison of baselines of activating targeted ability using 10% of the IT data. (e.g., +MMLU refers to the process where an in-domain dataset to guide the IT data selection.) The Bold represents the best performance respectively in each column.
Method
MMLU
BBH
GSM
TydiQA
CodeX
Overall
AVG
Δ ( ↑ )
Alpaca-GPT4
46.87
39.94
14.63
39.48
27.87
34.16
-
+ Hard (PPL)
45.52
22.22
8.72
27.87
20.56
24.98
-26.87%
+ Easy (PPL)
46.78
38.15
15.39
33.12
28.93
32.47
-5.53%
+ Hard (Length)
46.10
36.94
16.00
34.32
30.89
32.85
-3.83%
+ Easy (Length)
44.15
10.28
6.37
28.07
17.00
21.17
-38.03%
Appendix
Table 10: Performance comparison of different selection strategies using 10% of the IT data. The Bold represents the best performance respectively in each column. Δ ( ↑ ) indicates the performance improvement relative to the ID 01 baseline.
Method
En ⇒ De
De ⇒ En
Zh ⇒ En
En ⇒ Zh
Overall
COMET
BLEU
COMET
BLEU
COMET
BLEU
COMET
BLEU
COMET
BLEU
+ Full
84.54
28.24
83.13
29.06
79.80
22.89
84.15
34.58
82.91
28.69
+ Rand.
82.44
28.29
81.12
27.20
78.53
21.11
82.12
33.91
81.05
27.63
+ Nait
85.14
29.02
82.50
28.70
80.22
23.11
83.89
35.92
82.94
29.19
Appendix
Table 11: Overall results on machine translation with LLMs using the ALMA-7b. Nait indicates selecting the top 30% of data by Nait , Rand. refers to randomly selecting 30% of data, and Full uses the full dataset.
Targeted ability activated by TydiQA
Selected by Embedding
Selected by LESS
Selected by NAIT
Instruction: Translate the phrase ”Yechh” into Hindi. Output: The phrase “Yechh” is an expression of disgust or dislike and is not a word with a direct translation into Hindi. However, a similar expression of disgust in Hindi would be “.phahaH” or “chii” (Chhee) or “caDham” (chidam) or “are baapa re” (arey baap re).
Instruction: Construct a mathematical equation for the following problem: At a party, there are 3 types of food. Each type of food is served three times each. Guests can choose only 1 of each food serving. How many different combinations are there for the guests? Output: number of combinations = 3 x 3 x 3 = 27
Instruction: Create two authentic Thai dishes. Output: Great! Here are the two Thai dishes! 1. Thai Green Curry Chicken Ingredients: - 1 1/2 lbs boneless, skinless chicken breasts - 1 can (13/14-ounce) coconut milk - 1/4 cup green curry paste […]
Targeted ability activated by BBH
Selected by Embedding
Selected by LESS
Selected by NAIT
Instruction: Given a document, find the most frequently used words. Input: John went to the store to buy some candy. Output: Based on the given sentence, the most frequently used words are: - ”to” - which appears 2 times in the sentence. - ”the” - which appears 1 time in the sentence. - ”John” - which appears 1 time in the sentence. - ”went” - which appears 1 time in the sentence. - ”store” - which appears 1 time in the sentence. - ”buy” - which appears 1 time in the sentence. - ”some” - which appears 1 time in the sentence. - ”candy” - which appears 1 time in the sentence.
Instruction: Convert the following document into a list of five bullet points. Input: Sports play a key role in the lives of many people. It can teach young people about leadership and teamwork, as well as important values like perseverance and camaraderie. Sporting events bring communities together and also provide an opportunity for people to come and support their favourite teams or players. Output: - Sports play a key role in the lives of many people - Sports can teach young people about leadership and teamwork, as well as important values like perseverance and camaraderie. - Sporting events bring communities together. - Sporting events provide an opportunity for people to come and support their favourite teams or players. - Sports promote important values and bring people together.
Instruction: Design an algorithm to solve the Eight Queens puzzle Output: The Eight Queens puzzle is a classic problem where the goal is to place eight queens on an 8x8 chessboard in such a way that no two queens threaten each other. This means that no two queens can be placed in the same row, column, or diagonal. One solution to this problem is to use backtracking. Here is an algorithm to solve the Eight Queens puzzle using backtracking: 1. Start with an empty 8x8 chessboard and add the first queen to the first row in the first column. 2. Move to the next row and try placing a queen in each column. Check if the position is safe by verifying if any of the previously placed queens is attacking the current position. If it’s a safe position, place a queen in that column. 3.Boolean function is safe(board, row, col): […]
Targeted ability activated by MMLU
Appendix
Table 12: Qualitative comparison of examples selected by Embedding, LESS, and NAIT strategies across different benchmarks. The samples here from Nait are outside of the intersection of general core data.
Instruction fine-tuning is employed to enhance the instruction-following ability of large language models (LLMs). As the amount of instruction fine-tuning data increases, selecting the optimal core set becomes particularly important. However, ensuring the diversity of the core set remains a significant challenge. Existing methods predominantly distinguish different training data based on the text features themselves, decoupled from LLMs' own understanding and representation of the data. To address this issue, we propose a Model-Aware Diverse Core Set Selection method, which distinguishes data features based on the neural activation states during LLM inference. This approach serves as an efficient instantiation of coverage-based selection using model-intrinsic activation features to ensure the diversity in the core set. We extensively evaluate our method on six benchmarks that cover five distinct tasks. In our method, the core set selected by the 3B-parameter LLM performs effectively when utilized to fine-tune larger models with 7B, 8B, and 13B parameters. Experimental results on the Alpaca-GPT4 dataset, which comprises 52K instruction-response pairs, show that the core set, sized at 15% of the original dataset and selected by Llama-3.2-3B-Instruct, achieves an average improvement of 2.5% when fine-tuning four larger base models compared with training on the full dataset. The experimental results demonstrate that our method enhances model performance on multiple downstream tasks while reducing data requirements.
Yi Bai, Wenhao Zhang, Yao Chen +3
Shandong University, Qingdao, China · Inspurcloud, Jinan, China
With Large Language Model (LLM) pre-training and fine-tuning shifting its focus from data volume to data quality, quality data selection has emerged as a critical research topic. Existing online data selection methods for LLM training are typically "batch-constrained", limiting optimization to local utility within random batches. To overcome this, we propose GAIA (Global Adaptive Instruction tuning via GAussian processes), a framework that formulates data valuation as a global estimation process. GAIA employs Gaussian Process regression to model continuous utility manifolds across the semantic space, utilizing an adaptive strategy fusion mechanism to dynamically prioritize high-utility samples. By casting the strategy-posterior update as an instance of the classical fixed-share Hedge framework for tracking the best expert, we inherit a dynamic-regret guarantee that characterizes GAIA's robustness under non-stationary quality scores during training. Empirical evaluations on three datasets demonstrate that GAIA significantly outperforms state-of-the-art baselines like \greats, establishing our method as a scalable and robust solution for efficient instruction tuning.
Instruction tuning has optimized the specialized capabilities of large language models (LLMs), but it often requires extensive datasets and prolonged training times. The challenge lies in developing specific capabilities by identifying useful data and efficiently fine-tuning. High-quality and diverse pruned data can help models achieve lossless performance at a lower cost. In this paper, we propose SLAP, a novel batch-aware data selection framework that evaluates the learnability of entire batch compositions rather than individual. SLAP ensures comprehensive data distribution coverage through distribution-aware stratified sampling while maximizing intra-batch diversity through relative distance optimization. By leveraging Hessian-approximated gradient information for dynamic batch selection, SLAP significantly outperforms existing state-of-the-art methods across multiple model architectures (LLaMA, ChatGLM) and diverse downstream tasks including multi-turn dialogue, multilingual translation, and question answering. Most notably, SLAP achieves superior performance with 20-40% less training data compared to full dataset training, substantially reducing computational costs while maintaining or improving model capabilities. These results establish SLAP as a powerful approach for efficient and effective instruction tuning of large language models.
Run Zou, Jianhang Ding, Yifan Ding +3
Alibaba International Digital Commerce Group, HangZhou, China · Hangzhou Lingju Intelligence AI Lab, Hangzhou, China · Hangzhou Dianzi University, HangZhou, China