Data valuation is a cornerstone of data-centric learning, where prior efforts primarily focus on designing algorithms to classify training samples as either beneficial or detrimental for the learning task. However, leveraging these valuation estimates for subsequent data intervention remains underexplored; conventional approaches typically discard or downweight harmful samples, thereby underutilizing available data resources. In this paper, we present Dynamic Influence-based Valuation and Editing (DIVE), a novel and efficient framework that dynamically estimates sample values at the batch level and transforms detrimental data into beneficial contributions. Rather than altering the raw data, DIVE operates at the optimization level by strategically reversing the gradient directions of harmful samples during training, ensuring seamless integration with standard learning procedures with minimal overhead. Extensive empirical evaluations demonstrate that DIVE consistently improves classification performance, maximizes data efficiency, stabilizes optimization, and effectively generalizes to large language model fine-tuning.
Figures & tables
Method
CIFAR-10Na
CIFAR-10Nw
CIFAR-10Nr
CIFAR-100N
Animal-10N
Food-101N
Avg.
Vanilla
88.94 ±0.16
78.97 ± 0.22
86.62 ± 0.28
54.41 ± 0.49
82.60 ± 0.26
72.42 ± 0.14
77.33
LiSSA ( 22 )
89.03 ± 0.50
78.38 ± 0.50
86.32 ± 0.13
54.67 ± 0.30
82.41 ± 0.25
72.07 ± 0.22
77.15
IP ( 45 )
89.54 ± 0.19
77.82 ± 0.40
86.54 ± 0.11
55.03 ± 0.18
82.72 ± 0.35
72.13 ± 0.27
77.30
SPL ( 24 )
88.98 ± 0.26
70.53 ± 0.85
86.32 ± 0.34
38.47 ± 0.55
83.38 ± 0.18
72.97 ± 0.07
73.44
InfoBatch ( 31 )
89.64 ± 0.17
80.90 ± 0.34
87.92 ± 0.21
56.36 ± 0.47
83.52 ± 0.30
72.83 ± 0.09
78.53
GHOST ( 41 )
90.11 ± 0.16
83.25 ± 0.26
88.74 ± 0.17
57.52 ± 0.43
84.10 ± 0.38
72.73 ± 0.16
79.41
Table 1: Accuracy of data-centric methods on noisy benchmarks with a validation set
Method
CIFAR-10Na
CIFAR-10Nw
CIFAR-10Nr
CIFAR-100N
Animal-10N
Food-101N
Vanilla
1.00
1.00
1.00
1.00
1.00
1.00
LiSSA ( 22 )
2.50
2.65
2.64
2.62
2.80
2.50
IP ( 45 )
2.50
2.64
2.62
2.65
2.80
2.51
SPL ( 24 )
1.00
1.00
1.00
1.01
1.01
1.00
InfoBatch ( 31 )
1.01
1.02
1.02
1.01
1.02
1.01
GHOST ( 41 )
1.22
1.22
1.22
1.22
1.25
1.22
Table 2: Training overhead relative to the vanilla baseline.
Figure 1: In-depth exploration of DIVE on CIFAR-10N-w . (A) Test accuracy vs. epoch, shown from the end of warm-up. (B) Influence scores of three individual training samples across epochs (from the end of warm-up). (C) Fraction of samples corrected by DIVE per epoch. In (D)–(F) the horizontal axis is training runtime relative to vanilla, colour denotes the method, and marker shape denotes the ablated hyperparameter; points of the same method are joined by a line. (D) Warm-up-duration ablation (marker: 10/20/30/50%). (E) Editing-frequency ablation (marker: edit every k∈{2,3,5,10} mini-batches). (F) Scoring-layer ablation (marker: last 1/2/3 gradients for DIVE and LAI; GHOST uses the full head, a single point).
Method
CIFAR-10Na
CIFAR-10Nw
CIFAR-10Nr
CIFAR-100N
Animal-10N
Food-101N
Avg.
Vanilla
88.94 ± 0.16
78.97 ± 0.22
86.62 ± 0.28
54.41 ± 0.49
82.60 ± 0.26
72.42 ± 0.14
77.33
LiSSA ( 22 )
88.73 ± 0.15
78.38 ± 0.50
86.23 ± 0.22
54.45 ± 0.15
82.41 ± 0.25
72.00 ± 0.10
77.03
IP ( 45 )
88.67 ± 0.53
78.31 ± 0.42
86.56 ± 0.49
54.55 ± 0.25
82.60 ± 0.45
72.03 ± 0.16
77.12
SPL ( 24 )
89.00 ± 0.17
70.53 ± 0.85
86.01 ± 0.45
38.47 ± 0.55
83.27 ± 0.51
72.79 ± 0.13
73.34
InfoBatch ( 31 )
89.74 ± 0.06
81.12 ± 0.24
88.02 ± 0.19
56.42 ± 0.35
83.29 ± 0.40
72.80 ± 0.22
78.56
GHOST ( 41 )
89.71 ± 0.09
82.26 ± 0.32
88.45 ± 0.17
55.96 ± 0.30
84.16 ± 0.19
72.70 ± 0.09
78.87
Table 3: Accuracy of data-centric methods on noisy benchmarks without a validation set
Dataset
#Class
#Train
#Test
Test Loss
Test Accuracy
Vanilla
DIVE
Vanilla
DIVE
PAWS
2
49,401
8,000
0.1584
0.1549
0.9515
0.9514
ANLI_r3
3
100,459
1,200
0.8049
0.7374
0.6806
0.6988
Civil_Comments_b
2
200,000
97,320
0.1366
0.1150
0.8377
0.8590
Civil_Comments
2
1,804,874
97,320
0.0702
0.0676
0.9531
0.9518
Table 4: Performance of DIVE for LLM fine-tuning
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
# Train
# Test
# Classes
Noise Type
Noise Rate
CIFAR-10Na
50,000
10,000
10
Human-annotated (aggregate)
∼9%
CIFAR-10Nr
50,000
10,000
10
Human-annotated (random)
∼17%
CIFAR-10Nw
50,000
10,000
10
Human-annotated (worst)
∼40%
CIFAR-100N
50,000
10,000
100
Human-annotated
∼40%
Animal-10N
50,000
5,000
10
Human-annotation
∼8%
Food-101N
75,750
25,250
101
Intrinsic web-label
∼20%
Appendix
Table 5: Summary statistics of the six image benchmarks used in our experiments.
Figure 2: In-depth exploration of DIVE on Animal-10N .
Figure 3: In-depth exploration of DIVE on Food-101N .