cs.LGFeb 9, 2023
SaveThe Re-Label Method For Data-Centric Machine Learning
Abstract
In industry deep learning application, our manually labeled data has a certain number of noisy data. To solve this problem and achieve more than 90 score in dev dataset, we present a simple method to find the noisy data and re-label the noisy data by human, given the model predictions as references in human labeling. In this paper, we illustrate our idea for a broad set of deep learning tasks, includes classification, sequence tagging, object detection, sequence generation, click-through rate prediction. The dev dataset evaluation results and human evaluation results verify our idea.
Figures & tables
Figure 1: Re-label method applying to any machine learning application.
| model | Dev Dataset Accuracy | Human Evaluation Accuracy |
|---|---|---|
| model-v1 | 83.3% | 88.0% |
| model-v2 | 91.7% | 97.2% |
| model-v3 | 93.8% | 97.5% |
Table 1: The experiment result of text classification. The model-v1 and model-v2 are corresponding to the previous section
| model | Dev Dataset F1 | Human Evaluation Precision |
|---|---|---|
| model-v1 | 73.7% | 86.0% |
| model-v2 | 88.7% | 97.0% |
Table 2: The experiment result of NER. The model-v1 and model-v2 are corresponding to the previous section
| model | Human Evaluation |
|---|---|
| model-v1 | 94.1% |
| model-v2 | 98.3% |
Table 3: The experiment result of controllable text generation. The model-v1 and model-v2 are corresponding to the previous section. We evaluate the available rate of generated text
More in this topic
Learning with Noisy Labels
Learning with noisy labels studies how to train reliable models when annotations are incorrect, incomplete, subjective, or weakly supervised.
See all 150 papers