Aggregated Label

Aggregated labels, representing the combined judgments of multiple annotators on a data point, are a central challenge in machine learning, particularly for subjective tasks. Current research focuses on developing methods to effectively learn from aggregated data, including techniques like multiple instance learning, learning from label proportions, and novel label aggregation algorithms that leverage historical annotator data or annotator-specific models. These advancements aim to improve model accuracy and calibration while addressing issues like noise, bias, and the loss of information inherent in aggregating individual annotations, ultimately leading to more robust and reliable machine learning models across various applications.