Label Aggregation

Label aggregation focuses on combining multiple, potentially noisy or conflicting labels for the same data point to obtain a more accurate and reliable consensus label. Current research explores diverse approaches, including statistical models (like Bayesian networks and mixture models), graph neural networks leveraging semantic relationships (e.g., using Abstract Meaning Representations), and machine learning techniques that incorporate annotator characteristics or instance features to improve aggregation accuracy. This field is crucial for improving the quality of training data in machine learning, particularly in scenarios involving crowdsourcing, weak supervision, and the use of large language models as annotators, ultimately leading to more robust and reliable AI systems.