Expert Annotation
Expert annotation, the process of labeling data with domain-specific knowledge, is crucial for training accurate machine learning models, particularly in specialized fields like medicine and law. Current research focuses on leveraging large language models (LLMs) to either automate or assist human experts in this process, exploring techniques like active learning, data augmentation, and prompt engineering to improve efficiency and accuracy while mitigating biases introduced by LLMs. This work is significant because it addresses the high cost and time constraints associated with traditional expert annotation, enabling the development of more robust and reliable AI systems across various scientific and practical applications.
Papers
Large Language Models as Financial Data Annotators: A Study on Effectiveness and Efficiency
Toyin Aguda, Suchetha Siddagangappa, Elena Kochkina, Simerjot Kaur, Dongsheng Wang, Charese Smiley, Sameena Shah
"You are an expert annotator": Automatic Best-Worst-Scaling Annotations for Emotion Intensity Modeling
Christopher Bagdon, Prathamesh Karmalker, Harsha Gurulingappa, Roman Klinger