Large Language Model Annotations

Ask about this topic

Momentum

4 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Jul 6Week of Sep 21
View on map

Latest papers 62

All topics
CardsList

  1. Can LLMs Reliably Annotate Bioassay Metadata to Improve Data Readiness?

    Oct 1, 2026Laura van Weesep, Riccardo Tedoldi, Jens Sjölund +6Large Language Model AnnotationsMolecular Property Prediction

    Molecular AI, Discovery Sciences R&D, AstraZeneca · Department of Information Technology, Uppsala University · Robotics, Perception & Learning, KTH Royal Institute of Technology +3

  2. Paying for Too Many Tokens? Valid and Cost-Efficient Multimodal LLM Annotation with Simple Heuristics

    Sep 30, 2026Zhixi Zhu, Kristina GligoricLong Visual-Token SequencesLarge Language Model Annotations

    Johns Hopkins University

  3. onPanda: Efficient Annotation of On-Policy Alignment Data for LLMs and Agents via Token-Level Correction

    Sep 21, 2026Lei Yang, Mengyin Liu, Jia Wang +9Large Language Model AnnotationsLarge Language Model Alignment

    StepFun · Xiamen University

  4. Evaluating Decision Models for Text Annotation in Computational Social Science

    Sep 21, 2026Hazem Ibrahim, Yasir ZakiLarge Language Model AnnotationsComputational Social Science

    Computer Science, New York University Abu Dhabi, Abu Dhabi, UAE

  5. QLoRA Fine-Tuning of Ministral LLM for Sequence-to-Function Protein Annotation

    Sep 21, 2026Demian Pavlyshenko, Bohdan PavlyshenkoMessenger RnaLarge Language Model Annotations

  6. The Differential Reasoning Router: Operationalizing Cost-Aware LLM Annotation in E-commerce

    Aug 31, 2026Cheng Lyu, Jingyue Zhang, Vinny DeGenova +2Large Language Model AnnotationsLLM Reasoning Strategies

    Wayfair

  7. Debiased Inference for AI-Generated Data without Gold-Standard Labels: Identification via Multiple Imperfect Measurements

    Aug 18, 2026Naoki Egami, Sooahn ShinStatistical InferenceLarge Language Model Annotations

    Department of Political Science and Statistics and Data Science Center, Massachusetts Institute of Technology, Cambridge, MA 02139 · Department of Political Science, Massachusetts Institute of Technology, Cambridge, MA 02139

  8. H2: A Dual Hybrid Semantic Data Lake Architecture for Medical Data Harmonization with Human-In-the-Loop verified, LLM Driven Metadata Annotation System

    Aug 8, 2026Ioannis N. Tzortzis, Georgia Kapetadimitri, Agapi Davradou +6Data LakesElectronic Health Records

    Institute of Communication and Computer Systems (ICCS), 15780 Zografou, Athens, Greece. · University of Macedonia (UOM), 54636, Thessaloniki, Greece.

  9. Human-Like Anaphor Resolution in Large Language Models

    Aug 6, 2026Keane Zhang, Varshini Chinta, Raj Sanjay Shah +1Large Language Model AnnotationsCognitive Science

    Georgia Institute of Technology

  10. ChartAnno: Benchmarking Multimodal Large Language Models for Chart Annotation Generation

    Aug 4, 2026Zhenghan Chen, Zekai Shao, Lidan Tan +10ChartLarge Language Model Annotations

    Fudan University · Sun Yat-sen University · The Hong Kong University of Science and Technology (Guangzhou) +1

  11. Challenges in annotations by humans and LLMs: A case study of evaluative language

    Jul 30, 2026Mirela Imamovic, Aenne Cecilia Kristine Knierim, Khushi Pitroda +1Large Language Model AnnotationsHuman Annotations

    University of Hildesheim, Institute of Translation Studies and Specialised Communication, Germany · University of Paris 8 Vincennes-Saint Denis, TransCrit, France · University of Hildesheim, Institute of Information Science and Language Technology, Germany

  12. STEREODISCO: Discovering Stereotypicality in LLMs

    Jul 30, 2026Farane Jalali Farahani, Corina Dima, Mojtaba Nayyeri +2Stereotype MitigationLarge Language Model Annotations

    Institute for Artificial Intelligence, Stuttgart, Germany · Institute for Social Science, Stuttgart, Germany · University of Southampton, Southampton, United Kingdom

  13. LLMs Can Annotate Attribution Graphs

    Jul 28, 2026Ameen Patel, Max Zhang, Nathan HuLarge Language Model AnnotationsText-Attributed Graph

    Stanford University

  14. Where Animacy Lives in Large Language Models: Tracing the Circuits of the Animacy Concept

    Jul 23, 2026Samuele Punzo, Giovanni Cinà, Sandro PezzelleLarge Language Model AnnotationsContextual Cues

    Graduate School of Informatics, University of Amsterdam · Department of Medical Informatics, Amsterdam University Medical Center · ILLC, University of Amsterdam

  15. Self-Evolving Human-Centered Framework for Explainable Depression Symptom Annotation

    Jul 16, 2026Hoang-Loc Cao, Van Pham, Truong Thanh Hung Nguyen +4DepressionLarge Language Model Annotations

    University of Science, VNU-HCM, Vietnam · University of New Brunswick, Canada · Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany

  16. Dimensionality in Satisfaction Ratings

    Jul 13, 2026Andrew Hong, Jason PotteigerClientsSentiment

    Dimension Labs

  17. Demographic Prompting at Scale: When More Attributes Hurt LLM--Human Agreement

    Jul 12, 2026Mahammed Kamruzzaman, Shrabon Kumar Das, Gene Louis KimLarge Language Model AnnotationsDemographics

    Bellini College of AI, Cybersecurity and Computing · University of South Florida

  18. A knowledge-augmented dataset of high-risk driving scenarios with LLM annotations for autonomous driving

    Jul 8, 2026Heye Huang, Jingguang Li, Zhiyuan Zhou +4Naturalistic Driving DataAutonomous Driving

    Cho Chun Shik Graduate School of Mobility, Korea Advanced Institute of Science and Technology, Daejeon 34051, South Korea · College of Computer Science and Artificial Intelligence, Fudan University, Shanghai 200433, China · Media Lab, Massachusetts Institute of Technology, Cambridge, MA 02139, USA +2

  19. Evaluating Large Language Models for Antisemitic Incident Classification

    Jul 6, 2026Karina Halevy, Julia Mendelsohn, Chan Young Park +2Hate Speech DetectionLarge Language Model Annotations

    Carnegie Mellon University · University of Maryland · Microsoft Research +2

  20. Multi-Agentic System Leveraging Open-Source LLMs to Mitigate Disinformation Threats

    Jun 29, 2026Sebastian Kula, Martin TamajkaMisinformation DetectionMulti-Agent Large Language Model Systems

    Kempelen Institute of Intelligent Technologies, Bratislava, Slovakia · West Pomeranian University of Technology in Szczecin, Szczecin, Poland

  21. A Hybrid Framework for Song Lyric Annotation Based on Human-LLM Alignment

    Jun 28, 2026Rashini Liyanarachchi, Frank Tran, Md Mahmudul Hasan +2Large Language Model AnnotationsAnnotations

    School of Computer Science and Engineering, University of New South Wales, Sydney, Australia

  22. Majority Vote Silences Minority Values: Annotator Disagreement at the Hate/Offensive Boundary in HateXplain

    Jun 27, 2026Joshua Muhumuza, Joab Ezra Agaba, Mercy AmiyoHate Speech DetectionDisagreement

    Makerere University, Kampala, Uganda.

  23. Scaling Performance and Low-Resource Annotation with Many-Shot In-Context Learning for Named Entity Recognition

    Jun 20, 2026Qi Zhang, Fangping Lan, Cornelia Caragea +2Named-Entity RecognitionLarge Language Model Annotations

    Temple University · University of Illinois Chicago

  24. LLM-Based Multi-Reference Evaluation for Efficient and Robust Assessment of Phrase Break Annotations

    Jun 19, 2026Younghan Park, Hoyeon Lee, Hawon Jeong +1Large Language Model AnnotationsProsody

    NAVER Cloud, South Korea · Yonsei University, South Korea · KAIST AI, South Korea

  25. What sentiment analysis can't see: Measuring whether customers were helped, and what went wrong, across 70,000 support conversations

    Jun 18, 2026Jason PotteigerSentimentLarge Language Model Annotations

    Dimension Labs

  26. Detecting Sensitive Personal Information in Japanese Pre-Training Corpora for Large Language Models

    Jun 10, 2026Rei Minamoto, Yusuke Oda, Daisuke KawaharaLarge Language Model AnnotationsPrivacy

    Waseda University · Research and Development Center for LLMs, National Institute of Informatics

  27. Measuring Human Value Expression in Social Media Texts: Calibrated LLM Annotation and Encoder Transfer

    Jun 9, 2026Maria Milkova, Maksim RudnevLarge Language Model AnnotationsHuman Values

    Independent researcher, Lisbon, Portugal · University of Waterloo, ON, Canada

  28. Does Topic Sentiment Cause Perceived Ideology? Comparing Human and LLM Annotations in Political News Articles

    Jun 4, 2026Upasana ChatterjeeIdeologyPolitical Discourse

    Department of Computer Science Columbia University

  29. From Self to Other: Evaluating Demographic Perspective-Taking in LLM Hate Speech Annotation

    Jun 4, 2026Paloma Piot, Javier ParaparHate Speech DetectionLarge Language Model Annotations

    Information Retrieval Lab CITIC Research Centre Universidade da Coruña

  30. Framing Migration News with LLMs: Structured CoT as a Support for Human Interpretation

    Jun 2, 2026David Alonso del Barrio, Jing Wen, Daniel Gatica-PerezTask FramingLarge Language Model Annotations

    Idiap Research Institute and EPFL Switzerland

  31. The Ghost Annotator: a Framework to Explore Human Label Variation in Content Moderation through Conformal Prediction

    Jun 1, 2026Mirko Lai, Alessandra Urbinati, Simona Frenda +2Large Language Model AnnotationsHuman Annotators

    Heriot-Watt University, Edinburgh, Scotland · aequa-tech, Torino, Italy · Laboratory for the Modeling of Biological and Socio-technical Systems, Northeastern University, Boston, MA, USA +2

  32. When Rating Scales Fall Short: LLM-Assisted Discovery of ADHD Signals in Turkish Teacher Narratives

    Jun 1, 2026Baris Karacan, Irem Aktar Songur, Ahmet Ozaslan +1Autism Spectrum DisorderComputerized Adaptive Testing

    Department of Computer Science, University of Illinois Chicago · Department of Child and Adolescent Psychiatry, Gazi University

  33. On the Limits of LLM Adaptability: Impact of Model-Internalized Priors on Annotation Task Performance

    May 30, 2026Etienne Casanova, Rafal Kocielnik, R. Michael AlvarezLarge Language Model AnnotationsLarge Language Models Fail

    California Institute of Technology, Pasadena, CA, USA.

  34. Where LLM Annotators Fail: Label-Free Learning on Graphs with LLMs

    May 27, 2026Safal Thapaliya, Jiatan Huang, Chuxu ZhangGraph Representation LearningLarge Language Model Annotations

    University of Connecticut, USA

  35. Attribute-Based Diagnosis of LLM Alignment with Hate Speech Annotations

    May 26, 2026Mohammad Amine Jradi, Faeze Ghorbanpour, Alexander FraserHate Speech DetectionLarge Language Model Annotations

    School of Computation, Information and Technology, TU Munich Munich Center for Machine Learning (MCML)

  36. WhoSaidIt: Human-LLM Collaborative Annotation for Text-Based Multilingual Speaker-Attribute Classification

    May 25, 2026Lingyu Gao, Will Monroe, David Smith +2Large Language Model AnnotationsSpeaker

    Duolingo

  37. Refining and Reusing Annotation Guidelines for LLM Annotation

    May 20, 2026Kon Woo Kim, Jin-Dong Kim, Akiko AizawaLarge Language Model AnnotationsAnnotations

    The Graduate University for Advanced Studies, SOKENDAI · National Institute of Informatics (NII) · BioData Science Initiative (BSI), National Institute of Genetics (NIG)

  38. Can Multi-Agent LLMs Identify Their Peers? Stylometric Fingerprinting in Role-Constrained Political Analysis

    May 20, 2026Juergen DietrichLarge Language Model AnnotationsAnonymization

    Democracy Intelligence gGmbH, Berlin, Germany

  39. iPOE: Interpretable Prompt Optimization via Explanations

    May 18, 2026Jiahui Li, Yarik Menchaca Resendiz, Sean Papay +1Prior Prompt Optimization TechniquesPrompt Engineering

    Fundamentals of Natural Language Processing, University of Bamberg, Germany · Leibniz-Institut für Psychologie (ZPID), Trier, Germany

  40. LLMs for automatic annotation of Mandarin narrative transcripts

    May 17, 2026Qingwen Zhao, Hongao Zhu, Yunqi He +3Large Language Model AnnotationsHuman Annotations

    Shanghai Jiao Tong University, China · University of California San Diego, USA · The Hong Kong Polytechnic University, China +2

  41. Improving Reproducibility in Evaluation through Multi-Level Annotator Modeling

    May 13, 2026Deepak Pandita, Flip Korn, Chris Welty +1ReproducibilityLarge Language Model Annotations

    Rochester Institute of Technology · Google Research

  42. LLMs as annotators of credibility assessment in Danish asylum decisions: evaluating classification performance and errors beyond aggregated metrics

    May 13, 2026Galadrielle Humblot-Renaux, Mohammad N. S. Jahromi, Rohat Bakuri-Jørgensen +7Large Language Model AnnotationsCredibility

    Visual Analysis and Perception Lab, Aalborg University · Pioneer Center for AI, Denmark · Center of Excellence for Global Mobility Law, University of Copenhagen +1

  43. Two Wrongs, No Right: Auditing Social-Desirability Bias in LLM Annotators for Computational Social Science

    May 12, 2026Varun KotteLarge Language Model BiasLarge Language Model Annotations

    Independent Researcher

  44. MultiSoc-4D: A Benchmark for Diagnosing Instruction-Induced Label Collapse in Closed-Set LLM Annotation of Bengali Social Media

    May 7, 2026Souvik Pramanik, S. M. Riaz Rahman Antu, Shak Mohammad Abyad +2Large Language Model AnnotationsBangla

    North South University, Dhaka, Bangladesh

  45. Addressing Labelled Data Scarcity: Taxonomy-Agnostic Annotation of PII Values in HTTP Traffic using LLMs

    May 7, 2026Thomas Cory, Axel KüpperLarge Language Model AnnotationsHuman Annotations

    Technische Universität Berlin

  46. Annotation Quality in Aspect-Based Sentiment Analysis: A Case Study Comparing Experts, Students, Crowdworkers, and Large Language Model

    May 5, 2026Niklas Donhauser, Jakob Fehle, Nils Constantin Hellwig +3Large Language Model AnnotationsSentiment

    Media Informatics Group, University of Regensburg, Regensburg, Germany · Information Science Group, University of Regensburg, Regensburg, Germany

  47. Stable Behavior, Limited Variation: Persona Validity in LLM Agents for Urban Sentiment Perception

    Apr 30, 2026Neemias B da Silva, Rodrigo Minetto, Daniel Silver +1Artificial Intelligence PersonasPersona Consistency

    Universidade Tecnol´ogica Federal do Paran´a (UTFPR), Curitiba, Brazil · University of Toronto, Toronto, Canada

  48. LLMs Capture Emotion Labels, Not Emotion Uncertainty: Distributional Analysis and Calibration of Human-LLM Judgment Gaps

    Apr 30, 2026Keito Inoshita, Xiaokang Zhou, Akira Kawai +1Large Language Model AnnotationsEmotion Recognition

    Faculty of Business and Commerce, Kansai University, Osaka, Japan · Faculty of Business Data Science, Kansai University, Osaka, Japan · RIKEN Center for Advanced Intelligence Project, Tokyo, Japan +2

  49. Chinese-SkillSpan: A Span-Level Dataset for ESCO-Aligned Competency Extraction from Chinese Job Ads

    Apr 24, 2026Guojing Li, Zichuan Fu, Junyi Li +6JobsNamed-Entity Recognition

    City University of Hong Kong · Renmin University of China

  50. LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments

    Apr 24, 2026Xiomara Gonzalez, Gabriella Coloyan Fleming, Andrew Katz +2Qualitative ResearchSentiment

    Chandra Family Department of Electrical and Computer Engineering, University of Texas at Austin · Department of Engineering Education, Virginia Polytechnic Institute and State University · Gallogly College of Engineering, University of Oklahoma +1

  51. Whose Story Gets Told? Positionality and Bias in LLM Summaries of Life Narratives

    Apr 22, 2026Melanie Subbiah, Haaris Mian, Nicholas Deas +3Large Language Model BiasLarge Language Model Annotations

    Department of Computer Science, Columbia University · Department of Applied Physics and Applied Mathematics, Columbia University · Department of Psychology, Northwestern University

  52. REALM: Reliable Expertise-Aware Language Model Fine-Tuning from Noisy Annotations

    Apr 19, 2026Sajjad Ghiasvand, Mark Beliaev, Mahnoosh Alizadeh +1Large Language Model AnnotationsLarge Language Model Fine-Tuning

    Department of Electrical and Computer Engineering UC Santa Barbara Santa Barbara, CA 93106, USA

  53. Dynamic Emotion and Personality Profiling for Multimodal Deception Detection

    Apr 18, 2026Li Zheng, Yanyi Luo, Hao Fei +5Emotion RecognitionDeception

    Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University, Wuhan, China · National University of Singapore, Singapore, Singapore

  54. Sentiment Analysis of German Sign Language Fairy Tales

    Apr 17, 2026Fabrizio Nunnari, Siddhant Jain, Patrick GebhardSentimentSign Language Translation

    German Research Center for Artificial Intelligence (DFKI) Saarland Informatics Campus D3 2, 66123 Saarbrücken, Germany

  55. DiZiNER: Disagreement-guided Instruction Refinement via Pilot Annotation Simulation for Zero-shot Named Entity Recognition

    Apr 17, 2026Siun Kim, Hyung-Jin YoonNamed-Entity RecognitionLarge Language Model Annotations

    Seltasquare Seoul, Korea · Biomedical Research Institute, Seoul National University Hospital · Seoul National University College of Medicine Seoul, Korea

  56. Consistency Analysis of Sentiment Predictions using Syntactic & Semantic Context Assessment Summarization (SSAS)

    Apr 16, 2026Sharookh Daruwalla, Nitin Mayande, Shreeya Verma Kathuria +2SentimentLarge Language Model Annotations

    1Tellagence Inc.∗ · 2Villanova School of Business, Villanova University† · 3Maseeh College of Engineering and Computer Science, Portland State University‡

  57. Do We Still Need Humans in the Loop? Human vs. LLM Annotation in Active Learning for TikTok Hate Speech Detection

    Apr 15, 2026Ahmad Dawar Hakimi, Lea Hirlimann, Isabelle Augenstein +1Large Language Model AnnotationsAutomatic Labelling

    Center for Information and Language Processing, LMU Munich, Germany · Department of Computer Science, University of Copenhagen, Denmark · Munich Center for Machine Learning, Germany

  58. When Hate Meets Facts: LLMs-in-the-Loop for Check-worthiness Detection in Hate Speech

    Mar 26, 2026Nicolás Benjamín Ocampo, Tommaso Caselli, Davide CeolinHate Speech DetectionHate Speech

    Centrum Wiskunde & Informatica · Amsterdam, The Netherlands · University of Groningen +1

  59. Multi-Perspective LLM Annotations for Valid Analyses in Subjective Tasks

    Mar 22, 2026Navya Mehrotra, Adam Visokay, Kristina GligorićLarge Language Model AnnotationsHuman Annotations

    University of Washington

  60. Standardizing Longitudinal Radiology Report Evaluation via Large Language Model Annotation

    Jan 23, 2026Xinyi Wang, Grazziela Figueredo, Ruizhe Li +1Radiology Report GenerationLarge Language Model Annotations

    School of Computer Science, The University of Nottingham, Nottingham, NG7 2RD, United Kingdom. · School of Medicine, The University of Nottingham, Nottingham, NG7 2RD, United Kingdom.