LLM-Assisted Annotation

Ask about this topic

Momentum

17 papers in the last four weeks, up 143% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 123

All topics
CardsList

  1. Goldsmith: Gold-Loss-Guided Definition Optimization with an Agentic Annotation Harness

    Oct 7, 2026Yihan Li, Hanyi Zhang, Xiaoxi Jiang +1LLM-Assisted AnnotationLarge Language Model-Guided Optimization

    Sun Yat-sen University, Guangzhou, China · South China University of Technology, Guangzhou, China

  2. Agreement Is Not Validity: Cross-Model LLM Consensus in Diagnosing Student Failure Modes in K-12 Math Tutoring Dialogue

    Oct 6, 2026Clayton Cohn, Joyce Fonteles, Kirk Vanacore +5LLM EvaluationLLM-Assisted Annotation

    College of Computing and Information Science, Cornell University, Ithaca, USA. · College of Connected Computing, Vanderbilt University, Nashville, USA. · Third Space Learning, Swindon, UK.

  3. Can LLMs Reliably Annotate Bioassay Metadata to Improve Data Readiness?

    Oct 1, 2026Laura van Weesep, Riccardo Tedoldi, Jens Sjölund +6LLM AuditingLLM-Assisted Annotation

    Molecular AI, Discovery Sciences R&D, AstraZeneca · Department of Information Technology, Uppsala University · Robotics, Perception & Learning, KTH Royal Institute of Technology +3

  4. LLM-Assisted Discovery of Typed Semantic Links for Ontology Network Construction

    Oct 1, 2026Nouha Hayouni, Sheeba Samuel, Alsayed AlgergawyRelation ExtractionKG Construction

    Chair of Data and Knowledge Engineering, University of Passau, Germany · Distributed and Self-organizing Systems, Chemnitz University of Technology, Germany

  5. Paying for Too Many Tokens? Valid and Cost-Efficient Multimodal LLM Annotation with Simple Heuristics

    Sep 30, 2026Zhixi Zhu, Kristina GligoricVLM EvaluationVideo-Language Model Evaluation

    Johns Hopkins University

  6. QuanReview: Offline, Auditable Reconciliation of Human and LLM Span Annotations

    Sep 28, 2026Matteo Musacchio, Juan Cruz Giner Pulero, Isabel Castañeda +4Annotator DisagreementLLM-Assisted Annotation

    Universidad de San Andrés, Buenos Aires, Argentina · ISI Foundation, Turin, Italy · CONICET, Buenos Aires, Argentina +1

  7. EduBehaviors: Assertion-based Schemas for Auditable Coding of Educational Dialogues

    Sep 22, 2026Julian Bernado, Ana Trindade Ribeiro, Xander Beberman +1LLM-Assisted AnnotationAI in Education

    SCALE Initiative, Stanford University

  8. Experts Rise Where LLMs Disagree: Using Cross-Model Disagreement to Target Expert Effort in LLM Codebook Revision for Large-Scale Annotation

    Sep 22, 2026Zeyu He, Zhuqian Zhou, Kirk Vanacore +2LLM-Assisted AnnotationHuman-in-the-Loop Annotation

    The Pennsylvania State University, University Park, PA, USA · Cornell University, Ithaca, NY, USA

  9. onPanda: Efficient Annotation of On-Policy Alignment Data for LLMs and Agents via Token-Level Correction

    Sep 21, 2026Lei Yang, Mengyin Liu, Jia Wang +9LLM AlignmentLLM-Assisted Annotation

    StepFun · Xiamen University

  10. Evaluating Decision Models for Text Annotation in Computational Social Science

    Sep 21, 2026Hazem Ibrahim, Yasir ZakiLLM EvaluationLLM-Assisted Annotation

    Computer Science, New York University Abu Dhabi, Abu Dhabi, UAE

  11. QLoRA Fine-Tuning of Ministral LLM for Sequence-to-Function Protein Annotation

    Sep 21, 2026Demian Pavlyshenko, Bohdan PavlyshenkoLLM Fine-TuningLLM-Assisted Annotation

  12. Beyond Relevance: Structured Semantic Supervision for Product Search with LLM-Augmented Annotations

    Sep 20, 2026Girish A. Koushik, Swapnil Bhosale, Samarth Agrawal +4Learning to RankLLM-Assisted Annotation

    Computer Science, University of Surrey, Guildford, United Kingdom · People-Centred AI, University of Surrey, Guildford, United Kingdom · eBay USA +2

  13. Reproducibility is not construct validity: LLM measurement of institutionally situated communication

    Sep 17, 2026Veronika Batzdorfer, Carlo Romano Marcello Alessandro SantagiustinaLLM EvaluationLLM-Assisted Annotation

    KIT · Department of Sociology and Computational Sociology, Karlsruhe Institute of Technology (KIT), Douglasstr. 24, 76133 Karlsruhe, Germany · ALMAnaCH, médialab, Sciences Po +2

  14. CliniCIRCA: A Modular LLM Framework for Constructing Longitudinal Mental Health Patient Journeys from Raw EHR Narratives

    Sep 17, 2026Aiwei Ivy Zhang, Nimra Ishfaq, Mohit Chandra +5Temporal Reasoning in Language ModelsMental Health

    Georgia Institute of Technology · University of Texas at Austin · Northwell Health

  15. What Counts as a Mistake? Annotating Recitation Events in Quran Memorization Transcripts

    Sep 14, 2026Mohamad Al Mdfaa, Nursultan Askarbekuly, Ahmed Helaly +2ASR EvaluationLLM-Assisted Annotation

    LemoniLab FZCO (lemonilab.com) · Innopolis University (innopolis.ru)

  16. Extracting Dataset Mentions in Forced Displacement and FCV Documents: A Weakly Supervised Framework with LLM-Based Label Refinement

    Sep 14, 2026Rafael Macalaba, Aivin V. Solatorio, Patrick Michael Brock +1LLM-Assisted AnnotationDocument Information Extraction

    Development Data Group Office of the World Bank Group Chief Statistician The World Bank 1818 H Street N.W., Washington, 20433 District of Columbia, USA · World Bank UNHCR Joint Data Center UN City Marmorvej 51, 2100 Copenhagen, Denmark

  17. Cross-Lingual Clinical Annotation Projection as Constrained Text Generation: A Six-Language Study

    Sep 12, 2026Álvaro Rey-Blanes, Francisco J. Moreno-Barea, Francisco J. VeredasLLM-Assisted AnnotationControllable Text Generation

    Department of Programming Languages and Computer Sciences, Universidad de Málaga, Málaga, Spain · Research Institute of Multilingual Language Technologies, Universidad de Málaga, Málaga, Spain · IBIMA Plataforma BIONAND, Instituto de Investigación Biomédica de Málaga, Málaga, Spain

  18. Debate-to-Skill: Capability-Bound Process Supervision for Industrial Query-to-Agent Annotation

    Sep 12, 2026Shiyu Zhang, Leisheng Cheng, Huifu LiLLM-Assisted AnnotationProcess Supervision

    Baidu, Inc.

  19. SpeechAnnotator: A Context-Aware Multi-Agent Framework and Benchmark for Multidimensional Speech Annotation

    Sep 9, 2026Qirui Zhan, Shuiyuan Wang, Jingbin Hu +10LLM-Assisted AnnotationMulti-Agent Systems

    Northwestern Polytechnical University, China · yutuzhineng, China · Shenzhen Pimei Technology Co., Ltd., Guangdong, China +1

  20. PersianAnonymizer: Evaluating LLM-Labeled Training for Efficient NER-based Anonymization in Persian

    Sep 1, 2026Mohammad Hossein Shalchian, Mostafa Amiri, Amir Mahdi SadeghzadehNamed Entity RecognitionLLM-Assisted Annotation

    1Sharif University of Technology · University of Tehran

  21. Bridging Lexical Divergence: LLM-Assisted, Cost-Efficient, Zero-shot Scientific Entity Linking

    Aug 31, 2026Md Rasel Khondokar, Qiao Qiao, Farjana Sultana Samia +3LLM-Assisted Annotation

    Department of Computer Science, Iowa State University, Ames, Iowa, USA

  22. Error-Type-Aware Loss Reweighting for Robust Named Entity Recognition with Noisy LLM Labels

    Aug 31, 2026Elena Merdjanovska, Jonas Golde, Alan AkbikNamed Entity RecognitionLLM-Assisted Annotation

    Humboldt-Universität zu Berlin · Science of Intelligence

  23. Designing an Auditable LLM-Supported Workflow for Qualitative Thematic Analysis

    Aug 31, 2026Nadia Jul Jeldtoft, Tariq YousefLLM-Assisted Annotation

    Department of Culture and Language 2Department of Mathematics and Computer Science University of Southern Denmark

  24. The Differential Reasoning Router: Operationalizing Cost-Aware LLM Annotation in E-commerce

    Aug 31, 2026Cheng Lyu, Jingyue Zhang, Vinny DeGenova +2LLM-Assisted AnnotationHuman-in-the-Loop Annotation

    Wayfair

  25. Homo-RAG: Homology-Guided Retrieval-Augmented Generation for Cross-Species Gene Function Prediction

    Aug 26, 2026Azrin SultanaRetrieval-Augmented GenerationLLM-Assisted Annotation

    Department of Computer Science, American International University-Bangladesh, 408/1, Kuratoli, Khilkhet, 1229, Dhaka, Bangladesh.

  26. Debiased Inference for AI-Generated Data without Gold-Standard Labels: Identification via Multiple Imperfect Measurements

    Aug 18, 2026Naoki Egami, Sooahn ShinSemiparametric InferenceLatent Variable Models

    Department of Political Science and Statistics and Data Science Center, Massachusetts Institute of Technology, Cambridge, MA 02139 · Department of Political Science, Massachusetts Institute of Technology, Cambridge, MA 02139

  27. H2: A Dual Hybrid Semantic Data Lake Architecture for Medical Data Harmonization with Human-In-the-Loop verified, LLM Driven Metadata Annotation System

    Aug 8, 2026Ioannis N. Tzortzis, Georgia Kapetadimitri, Agapi Davradou +6LLM-Assisted AnnotationHuman-in-the-Loop Annotation

    Institute of Communication and Computer Systems (ICCS), 15780 Zografou, Athens, Greece. · University of Macedonia (UOM), 54636, Thessaloniki, Greece.

  28. ArtAnno: Annotating Implicit Semantics in Artworks through LLM Agent-Driven Bidirectional Human-AI Augmentation

    Aug 5, 2026Xiaoyan Gu, Yifang Wang, Wenqing Zheng +6Multi-Agent LLM SystemsLLM-Assisted Annotation

    School of Computer and Computing Science, Hangzhou City University Hangzhou, Zhejiang, China

  29. Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not Ground Truth

    Jul 30, 2026Alex Liu, Lief Esbenshade, Michael Xiao +4LLM-Assisted AnnotationHuman-in-the-Loop Evaluation

    University of Washington · Colleague AI/ · Colleague AI

  30. Human-LLM Collaborative Inductive Coding for Conceptualizing K-12 Educator AI Use

    Jul 30, 2026Alex Liu, Min Sun, Lief Esbenshade +4LLM-Assisted AnnotationHuman-in-the-Loop Annotation

    College of Education, University of Washington, Seattle, WA, United States · Colleague AI

  31. Challenges in annotations by humans and LLMs: A case study of evaluative language

    Jul 30, 2026Mirela Imamovic, Aenne Cecilia Kristine Knierim, Khushi Pitroda +1LLM EvaluationLLM-Assisted Annotation

    University of Hildesheim, Institute of Translation Studies and Specialised Communication, Germany · University of Paris 8 Vincennes-Saint Denis, TransCrit, France · University of Hildesheim, Institute of Information Science and Language Technology, Germany

  32. A Human-in-the-Loop Corpus for LLM-Based Simplification of Scientific Summaries

    Jul 28, 2026Kyuri Im, Michael FärberLLM-Assisted AnnotationScientific Communication

    ScaDS.AI, Technische Universität Dresden, Dresden, Germany

  33. LLMs Can Annotate Attribution Graphs

    Jul 28, 2026Ameen Patel, Max Zhang, Nathan HuLLM InterpretabilityLLM-Assisted Annotation

    Stanford University

  34. Interpretable Column Annotation with LLM-Symbolized Decision Process Materialization

    Jul 28, 2026Mengqi Wang, Jianwei Wang, Qing Liu +5LLM-Assisted AnnotationInterpretable ML

    UNSW Sydney · University of New South Wales Sydney, NSW, Australia · Data61, CSIRO +4

  35. A Classifier That Teaches Itself: Self-Improving, Frozen-gate Training (SIFT) for Dynamic Document Classification

    Jul 20, 2026Bogdan Raduta, Horia Velicu, Alexandru Preda +1LLM-Assisted AnnotationText Classification

  36. Self-Evolving Human-Centered Framework for Explainable Depression Symptom Annotation

    Jul 16, 2026Hoang-Loc Cao, Van Pham, Truong Thanh Hung Nguyen +4Explainable Artificial IntelligenceLLM-Assisted Annotation

    University of Science, VNU-HCM, Vietnam · University of New Brunswick, Canada · Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany

  37. Quantifying the Sources of Instability in LLM-Based Stance Analysis of Public Discourse

    Jul 12, 2026Bo ChenStance DetectionLLM-Assisted Annotation

    Institute of Computing Technology, Chinese Academy of Sciences

  38. Demographic Prompting at Scale: When More Attributes Hurt LLM--Human Agreement

    Jul 12, 2026Mahammed Kamruzzaman, Shrabon Kumar Das, Gene Louis KimLLM PromptingLLM-Assisted Annotation

    Bellini College of AI, Cybersecurity and Computing · University of South Florida

  39. Validity of LLMs as data annotators: AMALIA on authority

    Jul 9, 2026Manuel PitaMultilingual Language Model EvaluationMoral Reasoning in Language Models

    Artificial Intelligence, Social Interaction and Complexity Laboratory · CICANT, Universidade Lusófona

  40. Scalable and Culturally Specific Stereotype Dataset Construction via Human-LLM Collaboration

    Jul 8, 2026Weicheng Ma, John Guerrerio, Soroush VosoughiMultilingual Language Model EvaluationSocial Bias in Language Models

    College of Computing, Georgia Institute of Technology · Computer Science Department, Dartmouth College

  41. SynthAVE: Scalable Synthetic Labeling for E-Commerce with LLM-Arena Validation

    Jul 8, 2026Andrea Scarinci, Virginia Negri, Brayan Impata +3LLM-as-a-JudgeLLM-Assisted Annotation

    Amazon

  42. Semantic-Guided Reading Order Reconstruction in Historical Armenian Newspapers with LLMs

    Jul 1, 2026Chahan Vidal-Gorène, Nadi Tomeh, Victoria KhurshudyanHistorical Document AnalysisLLM-Assisted Annotation

    CJM, LIPN · 1 École nationale des chartes-PSL · Laboratoire d’Informatique Paris Nord +4

  43. Unlocking the Visual Record of Materials Science: A Large-Scale Multimodal Dataset from Scientific Literature

    Jun 29, 2026Subham Ghosh, Shubham Tiwari, Mohammad Ibrahim +1LLM-Assisted AnnotationMaterials Science

    Mehta Family School of Data Science and Artificial Intelligence Indian Institute of Technology Roorkee, Uttarakhand, India-247667 · Department of Metallurgical and Materials Engineering Indian Institute of Technology Roorkee, Uttarakhand, India-247667

  44. A Hybrid Framework for Song Lyric Annotation Based on Human-LLM Alignment

    Jun 28, 2026Rashini Liyanarachchi, Frank Tran, Md Mahmudul Hasan +2Annotator DisagreementLLM-Assisted Annotation

    School of Computer Science and Engineering, University of New South Wales, Sydney, Australia

  45. Labeling Training Data for Entity Matching Using Large Language Models

    Jun 27, 2026Aaron Steiner, Christian BizerLLM-Assisted AnnotationKnowledge Distillation

    Data and Web Science Group University of Mannheim Mannheim, Germany

  46. From Clicks to Intent: Cross-Platform Session Embeddings with LLM-Distilled Taxonomy for Financial Services Recommendations

    Jun 24, 2026Dianjing Fan, Yao Li, Kyaw Hpone Myint +4Sequential RecommendationRepresentation Learning

    Capital One McLean, VA, USA

  47. The Model as One Rater Among Several: Measuring Political Positions in Data-Sparse Regions with a Language-Model Panel

    Jun 22, 2026Tarek GaraInter-Rater ReliabilityLLM-Assisted Annotation

    *Independent researcher.

  48. Large Language Model-Assisted Cleaning of Report-Derived Labels in a Large-Scale Chest CT Dataset

    Jun 21, 2026Yosuke Yamagishi, Atsushi Takamatsu, Mototsugu Sato +4Training Data CurationLLM-Assisted Annotation

    Division of Radiology and Biomedical Engineering, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan · Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, Tokyo, Japan · Department of Radiology, Kanazawa University Hospital, Ishikawa, Japan +3

  49. Scaling Performance and Low-Resource Annotation with Many-Shot In-Context Learning for Named Entity Recognition

    Jun 20, 2026Qi Zhang, Fangping Lan, Cornelia Caragea +2Named Entity RecognitionLLM-Assisted Annotation

    Temple University · University of Illinois Chicago

  50. CNnotator: LLM-Guided Memory Safety Annotation Synthesis

    Jun 20, 2026Twain Byrnes, Mike DoddsLLM-Assisted AnnotationProgram Synthesis

    Carnegie Mellon University Pittsburgh, PA, USA · Galois, Inc. Portland, OR, USA

  51. TACO: Task-Aware Column Description Generation Using LLMs

    Jun 19, 2026Ting Cai, Rakesh R. Menon, Yiru Chen +8LLM-Assisted Annotation

    University of Wisconsin-Madison · Adobe · Purdue University

  52. What sentiment analysis can't see: Measuring whether customers were helped, and what went wrong, across 70,000 support conversations

    Jun 18, 2026Jason PotteigerLLM-Assisted AnnotationSentiment Analysis

    Dimension Labs

  53. LLM-Assisted Stance Detection in Scientific Discourse: A Test Case in Bayesian Cognitive Science

    Jun 14, 2026Eyup Engin Kucuk, Tarik Kelestemur, Ömer Dağlar TanrikuluStance DetectionLLM-Assisted Annotation

    University of New Hampshire · Department of Psychology · Independent Researcher

  54. LLM-Based Synthetic Ground Truth Generation for Audio-Based Emotion Classification via In-Context Learning

    Jun 10, 2026Qing Huang, Pooja Pol, Jianing ZhangLLM-Assisted AnnotationEmotion Recognition in Conversations

    School of Business, Technical University of Applied Sciences Augsburg, Friedberger Str. 4, 86161 Augsburg, Germany · Data Science und Autonome Systeme Technologietransferzentrum (TTZ), An der Schmiede 19, 86899 Landsberg am Lech, Germany

  55. Extracting Semantics: LLM-Guided Automatic Population of Robot Ontology from URDF

    Jun 10, 2026Bastien Dussard, Guillaume SarthouLLM-Assisted AnnotationRobotics

    LAAS-RIS, LAAS · LAAS-CNRS, Department of Robotics, Toulouse, France

  56. Measuring Human Value Expression in Social Media Texts: Calibrated LLM Annotation and Encoder Transfer

    Jun 9, 2026Maria Milkova, Maksim RudnevSoft-Label LearningLLM-Assisted Annotation

    Independent researcher, Lisbon, Portugal · University of Waterloo, ON, Canada

  57. Scribby: A Multi-Level LLM Framework for Semantic Video Analysis

    Jun 8, 2026Julian Abelarde, Hugo Garrido-Lestache BelinchonVideo SummarizationLLM-Assisted Annotation

    Department of Computer Science and Software Engineering Milwaukee School of Engineering Milwaukee, WI US

  58. When Better Codebooks Are Not Enough: Predictive Performance and Behavioral Reliability in LLM Political Event Coding

    Jun 4, 2026Zixian He, Bharath Raahul Murugesan, Patrick Brandt +1LLM EvaluationLLM-Assisted Annotation

    Independent Researcher · Illinois Institute of Technology · The University of Texas at Dallas

  59. Does Topic Sentiment Cause Perceived Ideology? Comparing Human and LLM Annotations in Political News Articles

    Jun 4, 2026Upasana ChatterjeeLLM-Assisted AnnotationPolitical Bias in Language Models

    Department of Computer Science Columbia University

  60. From Self to Other: Evaluating Demographic Perspective-Taking in LLM Hate Speech Annotation

    Jun 4, 2026Paloma Piot, Javier ParaparLLM-Assisted AnnotationHate Speech Detection

    Information Retrieval Lab CITIC Research Centre Universidade da Coruña