LLM-Assisted Annotation

Follow this topic Ask about this topic

Momentum

17 papers in the last four weeks, up 143% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 124

All topics

Addressing Labelled Data Scarcity: Taxonomy-Agnostic Annotation of PII Values in HTTP Traffic using LLMs
May 7, 2026Thomas Cory, Axel KüpperPrivacy Auditing LLM-Assisted Annotation
Technische Universität Berlin
Vidya: An AI-Driven Modular Pipeline for Archival Automation and Semantic Metadata Enrichment
May 7, 2026Cloter Migliorini Filho, Julia Graciela Machado, Edson Armando Silva +1LLM-Assisted Annotation LLM Information Extraction
1Museu Campos Gerais - Universidade Estadual de Ponta Grossa - UEPG · 2Universidade Tecnol´ogica Federal do Paran´a - UTFPR
Annotation Quality in Aspect-Based Sentiment Analysis: A Case Study Comparing Experts, Students, Crowdworkers, and Large Language Model
May 5, 2026Niklas Donhauser, Jakob Fehle, Nils Constantin Hellwig +3Annotator Disagreement LLM-Assisted Annotation
Media Informatics Group, University of Regensburg, Regensburg, Germany · Information Science Group, University of Regensburg, Regensburg, Germany
Stable Behavior, Limited Variation: Persona Validity in LLM Agents for Urban Sentiment Perception
Apr 30, 2026Neemias B da Silva, Rodrigo Minetto, Daniel Silver +1VLM Evaluation Personality Modeling in Language Models
Universidade Tecnol´ogica Federal do Paran´a (UTFPR), Curitiba, Brazil · University of Toronto, Toronto, Canada
LLMs Capture Emotion Labels, Not Emotion Uncertainty: Distributional Analysis and Calibration of Human-LLM Judgment Gaps
Apr 30, 2026Keito Inoshita, Xiaokang Zhou, Akira Kawai +1LLM Evaluation Annotator Disagreement
Faculty of Business and Commerce, Kansai University, Osaka, Japan · Faculty of Business Data Science, Kansai University, Osaka, Japan · RIKEN Center for Advanced Intelligence Project, Tokyo, Japan +2
DenseStep2M: A Scalable, Training-Free Pipeline for Dense Instructional Video Annotation
Apr 29, 2026Mingji Ge, Qirui Chen, Zeqian Li +1Video Understanding LLM-Assisted Annotation
SAI, Shanghai Jiao Tong University · Shanghai, China
Naamah: A Large Scale Synthetic Sanskrit NER Corpus via DBpedia Seeding and LLM Generation
Apr 29, 2026Akhil Rajeev P, Annarao KulkarniNamed Entity Recognition Synthetic Data Generation
Centre for Development of Advanced Computing (C-DAC), Bangalore
Scalable LLM-based Coding of Dialogue in Healthcare Simulation: Balancing Coding Performance, Processing Time, and Environmental Impact
Apr 25, 2026Kiyoshige Garces, Gloria Milena Fernandez-Nieto, Linxuan Zhao +4LLM Inference Efficiency LLM Prompting
RMIT University · Monash University · Adelaide University +2
LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments
Apr 24, 2026Xiomara Gonzalez, Gabriella Coloyan Fleming, Andrew Katz +2LLM-Assisted Annotation Sentiment Analysis
Chandra Family Department of Electrical and Computer Engineering, University of Texas at Austin · Department of Engineering Education, Virginia Polytechnic Institute and State University · Gallogly College of Engineering, University of Oklahoma +1
SketchVLM: Vision language models can annotate images to explain thoughts and guide users
Apr 23, 2026Brandon Collins, Logan Bolton, Hung Huy Nguyen +3Vision-Language Models LLM-Assisted Annotation
Auburn University · Adobe Research
'The Order in the Horse's Heart': A Case Study in LLM-Assisted Stylometry for the Discovery of Biblical Allusion in Modern Literary Fiction
Apr 21, 2026Ewan CameronComputational Literary Studies LLM-Assisted Annotation
Actuarial Mathematics & Statistics, School of Mathematical & Computer Sciences, Heriot-Watt University, Edinburgh EH14 4AS, UK
From Fallback to Frontline: When Can LLMs be Superior Annotators of Human Perspectives?
Apr 20, 2026Hasan Amin, Harry Yizhou Tian, Xiaoni Duan +3LLM Evaluation LLM-Assisted Annotation
Purdue University · Washington University in St. Louis
AutoVQA-G: Self-Improving Agentic Framework for Automated Visual Question Answering and Grounding Annotation
Apr 19, 2026Rongsheng Hu, Runwei Guan, Yicheng Di +2Visual Question Answering LLM-Assisted Annotation
School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China · Thrust of Artificial Intelligence, HKUST(GZ), Guangzhou, China
Can LLMs Understand the Impact of Trauma? Costs and Benefits of LLMs Coding the Interviews of Firearm Violence Survivors
Apr 17, 2026Jessica H. Zhu, Shayla Stringfield, Vahe Zaprosyan +3LLM-Assisted Annotation
University of Maryland, College Park
DiZiNER: Disagreement-guided Instruction Refinement via Pilot Annotation Simulation for Zero-shot Named Entity Recognition
Apr 17, 2026Siun Kim, Hyung-Jin YoonNamed Entity Recognition LLM-Assisted Annotation
Seltasquare Seoul, Korea · Biomedical Research Institute, Seoul National University Hospital · Seoul National University College of Medicine Seoul, Korea
MetaDent: Labeling Clinical Images for Vision-Language Models in Dentistry
Apr 16, 2026Meng-Xun Li, Wen-Hui Deng, Zhi-Xing Wu +6VLM Evaluation Medical Image Analysis
Department of Prosthodontics, State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine, Ministry of Education, Hubei Key Laboratory of Stomatology, School and Hospital of Stomatology, Wuhan University, Wuhan, Hubei, China · School of Artificial Intelligence, Wuhan University, Wuhan, Hubei, China · School of Computer Science, Wuhan University, Wuhan, Hubei, China +2
Exploring and Testing Skill-Based Behavioral Profile Annotation: Human Operability and LLM Feasibility under Schema-Guided Execution
Apr 16, 2026Yufeng WuLLM-Assisted Annotation Human-in-the-Loop Annotation
City University of Hong Kong
Generative Augmented Inference
Apr 16, 2026Cheng Lu, Mengxin Wang, Dennis J. Zhang +1LLM-Assisted Annotation Causal Inference
Olin Business School, Washington University in St. Louis, USA · Naveen Jindal School of Management, University of Texas at Dallas, USA · W. P. Carey School of Business, Arizona State University, USA
Do We Still Need Humans in the Loop? Human vs. LLM Annotation in Active Learning for TikTok Hate Speech Detection
Apr 15, 2026Ahmad Dawar Hakimi, Lea Hirlimann, Isabelle Augenstein +1LLM-Assisted Annotation Human-in-the-Loop Annotation
Center for Information and Language Processing, LMU Munich, Germany · Department of Computer Science, University of Copenhagen, Denmark · Munich Center for Machine Learning, Germany
ScheMatiQ: From Research Question to Structured Data through Interactive Schema Discovery
Apr 10, 2026Shahar Levy, Eliya Habba, Reshef Mintz +3LLM-Assisted Annotation Document Information Extraction
School of Computer Science and Engineering, The Hebrew University of Jerusalem · Faculty of Law, The Hebrew University of Jerusalem · Allen Institute for AI
When Hate Meets Facts: LLMs-in-the-Loop for Check-worthiness Detection in Hate Speech
Mar 26, 2026Nicolás Benjamín Ocampo, Tommaso Caselli, Davide CeolinLLM-Assisted Annotation Hate Speech Detection
Centrum Wiskunde & Informatica · Amsterdam, The Netherlands · University of Groningen +1
From Content to Audience: A Multimodal Annotation Framework for Broadcast Television Analytics
Mar 24, 2026Paolo Cupini, Francesco PierriLLM-Assisted Annotation Multimodal Large Language Models
Politecnico di Milano
Multi-Perspective LLM Annotations for Valid Analyses in Subjective Tasks
Mar 22, 2026Navya Mehrotra, Adam Visokay, Kristina GligorićAnnotator Disagreement LLM-Assisted Annotation
University of Washington
Is Human Annotation Necessary? Iterative MBR Distillation for Error Span Detection in Machine Translation
Mar 13, 2026Boxuan Lyu, Haiyue Song, Zhi QuMBR Decoding LLM-Assisted Annotation
Institute of Science Tokyo · National Institute of Information and Communications Technology, Japan
AuditoryHuM: Auditory Scene Label Generation and Clustering using Human-MLLM Collaboration
Feb 23, 2026Henry Zhong, Jörg M. Buchholz, Julian Maclaren +2Clustering LLM-Assisted Annotation
Australian Hearing Hub, Macquarie University, Sydney, Australia · Google Research Australia, Sydney, Australia
A Large-Scale Dataset for Molecular Structure-Language Description via a Rule-Regularized Method
Feb 2, 2026Feiyang Cai, Guijuan He, Yi Hu +7LLM-Assisted Annotation
Clemson University · Independent Researcher · University of Delaware +1
Standardizing Longitudinal Radiology Report Evaluation via Large Language Model Annotation
Jan 23, 2026Xinyi Wang, Grazziela Figueredo, Ruizhe Li +1Medical Report Generation LLM-Assisted Annotation
School of Computer Science, The University of Nottingham, Nottingham, NG7 2RD, United Kingdom. · School of Medicine, The University of Nottingham, Nottingham, NG7 2RD, United Kingdom.
Interactive In-Meeting Speaker Correction with Human Feedback
Sep 22, 2025Xinlu He, Yiwen Guan, Badrivishal Paurana +3LLM-Assisted Annotation Human-in-the-Loop Annotation
Worcester Polytechnic Institute
LLMs and their Limited Theory of Mind: Evaluating Mental State Annotations in Situated Dialogue
Sep 2, 2025Katharine Kowalyshyn, Matthias ScheutzLLM Evaluation Theory of Mind
Department of Computer Science Tufts University
Introducing HALC: A general pipeline for the systematic and reliable construction of prompts for automated coding with LLMs in the computational social sciences
Jul 29, 2025Andreas Reich, Claudia Thoms, Tobias SchrimpfLLM Prompting LLM-Assisted Annotation

May 7, 2026, cs.AIJ/K move · Enter open · S save

Addressing Labelled Data Scarcity: Taxonomy-Agnostic Annotation of PII Values in HTTP Traffic using LLMs

Thomas Cory, Axel Küpper

Technische Universität Berlin

Privacy Auditing LLM-Assisted Annotation LLM Information Extraction PII Detection

Open paper AskPDF arXiv

Automated privacy audits of web and mobile applications often analyse outbound HTTP traffic to detect Personally Identifiable Information (PII) leakage. However, existing learning-based detectors typically depend on scarce, manually labelled traffic and are tightly coupled to fixed label taxonomies, limiting transferability across domains and evolving definitions of PII. This paper investigates whether Large Language Models (LLMs) can support taxonomy-agnostic annotation of explicitly transmitted PII values in HTTP message bodies when the taxonomy is provided at runtime. We introduce a multi-stage LLM-based pipeline that combines deterministic pre-processing with label-level classification, targeted instance-level value annotation, and output validation. To enable controlled evaluation and exemplar-based prompting without relying on sensitive real-user captures, we further propose an LLM-based generator for synthetic HTTP traffic with manually validated, taxonomy-derived PII annotations. We evaluate the approach across three taxonomies spanning different PII domains and granularity levels. Results show that the pipeline accurately detects PII types and extracts corresponding values for concrete PII taxonomies. Overall, our findings position LLMs as a promising foundation for flexible, taxonomy-agnostic traffic annotation and for creating labelled data under evolving privacy taxonomies.

Momentum

Related topics

Latest papers 124