Human-in-the-Loop AI

Ask about this topic

Overview

Human-in-the-loop AI studies how people can teach, correct, guide, or oversee AI systems, from labeling examples in active learning to providing preferences in reinforcement learning and reviewing agent decisions. Common approaches include interactive interfaces, learning to defer uncertain cases to experts, and using human demonstrations or corrections to adapt robots and other systems during deployment. Applications range from medical decision support and education to robotics, scientific research, and software engineering, evaluated through benchmarks, user studies, and real-world trials. A central tension is how to make human input effective without creating excessive workload or encouraging overreliance, while preserving people’s ability to understand, challenge, and override AI decisions.

Latest papers

CardsList

290 papers30 papers in the last four weeks, down 9% on the four weeks before. 0.2% of all new papers.

  1. Closing the Auto-Research Loop: An AI Co-Scientist for Production Search Ranking

    Mar 23, 2026Liwei Wu, Cho-Jui HsiehAutonomous Research AgentsLearning to Rank

    Trip.com Group · UCLA

  2. NuHF Claw: A Risk Constrained Cognitive Agent Framework for Human Centered Procedure Support in Digital Nuclear Control Rooms

    Mar 23, 2026Xingyu Xiao, Jiejuan Tong, Jun Sun +4AI Agent SafetyHuman-in-the-Loop AI

    Institute of Nuclear and New Energy Technology, Tsinghua University, Beijing, 100084, China · National Key Laboratory of Human Factors Engineering, Beijing, 100094, China · Institute of Software, Chinese Academy of Sciences, Beijing, 100045, China

  3. Design-OS: A Specification-Driven Framework for Engineering System Design with a Control-Systems Design Case

    Mar 20, 2026H. Sinan Bank, Daniel R. Herber, Thomas H. BradleyAgentic WorkflowsHuman-in-the-Loop AI

    Department of Systems Engineering Colorado State University Fort Collins, CO 80523

  4. Demonstration of Adapt4Me: An Uncertainty-Aware Authoring Environment for Personalizing Automatic Speech Recognition to Non-normative Speech

    Mar 20, 2026Niclas Pokel, Yiming Zhao, Pehuén Moure +2Personalized Speech RecognitionAutomatic Speech Recognition

    Institute of Neuroinformatics, University of Zurich and ETH Zurich · Department of Computer Science, ETH Zurich · Department of Computational Linguistics, University of Zurich

  5. Orchestrating Human-AI Software Delivery: A Retrospective Longitudinal Field Study of Three Software Modernization Programs

    Mar 20, 2026Maximiliano Armesto, Christophe KolbSoftware EngineeringHuman-AI Collaboration

    Taller Technologies

  6. kRAIG: A Natural Language-Driven Agent for Automated DataOps Pipeline Generation

    Mar 19, 2026Rohan Siva, Kai Cheung, Lichi Li +1Workflow AutomationTool-Augmented Language Model Agents

    Cisco

  7. A Human-in/on-the-Loop Framework for Accessible Text Generation

    Mar 19, 2026Lourdes Moreno, Paloma MartínezText SimplificationHuman-in-the-Loop AI

    Universidad Carlos III de Madrid Av. Universidad, 30, Leganés, 28911, Spain

  8. Competing with AI Scientists: Agent-Driven Approach to Astrophysics Research

    Mar 18, 2026Thomas Borrett, Licong Xu, Andy Nilipour +7Scientific Workflow AutomationAutonomous Research Agents

    Cavendish Astrophysics, University of Cambridge, Madingley Road, Cambridge CB3 0HA, UK · Kavli Institute for Cosmology, University of Cambridge, Madingley Road, Cambridge CB3 0HA, UK · Institute of Astronomy, University of Cambridge, Madingley Road, Cambridge CB3 0HA, UK +4

  9. Deployment and Evaluation of an EHR-integrated, Large Language Model-Powered Tool to Triage Surgical Patients

    Mar 18, 2026Jane Wang, Timothy Keyes, April S Liang +10Clinical TriageClinical Decision Support

    Division of Hospital Medicine, Department of Medicine, Stanford University, Palo Alto, CA · Stanford University School of Medicine, Palo Alto, CA · Stanford Health Care, Technology and Digital Solutions, Palo Alto, CA +2

  10. Intent Formalization: A Grand Challenge for Reliable Coding in the Age of AI Agents

    Mar 17, 2026Shuvendu K. LahiriAI Agent ReliabilityAutoformalization

    Microsoft Research, USA

  11. Proactive Rejection and Grounded Execution: A Dual-Stage Intent Analysis Paradigm for Safe and Efficient AIoT Smart Homes

    Mar 17, 2026Xinxin Jin, Zhengwei Ni, Zhengguo Sheng +1Language Model-Based ControlAI Agent Reliability

    School of Information and Electronic Engineering (Sussex Artificial Intelligence Institute), Zhejiang Gongshang University, Hangzhou 310018, China. · Sussex Artificial Intelligent Institute, Zhejiang Gongshang University, Hangzhou 310018, China · Department of Engineering and Design, University of Sussex, BN1 9RH, Brighton, U.K. +3

  12. Lean Atlas: An Integrated Proof Environment for Scalable Human-AI Collaborative Formalization

    Mar 16, 2026Banri Yanahama, Akiyoshi SannaiFormal VerificationLean Theorem Proving

    Nyx Foundation · Kyoto University · Large Language Model Research and Development Center, National Institute of Informatics

  13. Bridging the Experimental Last Mile: Digitizing Laboratory Know-How for Safe AI-Assisted Support

    Mar 16, 2026Akira Miura, Yuki Sasahara, Momoka Demura +3AI for ScienceHuman-in-the-Loop AI

    Division of Applied Chemistry, Faculty of Engineering, Hokkaido University, N13W8 Sapporo 060-8628 Japan · Graduate School of Chemical Sciences and Engineering, Hokkaido University, N13W8 Sapporo, 060-8628 Japan · Graduate School of Information Science and Technology, Hokkaido University, N13W8 Sapporo 060-8628 Japan +1

  14. When the Loop Closes: Architectural Limits of In-Context Isolation, Metacognitive Co-option, and the Two-Target Design Problem in Human-LLM Systems

    Mar 14, 2026Z. Cheng, N. SongLLM Context ManagementHuman-in-the-Loop AI

    Independent Researcher

  15. Easy-IIL: Reducing Human Operational Burden in Interactive Imitation Learning via an Assistant Expert

    Mar 13, 2026Chengjie Zhang, Chao Tang, Wenlong Dong +3Imitation LearningInteractive Imitation Learning

    Shenzhen Key Laboratory of Robotics and Computer Vision, Southern University of Science and Technology, Shenzhen, China. · Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China. · Department of Robotics, Perception, and Learning, KTH Royal Institute of Technology, Stockholm, Sweden.

  16. Using a Human-AI Teaming Approach to Create and Curate Scientific Datasets with the SCILIRE System

    Mar 13, 2026Necva Bölücü, Jessica Irons, Changhyun Lee +5LLM Information ExtractionScientific Workflow Automation

    CSIRO, Sydney, Australia · ITIS, University of Málaga, Málaga, Spain

  17. Human-AI Collaborative Autonomous Experimentation With Proxy Modeling for Comparative Observation

    Mar 13, 2026Arpan Biswas, Hiroshi Funakubo, Yongtao LiuAutonomous ExperimentationBayesian Optimization

    University of Tennessee-Oak Ridge Innovation Institute, University of Tennessee, Knoxville, TN 37996, USA · Department of Material Science and Engineering, Institute of Science Tokyo, Yokohama, 226-8502, Japan · Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, Oak Ridge, TN 37830, USA

  18. From Control to Foresight: Simulation as a New Paradigm for Human-Agent Collaboration

    Mar 12, 2026Gaole He, Brian Y. LimHuman-AI CollaborationHuman-in-the-Loop AI

    National University of Singapore, Singapore

  19. Agentic Control Center for Data Product Optimization

    Mar 10, 2026Priyadarshini Tamilselvan, Gregory Bramble, Sola Shirai +3Data Analysis AgentsHuman-in-the-Loop AI

    Georgia Tech · IBM Research

  20. A prospective clinical feasibility study of a conversational diagnostic AI in an ambulatory primary care clinic

    Mar 9, 2026Peter Brodeur, Jacob M. Koshy, Anil Palepu +45HealthcareMedical LLMs

  21. Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learning

    Mar 9, 2026Wei Yang, Defu Cao, Jiacheng Pang +2Human-in-the-Loop AIMulti-Agent LLM Systems

    University of Southern California

  22. AI Agents, Language, Deep Learning and the Next Revolution in Science

    Mar 9, 2026Ke Li, Beijiang Liu, Bruce Mellado +2AI Agents for Scientific DiscoveryHuman-in-the-Loop AI

    Institute of High Energy Physics, Chinese Academy of Sciences, Beijing 100049, China · University of Chinese Academy of Sciences, Beijing 100049, China · School of Physics and Institute for Collider Particle Physics, University of the Witwatersrand, Johannesburg, Wits 2050, South Africa

  23. HLER: Human-in-the-Loop Economic Research via Multi-Agent Pipelines for Empirical Discovery

    Mar 8, 2026Chen Zhu, Xiaolu WangMulti-Agent LLM SystemsHuman-in-the-Loop AI

    China Agricultural University

  24. Solving Jigsaw Puzzles in the Wild: Human-Guided Reconstruction of Cultural Heritage Fragments

    Mar 6, 2026Omidreza Safaei, Sinem Aslan, Sebastiano Vascon +3Jigsaw Puzzle SolvingHuman-in-the-Loop AI

    DAIS, Ca’ Foscari University of Venice, Venice, Italy · Department of Historical Studies, University of Milan, Milan, Italy · Zhejiang Normal University, Jinhua, Zhejiang Province, China

  25. An Interactive LLM-Based Simulator for Dementia-Related Activities of Daily Living

    Mar 6, 2026Kruthika Gangaraju, Shu-Fen Wung, Kevin Berner +2Dementia CareLarge Language Model-Based Role-Play Simulation

    Worcester Polytechnic Institute Worcester, Massachusetts, USA · University of California Davis Sacramento, CA, USA · MGH Institute of Health Professionals Boston, MA, USA +1

  26. Towards automated data analysis: A guided framework for LLM-based risk estimation

    Mar 4, 2026Panteleimon RodisAI Risk ManagementData Analysis Agents

  27. Delegation and Verification Under AI

    Mar 3, 2026Lingxiao Huang, Wenyang Xiao, Nisheeth K. VishnoiHuman-in-the-Loop AIHuman-AI Decision Making

    Nanjing University · Yale University

  28. Exploring Plan Space through Conversation: An Agentic Framework for LLM-Mediated Explanations in Planning

    Mar 2, 2026Guilhem Fouilhé, Rebecca Eifler, Antonin Poché +2Multi-Agent LLM SystemsExplainable Artificial Intelligence

    IRIT, Toulouse, France · LAAS-CNRS, Toulouse, France · IRT Saint Exupery, Toulouse, France +2

  29. Rationale Behind Human-Led Autonomous Truck Platooning

    Mar 1, 2026Yukun Lu, Chenzhao Li, Xintong Jiang +1Autonomous DrivingHuman-in-the-Loop AI

    Dept. of Mechanical Engineering, University of New Brunswick, Fredericton, NB E3B 5A3, Canada · Faculty of Interdisciplinary Studies, University of New Brunswick, Fredericton, NB E3B 5A3, Canada

  30. MOSAIC: A Universal Agent-Level Interface for Cross-Paradigm Agent Mixing and Human-AI Collaboration

    Mar 1, 2026Abdulhamid M. Mousa, Jinhui Pang, Rakhmonberdi Khajiev +5AI Agent EvaluationAgent Evaluation

    School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China · School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China · Mathematics Department, Faculty of Science, Ain Shams University, Cairo, Egypt +1