Human in the Loop

Human-in-the-loop (HITL) systems integrate human expertise with artificial intelligence to improve the performance, robustness, and ethical considerations of AI systems. Current research focuses on optimizing HITL interactions across diverse applications, including autonomous driving, medical image analysis, and robotics, often employing techniques like active learning, reinforcement learning, and large language models to guide human input and improve model efficiency. The significance of HITL lies in its ability to address limitations of fully automated systems, particularly in complex or ambiguous tasks, leading to more accurate, reliable, and ethically sound AI solutions across various scientific and practical domains.

Papers

October 7, 2022

Advice Conformance Verification by Reinforcement Learning agents for Human-in-the-Loop
Mudit Verma, Ayush Kharkwal, Subbarao Kambhampati
Reinforcement Learning Reinforcement Learning Agent Human in the Loop Multi Agent Sequential Decision Advice Quality Conformance Checking

October 6, 2022

September 28, 2022

Human-in-the-loop Robotic Grasping using BERT Scene Representation
Yaoxian Song, Penglei Sun, Pengfei Fang, Linyi Yang, Yanghua Xiao, Yue Zhang
Scene Graph Vision Based Robotic Grasping Human in the Loop Open World Grasping

September 16, 2022

Comprehensively identifying Long Covid articles with human-in-the-loop machine learning
Robert Leaman, Rezarta Islamaj, Alexis Allot, Qingyu Chen, W. John Wilbur, Zhiyong Lu
Human in the Loop Post COVID 19 Long COVID

August 10, 2022

D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias
Bhavya Ghai, Klaus Mueller
Human in the Loop Algorithmic Bias Length Bias Bias Related Issue

July 14, 2022

Instance Selection Mechanisms for Human-in-the-Loop Systems in Few-Shot Learning
Johannes Jakubik, Benedikt Blumenstiel, Michael Vössing, Patrick Hemmer
Machine Learning LeArning Abstract Shot Learning Human in the Loop Instance Selection Shot Learning Evaluation

July 11, 2022

Increased Complexity of a Human-Robot Collaborative Task May Increase the Need for a Socially Competent Robot
Rebeka Kropivšek Leskovar, Tadej Petrič
Complexity Matter Human Robot Collaboration Social Robot Human in the Loop Community Need Robot Control Leader Follower Human Collaboration Human Robot Collaborative Task

June 16, 2022

Balancing Cost and Quality: An Exploration of Human-in-the-loop Frameworks for Automated Short Answer Scoring
Hiroaki Funayama, Tasuku Sato, Yuichiroh Matsubayashi, Tomoya Mizumoto, Jun Suzuki, Kentaro Inui
Environment Exploration Quality Issue Human in the Loop Short Answer Balancing Market

June 14, 2022

An Intelligent Assistant for Converting City Requirements to Formal Specification
Zirong Chen, Isaac Li, Haoxiang Zhang, Sarah Preum, John Stankovic, Meiyi Ma
Human in the Loop Smart City Intelligent Assistant Formal Specification Software Requirement

June 13, 2022

Efficient Human-in-the-loop System for Guiding DNNs Attention
Yi He, Xi Yang, Chia-Ming Chang, Haoran Xie, Takeo Igarashi
Deep Learning Human in the Loop Pixel Level Annotation Dataset Bias Image Classification Task

June 1, 2022

Natural Language Sentence Generation from API Specifications
Siyu Huo, Kushal Mukherjee, Jayachandu Bandlamudi, Vatche Isahagian, Vinod Muthusamy, Yara Rizk
Chatbot Response Intent Detection Human in the Loop Natural Sounding Speech Sentence Generation Application Programming Interface Documentation

May 27, 2022

Personalized Algorithmic Recourse with Preference Elicitation
Giovanni De Toni, Paolo Viappiani, Stefano Teso, Bruno Lepri, Andrea Passerini
Monte Carlo Tree Search Reinforcement Learning Agent Human in the Loop Algorithmic Recourse Preference Elicitation

May 23, 2022

Human-in-the-loop: Provably Efficient Preference-based Reinforcement Learning with General Function Approximation
Xiaoyu Chen, Han Zhong, Zhuoran Yang, Zhaoran Wang, Liwei Wang
Optimal Policy Human in the Loop Preference Based Reinforcement Learning Human in the Loop Reinforcement Trajectory Preference

May 21, 2022

Shared-Control Robotic Manipulation in Virtual Reality
Shiyu Xu, Scott Moore, Akansel Cosgun
Robotic Manipulation Robotic Manipulator Human in the Loop End Effector Virtual Reality Teleoperation

May 18, 2022

A Scalable Workflow to Build Machine Learning Classifiers with Clinician-in-the-Loop to Identify Patients in Specific Diseases
Jingqing Zhang, Atri Sharma, Luis Bolanos, Tong Li, Ashwani Tanwar, Vibhor Gupta, Yike Guo
Electronic Health Record Common Disease Human in the Loop Structured Data Patient Friendly Language Machine Learning Classifier Unstructured Clinical International Classification of Disease

April 29, 2022

Human-in-the-loop online multi-agent approach to increase trustworthiness in ML models through trust scores and data augmentation
Gusseppe Bravo-Rocca, Peini Liu, Jordi Guitart, Ajay Dholakia, David Ellison, Miroslav Hodak
Machine Learning Data Augmentation Machine Learning Model Multi Agent System Human in the Loop Robust AI Human Like Agent

April 13, 2022

Rapid model transfer for medical image segmentation via iterative human-in-the-loop update: from labelled public to unlabelled clinical datasets for multi-organ segmentation in CT
Wenao Ma, Shuang Zheng, Lei Zhang, Huimao Zhang, Qi Dou
Medical Image Segmentation Segmentation Model Large Scale Dataset Human in the Loop Multi Organ Segmentation Clinical Datasets Model Transfer

April 11, 2022

Correcting Robot Plans with Natural Language Feedback
Pratyusha Sharma, Balakumar Sundaralingam, Valts Blukis, Chris Paxton, Tucker Hermans, Antonio Torralba, Jacob Andreas, Dieter Fox
Human in the Loop Robot Motion Goal Specification Robot Error

March 29, 2022

OdontoAI: A human-in-the-loop labeled data set and an online platform to boost research on dental panoramic radiographs
Bernardo Silva, Laís Pinheiro, Brenda Sobrinho, Fernanda Lima, Bruna Sobrinho, Kalyf Abdalla, Matheus Pithon, Patrícia Cury, Luciano Oliveira
DH Research Human in the Loop OpenAI Codex Panoramic X Ray Online Platform Panoramic Radiograph

Human in the Loop

Papers

Advice Conformance Verification by Reinforcement Learning agents for Human-in-the-Loop

KnowledgeShovel: An AI-in-the-Loop Document Annotation System for Scientific Knowledge Base Construction

Learning functional sections in medical conversations: iterative pseudo-labeling and human-in-the-loop approach

Human-in-the-loop Robotic Grasping using BERT Scene Representation

Comprehensively identifying Long Covid articles with human-in-the-loop machine learning

D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias

Instance Selection Mechanisms for Human-in-the-Loop Systems in Few-Shot Learning

Increased Complexity of a Human-Robot Collaborative Task May Increase the Need for a Socially Competent Robot

Balancing Cost and Quality: An Exploration of Human-in-the-loop Frameworks for Automated Short Answer Scoring

An Intelligent Assistant for Converting City Requirements to Formal Specification

Efficient Human-in-the-loop System for Guiding DNNs Attention

Natural Language Sentence Generation from API Specifications

Personalized Algorithmic Recourse with Preference Elicitation

Human-in-the-loop: Provably Efficient Preference-based Reinforcement Learning with General Function Approximation

Shared-Control Robotic Manipulation in Virtual Reality

A Scalable Workflow to Build Machine Learning Classifiers with Clinician-in-the-Loop to Identify Patients in Specific Diseases

Human-in-the-loop online multi-agent approach to increase trustworthiness in ML models through trust scores and data augmentation

Rapid model transfer for medical image segmentation via iterative human-in-the-loop update: from labelled public to unlabelled clinical datasets for multi-organ segmentation in CT

Correcting Robot Plans with Natural Language Feedback

OdontoAI: A human-in-the-loop labeled data set and an online platform to boost research on dental panoramic radiographs