Active Learning

Follow this topic Ask about this topic

Active learning studies how to choose data or queries so models can learn effectively with fewer labels, measurements, or experiments. Common approaches include uncertainty sampling, diversity and coreset selection, Bayesian information gain, batch strategies, and cost-aware or human-in-the-loop querying; scientific applications also use Gaussian processes, multi-fidelity models, and physics-based acquisition to guide simulations and laboratory work. Work spans image and language benchmarks, medical annotation, robotics, and scientific discovery, with newer methods using foundation-model representations and adapting selection to class imbalance, domain shift, or label noise. A central open tension is that no acquisition strategy wins everywhere: gains depend on the task, budget, model and data, and random sampling can match or outperform sophisticated methods, making realistic evaluations and strong baselines essential.

Latest papers

296 papers39 papers in the last four weeks, up 22% on the four weeks before. 0.2% of all new papers.

Beyond Labels: Information-Efficient Human-in-the-Loop Learning using Ranking and Selection Queries
Feb 17, 2026Belén Martín-Urcelay, Yoonsang Lee, Matthieu R. Bloch +1Human-in-the-Loop Annotation Active Learning
School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332 USA.
Distilling LLM Reasoning into Graph of Concept Predictors
Feb 3, 2026Ziyang Yu, Liang ZhaoCoT Distillation Active Learning
Department of Computer Science Emory University Atlanta, USA
Correcting Boundary Bias and Observation Independence in Bayesian Experimental Design
Feb 2, 2026Sanna Jarl, Jens Sjölund, Jonathan J. S. Scragg +1Active Learning Adaptive Sampling
Department of Material Science & Engineering Uppsala University RISE · Department of Information Technology Uppsala University · RISE
Towards Optimality in Active Preference Learning
Feb 2, 2026Yao Zhao, Kwang-Sung JunActive Preference Learning Active Learning
Department of Computer Science, University of Arizona, Tucson, USA.
Active Learning under Simulated Class Imbalance and Label Shift in Binary Semantic Segmentation
Jan 8, 2026Julien Combes, Jean-François Coeurjolly, Alexandre DervilleActive Learning Label Shift
SVH, Michelin · SVH · Michelin
Active learning for data-driven reduced models of parametric differential systems with Bayesian operator inference
Dec 30, 2025Shane A. McQuarrie, Mengwu Guo, Anirban ChaudhuriActive Learning Partial Differential Equations
Department of Mathematics, Brigham Young University · Centre for Mathematical Sciences, Lund University · Oden Institute for Computational Engineering and Sciences, The University of Texas at Austin
Monte Carlo Query Search: Active Capability Assessment of AI Agents
Dec 18, 2025Daniel Bramblett, Rushang Karia, Adrian Ciotinga +3AI Agent Evaluation Active Learning
Arizona State University, AZ, USA · Indian Institute of Technology Madras, India
Active Learning with Imperfect Labels: Optimal Labeler Assignment and Sample Selection
Dec 14, 2025Pouya Ahadi, Blair Winograd, Camille Zaug +3Human-in-the-Loop Annotation Active Learning
School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332, USA · Ford Motor Company
Batch Bayesian Active Learning with Partial Batch Label Sampling
Oct 10, 2025Kangping Hu, Stephen MussmannActive Learning
Department of Computer Science, Georgia Institute of Technology, Atlanta, GA 30332
Large Language Model Selection with Limited Annotations
Oct 10, 2025Yavuz Durmazkeser, Patrik Okanovic, Andreas Kirsch +2Model Selection LLM Evaluation
TU Delft · ETH Zurich
Multidimensional Bayesian Active Machine Learning of Working Memory Task Performance
Oct 1, 2025Dom CP Marticorena, Chris Wissmann, Zeyu Lu +1Active Experimental Design Active Learning
Department of Biomedical Engineering, Washington University, 1 Brookings Drive, St. Louis, MO 63130 · Department of Computer Science and Engineering, Washington University, 1 Brookings Drive, St. Louis, MO 63130
CPATTA: Conformal Supervision Allocation For Active Test-Time Adaptation
Sep 30, 2025Tingyu Shi, Fan Lyu, Haihua Zhu +2Distribution Shift Robustness Test-Time Adaptation
Computer Science and Engineering, University of California San Diego · New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences · College of Computer Science and Electronic Engineering, Hunan University
Combining Discrepancy-Confusion Uncertainty and Calibration Diversity for Active Fine-Grained Image Classification
Sep 29, 2025Yinghao Jin, Xi YangActive Learning Fine-Grained Image Classification
Jilin University, China
Actively Learning Halfspaces without Synthetic Data
Sep 25, 2025Hadley Black, Kasper Green Larsen, Arya Mazumdar +2Query Complexity Boolean Function Learning
Baruch College, CUNY · Aarhus University · University of California, San Diego
Ultra Strong Machine Learning: LLM-Generated Explanations Do Not Yet Suffice for Teaching Humans Active Learning Strategy
Aug 31, 2025Lun Ai, Johannes Langer, Ute Schmid +1Active Learning Human-in-the-Loop AI
Department of Computing, Imperial College London, UK · Faculty Information Systems and Applied Computer Science, University of Bamberg, Germany
Why Pool When You Can Flow? Active Learning with GFlowNets
Aug 31, 2025Renfei Zhang, Mohit Pandey, Artem Cherkasov +1Active Learning Virtual Screening
School of Computer Science, Simon Fraser University, Burnaby, BC, Canada · Vancouver Prostate Centre, University of British Columbia, Vancouver, BC, Canada · Diagen AI +1
Interactive Multi-Objective Probabilistic Preference Learning with Soft and Hard Bounds
Jun 27, 2025Edward Chen, Sang T. Truong, Natalie Dullerud +2Active Learning Multi-Criteria Decision Analysis
Department of Computer Science, Stanford University, Stanford, California, USA
PCoreSet: Effective Active Learning through Knowledge Distillation from Vision-Language Models
Jun 1, 2025Seongjae Kang, Dong Bok Lee, Hyungjoon Jang +2Active Learning Coreset Selection
VUNO Inc. · KAIST
Investigating Active Sampling for Hardness Classification with Vision-Based Tactile Sensors
May 19, 2025Junyi Chen, Frederik Heller, Mario Gómez Andreu +7Active Learning Robotic Tactile Sensing
Intelligent Autonomous Systems Lab, Department of Computer Science, TU Darmstadt, Germany. · German Research Center for AI (DFKI) · Department of Psychology, University of Giessen, Germany +2
Near-Polynomially Competitive Active Logistic Regression
Mar 7, 2025Yihan Zhou, Eric Price, Trung NguyenActive Learning Logistic Regression
The University of Texas at Austin
Active Learning with Bayesian Multi-Fidelity Laplace Neural Operators for Oscillatory Parametric PDEs
Feb 1, 2025Bongseok Kim, Haoyang Zheng, Michael Penwarden +1Dynamical Systems Surrogate Modeling
School of Mechanical Engineering, Purdue University, 585 Purdue Mall, West Lafayette, 47907, IN, USA. · Department of Mathematics, Purdue University, 150 North University Street, West Lafayette, 47907, IN, USA.
Active Learning Enables Generation of Molecules that Advance the Known Pareto Front
Jan 3, 2025Evan R. Antoniuk, Peggy Li, Nathan Keilbart +3Molecular Optimization OOD Generalization
SegCol Challenge: Semantic Segmentation for Tools and Fold Edges in Colonoscopy data
Dec 20, 2024Xinwei Ju, Rema Daher, Razvan Caramalau +7Endoscopy Image Segmentation
UCL Hawkes Institute, University College London, London, UK · Department of Computer Science, University College London, London, UK · Medtronic, London, UK +3
A Utility-Mining-Driven Active Learning Approach for Analyzing Clickstream Sequences
Oct 9, 2024Danny Y. C. Wang, Lars Arne Jordanger, Jerry Chun-Wei LinActive Learning Conversion Rate Prediction
Department of C. E. M., Høgskulen på Vestlandet, Bergen, Norway
Human-In-The-Loop Machine Learning for Safe and Ethical Autonomous Vehicles: Principles, Challenges, and Opportunities
Aug 22, 2024Yousef Emami, Mohammadhossein Homaei, Miguel Gutiérrez Gaitán +4Active Learning Autonomous Driving
A Competitive Algorithm for Agnostic Active Learning
Oct 28, 2023Eric Price, Yihan ZhouActive Learning Agnostic Learning
Department of Computer Science, University of Texas at Austin

Feb 17, 2026J/K move · Enter open · S save

Beyond Labels: Information-Efficient Human-in-the-Loop Learning using Ranking and Selection Queries

Belén Martín-Urcelay, Yoonsang Lee, Matthieu R. Bloch, Christopher J. Rozell

School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332 USA.

Human-in-the-Loop Annotation Active Learning

Open paper AskPDF arXiv

Integrating human expertise into machine learning systems often reduces the role of experts to labeling oracles, a paradigm that limits the amount of information exchanged and fails to capture the nuances of human judgment. We address this challenge by developing a human-in-the-loop framework to learn binary classifiers with rich query types, consisting of item ranking and exemplar selection. We first introduce probabilistic human response models for these rich queries motivated by the relationship experimentally observed between the perceived implicit score of an item and its distance to the unknown classifier. Using these models, we then design active learning algorithms that leverage the rich queries to increase the information gained per interaction. We provide theoretical bounds on sample complexity and develop a tractable and computationally efficient variational approximation. Through experiments with simulated annotators derived from crowdsourced word-sentiment and image-aesthetic datasets, we demonstrate significant reductions on sample complexity. We further extend active learning strategies to select queries that maximize information rate, explicitly balancing informational value against annotation cost. This algorithm in the word sentiment classification task reduces learning time by more than 57% compared to traditional label-only active learning.