cs.ROMar 24, 2026

Human-in-the-Loop Pareto Optimization: Trade-off Characterization for Assist-as-Needed Training and Performance Evaluation

Authors: Harun Tolasa, Volkan Patoglu

Organizations: the Faculty of Engineering and Natural Sciences at Sabanci University, Istanbul, Turkiye.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

During human motor skill training and physical rehabilitation, there is an inherent trade-off between task difficulty and user performance. Characterizing this trade-off is crucial for evaluating user performance, designing assist-as-needed (AAN) protocols, and assessing the efficacy of training protocols. In this study, we propose a novel human-in-the-loop (HiL) Pareto optimization approach to characterize the trade-off between task performance and the perceived challenge level of motor learning or rehabilitation tasks. We adapt Bayesian multi-criteria optimization to systematically and efficiently perform HiL Pareto characterizations. Our HiL optimization employs a hybrid model that measures performance with a quantitative metric, while the perceived challenge level is captured with a qualitative metric. We demonstrate the feasibility of the proposed HiL Pareto characterization through a user study. Furthermore, we present the utility of the framework through three use cases in the context of a manual skill training task with haptic feedback. First, we demonstrate how the characterized trade-off can be used to design a sample AAN training protocol for a motor learning task and to evaluate the group-level efficacy of the proposed AAN protocol relative to a baseline adaptive assistance protocol. Second, we demonstrate that individual-level comparisons of the trade-offs characterized before and after the training session enable fair evaluation of training progress under different assistance levels. This evaluation method is more general than standard performance evaluations, as it can provide insights even when users cannot perform the task without assistance. Third, we show that the characterized trade-offs also enable fair performance comparisons among different users, as they capture the best possible performance of each user under all feasible assistance levels.

Figures & tables

Fig. 1: Proposed HiL Pareto characterization scheme and its application to AAN training, within- and between-participant performance evaluations.
Fig. 2: Selection of non-dominated solutions from the Pareto-front by introducing design constraints after characterizing the trade-off
Fig. 3: (a) A participant holding a force-feedback joystick (HandsOn-SEA) while interacting with the pendulum game, (b) HandsOn-SEA haptic interface
Fig. 4: Experimental procedure
Fig. 5: In the top rows, the orange and blue lines show the mean of the surrogate function for the perceived challenge and the task performance, respectively, while the shading depicts their standard deviation. The bottom rows present the Pareto solutions with dark red dots, while the dominated solutions in the feasible set are shown with orange dots. The Pareto solutions inside the red squares are used during the training of the participants in the test group.
Fig. 6: Results for a sample participant with low performance. The presentation follows the same format as in Figure 5 .
Fig. 7: Violin plots of the unassisted pre- and post-training performance
Fig. 8: Comparison of aggregate Pareto solutions of control (a) and test (b) groups during pre- and post-training. In these figures, a shift of the Pareto front from left to right indicates improvement in task performance, while a shift from top to bottom indicates the task being perceived as less challenging. (c) Percent improvement of the available performance of the control and test groups.
Fig. 9: Assistance provided to participants in the control group (adaptive assistance) versus the prospective assistance that would have been provided to these participants in the control group if they were in the test group (Pareto approach). Lines show the average assistance levels provided to participants, while shaded areas indicate one standard deviation.
Fig. 10: A snapshot during HiL Pareto characterization and AAN training with the AssistOn-Arm upper-extremity exoskeleton
Figure 11
Figure 12
More in this topic

Human-in-the-Loop Evaluation

Human-in-the-Loop Evaluation studies how human judgment and feedback can assess, guide, and monitor AI systems in realistic settings.

See all 158 papers

Explore similar work

CardsList
Sep 25, 2026

Multi-Objective Human-in-the-Loop Bayesian Optimization of a Lower-Limb Exoskeleton

Human-in-the-loop optimization (HILO) is a common approach for optimizing the control of assistive devices to account for the wearer's unique biomechanics and subjective preferences. However, despite research suggesting that a person may have a different prioritization of objectives depending on time-varying factors such as the environment, their mood, or energy levels, existing HILO approaches only consider a single objective or enforce a fixed weighting on a set of objectives. Neither approach is capable of representing an individual's preferences over objectives. In this work, we propose Multi-Objective Human-in-the-loop Bayesian Optimization (MO-HILBO), which builds on explicit multi-objective Bayesian optimization to efficiently infer a personalized set of Pareto-optimal controllers. We compare our approach with an existing multi-objective HILO method and experimentally demonstrate MO-HILBO on a lower-limb exoskeleton across two objectives: metabolic cost (efficiency) and ordinal human feedback (comfort). We find that MO-HILBO (1) discovers Pareto-optimal controllers, and (2) that the pairwise ordering of points on the Pareto front itself is consistent with validation trials. Lastly, we open-source mohilo, a Python package for running both HILO and MO-HILBO on wearable devices: https://dynamicmobility.github.io/mohilo/.

Neil Janwani, Matthew T. Lerner, Aaron J. Young +1

Georgia Institute of Technology

Human-in-the-Loop AIBayesian OptimizationAssistive RoboticsAnkle Exoskeleton Control
AskPDF
Sep 18, 2026

Simultaneous Forward and Inverse Human-in-the-Loop Optimization

Subjective user experience is important to human-robot interaction, but the outcomes users value, and how those preferences vary across individuals and contexts, are often unknown. While inverse learning approaches using human data can help identify user rewards, in many assistive settings the experimental costs of executing a control policy, measuring biomechanical or physiological outcomes, and collecting user feedback often limit the number of queries and optimization iterations. Here, we present Simultaneous Forward and Inverse Human-In-the-Loop Optimization (SFIHILO), which efficiently infers individual-specific reward functions from preferences over human outcomes and identifies a final control policy that maximizes the learned reward. SFIHILO bootstraps a forward model to predict user outcomes from control policies, uses this model for active querying to accelerate inverse reward learning, and then optimizes the final control policy without additional user trials. We score candidate policies by their expected reduction in uncertainty across both forward and inverse beliefs, targeting a regional preference boundary to robustly inform this simultaneous learning process. In simulation, we show that SFIHILO was effective across user heterogeneity, outcome dimensionalities, precision requirements, noise levels, and nonstationarity; compared with mutual information approaches, the proposed active querying strategy significantly improved sample efficiency in inverse learning while preserving forward model accuracy. This approach demonstrates the potential to infer latent human goals, enabling more transferable and effective human-robot interaction.

Kyeongwon Park, Steven H. Collins

Department of Mechanical Engineering, Stanford University, United States

Human-in-the-Loop AIHuman-Robot InteractionPreference-Based RLAssistive RoboticsReinforcement Learning
AskPDF
Oct 10, 2026

Rethinking Opinion Measurement Interfaces for Human-in-the-Loop Optimization

Human-in-the-loop optimization refers to systems that present choices to a human decision-maker and query their judgment to optimize system parameters iteratively. The user interface typically provides either an absolute rating scale or asks for a preferred option. However, these approaches can potentially suffer from calibration problems, inconsistencies, and completeness violations of user preferences in such a sequential evaluation. This paper revisits the design space of opinion measurement interfaces for human-in-the-loop optimization. We conducted a study (N=360) using text summarization and image color enhancement tasks to evaluate human sequential feedback in six representative opinion measurement interfaces. Based on our analysis, we recommend using a listwise approach to support sequential opinion measurement, which counterweighs individual rating and preferential choice limitations and conveys more information for subsequent use. We further discuss the trade-offs in different user interface designs and provide guidelines to inform future research.

Changkun Ou, Sven Mayer, Daniel Buschek +1

LMU Munich, Germany · University of Bayreuth, Germany

Human-in-the-Loop AIPreference ElicitationHuman Preference Evaluation
AskPDF