Bayesian Preference Elicitation: Human-In-The-Loop Optimization of An Active Prosthesis

Human-in-the-loop optimization (HILO) is a common approach for optimizing the control of assistive devices to account for the wearer's unique biomechanics and subjective preferences. However, despite research suggesting that a person may have a different prioritization of objectives depending on time-varying factors such as the environment, their mood, or energy levels, existing HILO approaches only consider a single objective or enforce a fixed weighting on a set of objectives. Neither approach is capable of representing an individual's preferences over objectives. In this work, we propose Multi-Objective Human-in-the-loop Bayesian Optimization (MO-HILBO), which builds on explicit multi-objective Bayesian optimization to efficiently infer a personalized set of Pareto-optimal controllers. We compare our approach with an existing multi-objective HILO method and experimentally demonstrate MO-HILBO on a lower-limb exoskeleton across two objectives: metabolic cost (efficiency) and ordinal human feedback (comfort). We find that MO-HILBO (1) discovers Pareto-optimal controllers, and (2) that the pairwise ordering of points on the Pareto front itself is consistent with validation trials. Lastly, we open-source mohilo, a Python package for running both HILO and MO-HILBO on wearable devices: https://dynamicmobility.github.io/mohilo/.

A significant challenge in exoskeleton robotics is the need to dynamically adapt control profiles to individual motion preferences, thereby ensuring both efficient and comfortable assistance. Currently, since user experience can serve as a comprehensive metric for evaluating the effectiveness of assistance, user preference-based optimization methods have been widely studied for parameter tuning. However, the existing methods rely heavily on extensive human-robot online interactions and suffer from slow optimization speed, which not only induces user fatigue but also compromises optimization effectiveness. Therefore, this paper aims to explore an efficient preference-based optimization framework for personalized exoskeleton assistance that can learn optimal parameters with minimal interaction. We propose a preference-based Bayesian optimization (PbBO) approach that can improve sample efficiency by leveraging knowledge about the sampling distribution of candidate sets. For optimizing six control parameters, PbBO can converge to user-preferred parameters with 90.7% validation accuracy via 20 iterations. Moreover, the hierarchical controller is designed to generate personalized torque for different tasks and achieve interaction torque tracking in real time. The results of treadmill and outdoor experiments demonstrate that the optimized parameters can reduce metabolic rate by 14.5%-15.4%, heart rate by 6.3%-7.6%, and muscle activation by 6.7%-31.5% compared to unassisted walking.

Tuning control policies manually to meet high-level objectives is often time-consuming. Bayesian optimization provides a data-efficient framework for automating this process using numerical evaluations of an objective function. However, many systems, particularly those involving humans, require optimization based on subjective criteria. Preferential Bayesian optimization addresses this by learning from pairwise comparisons instead of quantitative measurements, but relying solely on preference data can be inefficient. We propose a multi-fidelity, multi-modal Bayesian optimization framework that integrates low-fidelity numerical data with high-fidelity human preferences. Our approach employs Gaussian process surrogate models with both hierarchical, autoregressive and non-hierarchical, coregionalization-based structures, enabling efficient learning from mixed-modality data. We illustrate the framework by tuning an autonomous vehicle's trajectory planner, showing that combining numerical and preference data significantly reduces the need for experiments involving the human decision maker while effectively adapting driving style to individual preferences.