Natural Multicontact Walking for Robotic Assistive Devices via Musculoskeletal Models and Hybrid Zero Dynamics

Generating provably stable walking gaits that yield natural locomotion when executed on robotic-assistive devices is a challenging task that often requires hand-tuning by domain experts. This paper presents an alternative methodology, where we propose the addition of musculoskeletal models directly into the gait generation process to intuitively shape the resulting behavior. In particular, we construct a multi-domain hybrid system model that combines the system dynamics with muscle models to represent natural multicontact walking. Stable walking gaits can then be formally generated for this model via the hybrid zero dynamics method. We experimentally apply our framework towards achieving multicontact locomotion on a dual-actuated transfemoral prosthesis, AMPRO3. The results demonstrate that enforcing feasible muscle dynamics produces gaits that yield natural locomotion (as analyzed via electromyography), without the need for extensive manual tuning. Moreover, these gaits yield similar behavior to expert-tuned gaits. We conclude that the novel approach of combining robotic walking methods (specifically HZD) with muscle models successfully generates anthropomorphic robotic-assisted locomotion.

Learning Controller Gains on Bipedal Walking Robots via User Preferences

Experimental demonstration of complex robotic behaviors relies heavily on finding the correct controller gains. This painstaking process is often completed by a domain expert, requiring deep knowledge of the relationship between parameter values and the resulting behavior of the system. Even when such knowledge is possessed, it can take significant effort to navigate the nonintuitive landscape of possible parameter combinations. In this work, we explore the extent to which preference-based learning can be used to optimize controller gains online by repeatedly querying the user for their preferences. This general methodology is applied to two variants of control Lyapunov function based nonlinear controllers framed as quadratic programs, which have nice theoretic properties but are challenging to realize in practice. These controllers are successfully demonstrated both on the planar underactuated biped, AMBER, and on the 3D underactuated biped, Cassie. We experimentally evaluate the performance of the learned controllers and show that the proposed method is repeatably able to learn gains that yield stable and robust locomotion.

Preference-Based Learning for User-Guided HZD Gait Generation on Bipedal Walking Robots

This paper presents a framework that leverages both control theory and machine learning to obtain stable and robust bipedal locomotion without the need for manual parameter tuning. Traditionally, gaits are generated through trajectory optimization methods and then realized experimentally -- a process that often requires extensive tuning due to differences between the models and hardware. In this work, the process of gait realization via hybrid zero dynamics (HZD) based optimization is formally combined with preference-based learning to systematically realize dynamically stable walking. Importantly, this learning approach does not require a carefully constructed reward function, but instead utilizes human pairwise preferences. The power of the proposed approach is demonstrated through two experiments on a planar biped AMBER-3M: the first with rigid point-feet, and the second with induced model uncertainty through the addition of springs where the added compliance was not accounted for in the gait generation or in the controller. In both experiments, the framework achieves stable, robust, efficient, and natural walking in fewer than 50 iterations with no reliance on a simulation environment. These results demonstrate a promising step in the unification of control theory and learning.

ROIAL: Region of Interest Active Learning for Characterizing Exoskeleton Gait Preference Landscapes

Characterizing what types of exoskeleton gaits are comfortable for users, and understanding the science of walking more generally, require recovering a users utility landscape. Learning these landscapes is challenging, as walking trajectories are defined by numerous gait parameters, data collection from human trials is expensive, and user safety and comfort must be ensured. This work proposes the Region of Interest Active Learning (ROIAL) framework, which actively learns each users underlying utility function over a region of interest that ensures safety and comfort. ROIAL learns from ordinal and preference feedback, which are more reliable feedback mechanisms than absolute numerical scores. The algorithms performance is evaluated both in simulation and experimentally for three non-disabled subjects walking inside of a lower-body exoskeleton. ROIAL learns Bayesian posteriors that predict each exoskeleton users utility landscape across four exoskeleton gait parameters. The algorithm discovers both commonalities and discrepancies across users gait preferences and identifies the gait parameters that most influenced user feedback. These results demonstrate the feasibility of recovering gait utility landscapes from limited human trials.

Human Preference-Based Learning for High-dimensional Optimization of Exoskeleton Walking Gaits

Optimizing lower-body exoskeleton walking gaits for user comfort requires understanding users preferences over a high-dimensional gait parameter space. However, existing preference-based learning methods have only explored low-dimensional domains due to computational limitations. To learn user preferences in high dimensions, this work presents LineCoSpar, a human-in-the-loop preference-based framework that enables optimization over many parameters by iteratively exploring one-dimensional subspaces. Additionally, this work identifies gait attributes that characterize broader preferences across users. In simulations and human trials, we empirically verify that LineCoSpar is a sample-efficient approach for high-dimensional preference optimization. Our analysis of the experimental data reveals a correspondence between human preferences and objective measures of dynamicity, while also highlighting differences in the utility functions underlying individual users gait preferences. This result has implications for exoskeleton gait synthesis, an active field with applications to clinical use and patient rehabilitation.

Preference-Based Learning for Exoskeleton Gait Optimization

This paper presents a personalized gait optimization framework for lower-body exoskeletons. Rather than optimizing numerical objectives such as the mechanical cost of transport, our approach directly learns from user preferences, e.g., for comfort. Building upon work in preference-based interactive learning, we present the CoSpar algorithm. CoSpar prompts the user to give pairwise preferences between trials and suggest improvements; as exoskeleton walking is a non-intuitive behavior, users can provide preferences more easily and reliably than numerical feedback. We show that CoSpar performs competitively in simulation and demonstrate a prototype implementation of CoSpar on a lower-body exoskeleton to optimize human walking trajectory features. In the experiments, CoSpar consistently found user-preferred parameters of the exoskeletons walking gait, which suggests that it is a promising starting point for adapting and personalizing exoskeletons (or other assistive devices) to individual users.

Stabilization of Exoskeletons through Active Ankle Compensation

This paper presents an active stabilization method for a fully actuated lower-limb exoskeleton. The method was tested on the exoskeleton ATALANTE, which was designed and built by the French start-up company Wandercraft. The main objective of this paper is to present a practical method of realizing more robust walking on hardware through active ankle compensation. The nominal gait was generated through the hybrid zero dynamic framework. The ankles are individually controlled to establish three main directives; (1) keeping the non-stance foot parallel to the ground, (2) maintaining rigid contact between the stance foot and the ground, and (3) closing the loop on pelvis orientation to achieve better tracking. Each individual component of this method was demonstrated separately to show each components contribution to stability. The results showed that the ankle controller was able to experimentally maintain static balance in the sagittal plane while the exoskeleton was balanced on one leg, even when disturbed. The entire ankle controller was then also demonstrated on crutch-less dynamic walking. During testing, an anatomically correct manikin was placed in the exoskeleton, in lieu of a paraplegic patient. The pitch of the pelvis of the exoskeleton-manikin system was shown to track the gait trajectory better when ankle compensation was used. Overall, active ankle compensation was demonstrated experimentally to improve balance in the sagittal plane of the exoskeleton manikin system and points to an improved practical approach for stable walking.

Towards Variable Assistance for Lower Body Exoskeletons

This paper presents and experimentally demonstrates a novel framework for variable assistance on lower body exoskeletons, based upon safety-critical control methods. Existing work has shown that providing some freedom of movement around a nominal gait, instead of rigidly following it, accelerates the spinal learning process of people with a walking impediment when using a lower body exoskeleton. With this as motivation, we present a method to accurately control how much a subject is allowed to deviate from a given gait while ensuring robustness to patient perturbation. This method leverages control barrier functions to force certain joints to remain inside predefined trajectory tubes in a minimally invasive way. The effectiveness of the method is demonstrated experimentally with able-bodied subjects and the Atalante lower body exoskeleton.