No Arabic abstract
The dexterity of conventional myoelectric prostheses is limited in part by the small datasets used to train the control algorithms. Variations in surface electrode positioning make it difficult to collect consistent data and to estimate motor intent reliably over time. To address these challenges, we developed an inexpensive, easy-to-don sleeve that can record robust and repeatable surface electromyography from 32 embedded monopolar electrodes. Embedded grommets are used to consistently align the sleeve with natural skin markings (e.g., moles, freckles, scars). The sleeve can be manufactured in a few hours for less than $60. Data from seven intact participants show the sleeve provides a signal-to-noise ratio of 14, a don-time under 11 seconds, and sub-centimeter precision for electrode placement. Furthermore, in a case study with one intact participant, we use the sleeve to demonstrate that neural networks can provide simultaneous and proportional control of six degrees of freedom, even 263 days after initial algorithm training. We also highlight that consistent recordings, accumulated over time to establish a large dataset, significantly improve dexterity. These results suggest that deep learning with a 74-layer neural network can substantially improve the dexterity and stability of myoelectric prosthetic control, and that deep-learning techniques can be readily instantiated and further validated through inexpensive sleeves/sockets with consistent recording locations.
Multiarticulate bionic arms are now capable of mimicking the endogenous movements of the human hand. 3D-printing has reduced the cost of prosthetic hands themselves, but there is currently no low-cost alternative to dexterous electromyographic (EMG) control systems. To address this need, we developed an inexpensive (~$675) and portable EMG control system by integrating low-cost microcontrollers with an EMG acquisition device. We validated signal acquisition by comparing the signal-to-noise ratio (SNR) of our system with that of a high-end research-grade system. We also demonstrate the ability to use the low-cost control system for proportional and independent control of various prosthetic hands in real-time. We found that the SNR of the low-cost control system was statistically no worse than 44% of the SNR of a research-grade control system. The RMSEs of predicted hand movements (from a modified Kalman filter) were typically a few percent better than, and not more than 6% worse than, RMSEs of a research-grade system for up to six degrees of freedom when only relatively few (six) EMG electrodes were used. However, RMSEs were generally higher than RMSEs of research-grade systems that utilize considerably more (32) EMG electrodes, guiding future work towards increasing electrode count. Successful instantiation of this low-cost control system constitutes an important step towards the commercialization and wide-spread availability of dexterous bionic hands.
Human environments contain numerous objects configured in a variety of arrangements. Our goal is to enable robots to repose previously unseen objects according to learned semantic relationships in novel environments. We break this problem down into two parts: (1) finding physically valid locations for the objects and (2) determining if those poses satisfy learned, high-level semantic relationships. We build our models and training from the ground up to be tightly integrated with our proposed planning algorithm for semantic placement of unknown objects. We train our models purely in simulation, with no fine-tuning needed for use in the real world. Our approach enables motion planning for semantic rearrangement of unknown objects in scenes with varying geometry from only RGB-D sensing. Our experiments through a set of simulated ablations demonstrate that using a relational classifier alone is not sufficient for reliable planning. We further demonstrate the ability of our planner to generate and execute diverse manipulation plans through a set of real-world experiments with a variety of objects.
Bypass sockets allow researchers to perform tests of prosthetic systems from the prosthetic users perspective. We designed a modular upper-limb bypass socket with 3D-printed components that can be easily modified for use with a variety of terminal devices. Our bypass socket preserves access to forearm musculature and the hand, which are necessary for surface electromyography and to provide substituted sensory feedback. Our bypass socket allows a sufficient range of motion to complete tasks in the frontal working area, as measured on non-amputee participants. We examined the performance of non-amputee participants using the bypass socket on the original and modified Box and Block Tests. Participants moved 11.3 +/- 2.7 and 11.7 +/- 2.4 blocks in the original and modified Box and Block Tests (mean +/- SD), respectively, within the range of reported scores using amputee participants. Range-of-motion for users wearing the bypass socket meets or exceeds most reported range-of-motion requirements for activities of daily living. The bypass socket was originally designed with a freely rotating wrist; we found that adding elastic resistance to user wrist rotation while wearing the bypass socket had no significant effect on motor decode performance. We have open-sourced the design files and an assembly manual for the bypass socket. We anticipate that the bypass socket will be a useful tool to evaluate and develop sensorized myoelectric prosthesis technology.
Reinforcement learning (RL) is a promising approach and has limited success towards real-world applications, because ensuring safe exploration or facilitating adequate exploitation is a challenges for controlling robotic systems with unknown models and measurement uncertainties. Such a learning problem becomes even more intractable for complex tasks over continuous space (state-space and action-space). In this paper, we propose a learning-based control framework consisting of several aspects: (1) linear temporal logic (LTL) is leveraged to facilitate complex tasks over an infinite horizons which can be translated to a novel automaton structure; (2) we propose an innovative reward scheme for RL-agent with the formal guarantee such that global optimal policies maximize the probability of satisfying the LTL specifications; (3) based on a reward shaping technique, we develop a modular policy-gradient architecture utilizing the benefits of automaton structures to decompose overall tasks and facilitate the performance of learned controllers; (4) by incorporating Gaussian Processes (GPs) to estimate the uncertain dynamic systems, we synthesize a model-based safeguard using Exponential Control Barrier Functions (ECBFs) to address problems with high-order relative degrees. In addition, we utilize the properties of LTL automatons and ECBFs to construct a guiding process to further improve the efficiency of exploration. Finally, we demonstrate the effectiveness of the framework via several robotic environments. And we show such an ECBF-based modular deep RL algorithm achieves near-perfect success rates and guard safety with a high probability confidence during training.
Non-rigid cortical registration is an important and challenging task due to the geometric complexity of the human cortex and the high degree of inter-subject variability. A conventional solution is to use a spherical representation of surface properties and perform registration by aligning cortical folding patterns in that space. This strategy produces accurate spatial alignment but often requires a high computational cost. Recently, convolutional neural networks (CNNs) have demonstrated the potential to dramatically speed up volumetric registration. However, due to distortions introduced by projecting a sphere to a 2D plane, a direct application of recent learning-based methods to surfaces yields poor results. In this study, we present SphereMorph, a diffeomorphic registration framework for cortical surfaces using deep networks that addresses these issues. SphereMorph uses a UNet-style network associated with a spherical kernel to learn the displacement field and warps the sphere using a modified spatial transformer layer. We propose a resampling weight in computing the data fitting loss to account for distortions introduced by polar projection, and demonstrate the performance of our proposed method on two tasks, including cortical parcellation and group-wise functional area alignment. The experiments show that the proposed SphereMorph is capable of modeling the geometric registration problem in a CNN framework and demonstrate superior registration accuracy and computational efficiency. The source code of SphereMorph will be released to the public upon acceptance of this manuscript at https://github.com/voxelmorph/spheremorph.