Vision-Based Force Planning and Voice-Based Human-Machine Interface of an Assistive Robotic Exoskeleton Glove for Brachial Plexus Injuries

This dissertation focuses on improving the capabilities of an assistive robotic exoskeleton glove designed for patients with Brachial Plexus Injuries (BPI). The aim of this research is to develop a force control method, an automatic force planning method, and a Human-Machine Interface (HMI) to refine the grasping functionalities of the exoskeleton glove, thus helping rehabilitation and independent living for individuals with BPI. The exoskeleton glove is a useful tool in post-surgery therapy for patients with BPI, as it helps counteract hand muscle atrophy by allowing controlled and assisted hand movements. This study introduces an assistive exoskeleton glove with rigid side-mounted linkages driven by Series Elastic Actuators (SEAs) to perform five different types of grasps. In the aspect of force control, data-driven SEA fingertip force prediction methods were developed to assist force control with the Linear Series Elastic Actuators (LSEAs). This data-driven force prediction method can provide precise prediction of SEA fingertip force taking into account the deformation and friction force on the exoskeleton glove. In the aspect of force planning, a slip-grasp force planning method with hybrid slip detection is implemented. This method incorporates a vision-based approach to estimate object properties to refine grasp force predictions, thus mimicking human grasping processes and reducing the trial-and-error iterations required for the slip- grasp method, increasing the grasp success rate from 71.9% to 87.5%. In terms of HMI, the Configurable Voice Activation and Speaker Verification (CVASV) system was developed to control the proposed exoskeleton glove, which was then complemented by an innovative one-shot learning-based alternative, which proved to be more effective than CVASV in terms of training time and connectivity requirements. Clinical trials were conducted successfully in patients with BPI, demonstrating the effectiveness of the exoskeleton glove.