Autonomous Cricothyroid Membrane Detection and Manipulation using Neural Networks and Robot Arm for First-Aid Airway Management

Han, Xiaoxue

Autonomous Cricothyroid Membrane Detection and Manipulation using Neural Networks and Robot Arm for First-Aid Airway Management

dc.contributor.author	Han, Xiaoxue	en
dc.contributor.committeechair	Ben-Tzvi, Pinhas	en
dc.contributor.committeemember	Southward, Steve C.	en
dc.contributor.committeemember	Sandu, Corina	en
dc.contributor.department	Mechanical Engineering	en
dc.date.accessioned	2021-11-25T07:00:27Z	en
dc.date.available	2021-11-25T07:00:27Z	en
dc.date.issued	2020-06-02	en
dc.description.abstract	The thesis focuses on applying deep learning and reinforcement learning techniques on human keypoint detection and robot arm manipulation. Inspired by Semi-Autonomous Victim Extraction Robot (SAVER), an autonomous first-aid airway-management robotic system designed to perform Cricothyrotomy on patients is proposed. Perception, decision-making, and control are embedded in the system. In this system, first, the location of the cricothyroid membrane (CTM)-the incision site of Cricothyrotomy- is detected; then, the robot arm is controlled to reach the detected position on a medical manikin. A hybrid neural network (HNNet) that can balance both speed and accuracy is proposed. HNNet is an ensemble-based network architecture that consists of two ensembles: the region proposal ensemble and the keypoint detection ensemble. This architecture can maintain the original high resolution of the input image without heavy computation and can meet the high-precision and real-time requirements at the same time. A dataset containing more than 16,000 images from 13 people, with a clear view of the neck area, and with CTM position labeled by a medical expert was built to train and validate the proposed model. It achieved a success rate of $99.6%$ to detect the position of the CTM with an error of less than 5mm. The robot arm manipulator was trained with the reinforcement learning model to reach the detected location. Finally, the detection neural network and the manipulation process are combined as an integrated system. The system was validated in real-life experiments on a human-sized medical manikin using a Kinect V2 camera and a MICO robot arm manipulator.	en
dc.description.abstractgeneral	The thesis focuses on applying deep learning and reinforcement learning techniques on human keypoint detection and robot arm manipulation. Inspired by Semi-Autonomous Victim Extraction Robot (SAVER), an autonomous first-aid airway-management robotic system designed to perform Cricothyrotomy on patients is proposed. Perception, decision-making, and control are embedded in the system. In this system, first, the location of the cricothyroid membrane(CTM)-the incision site of Cricothyrotomy- is detected; then, the robot arm is controlled to reach the detected position on a medical manikin. A hybrid neural network (HNNet) that can balance both speed and accuracy is proposed. HNNet is an ensemble-based network architecture that consists of two ensembles: the region proposal ensemble and the keypoint detection ensemble. This architecture can maintain the original high resolution of the input image without heavy computation and can meet the high-precision and real-time requirements at the same time. Finally, the detection neural network and the manipulation process are combined as an integrated system. The robot arm manipulator was trained with the reinforcement learning model to reach the detected location. The system was validated in real-life experiments on a human-sized medical manikin using an RGB-D camera and a robot arm manipulator.	en
dc.description.degree	Master of Science	en
dc.format.medium	ETD	en
dc.identifier.other	vt_gsexam:25913	en
dc.identifier.uri	http://hdl.handle.net/10919/106738	en
dc.publisher	Virginia Tech	en
dc.rights	In Copyright	en
dc.rights.uri	http://rightsstatements.org/vocab/InC/1.0/	en
dc.subject	Human-Robot Interaction	en
dc.subject	Deep learning (Machine learning)	en
dc.subject	Reinforcement Learning	en
dc.subject	Medical Robots	en
dc.title	Autonomous Cricothyroid Membrane Detection and Manipulation using Neural Networks and Robot Arm for First-Aid Airway Management	en
dc.type	Thesis	en
thesis.degree.discipline	Mechanical Engineering	en
thesis.degree.grantor	Virginia Polytechnic Institute and State University	en
thesis.degree.level	masters	en
thesis.degree.name	Master of Science	en