MIND: A Multimodal AI Framework for Detecting and Forecasting Motor RRBs among Children with ASD

Abstract

Motor restricted and repetitive behaviors (RRBs), including self-injurious behavior (SIB) and stereotypical motor movements (SMM), hinder social interactions and adversely impact the physical and psychological well-being of individuals with autism spectrum disorder (ASD) and their families. Although behavioral interventions can effectively address RRBs, their accurate detection and forecasting were previously considered unattainable due to their impulsive nature and individualized behavior types, triggers, and patterns. Monitoring these behaviors may be possible via wearable sensors, but is challenged by expected inconsistencies in how sensors are worn, especially given the often low compliance observed among children with ASD. In this study, we introduced a novel AI framework for detecting and forecasting motor RRBs – Multimodal, Interpretation, Numeration, and Deep neural decoding (MIND). We observed what we term ”transition behaviors,” in which participants exhibited subtle changes in their behavior patterns or facial expressions immediately preceding the onset of motor RRBs. Identifying these transition behaviors provided evidence that motor RRBs can, in fact, be forecasted. Through a series of assumptions, the multimodal interpretation within MIND connects wearable sensor functionality to existing behavioral and psychological evidence about motor RRBs. Additionally, novel signal processing guidelines categorize modality into motion and biological modalities. These guidelines process the signal based on their generalized functionality, ensuring robustness to inconsistent data and minimizing the impact of sensor specifications (i.e., range and units of measurement, sensor resolution, sensor orientation). Analyses of modalities supported the noted assumptions. The multimodal optimization under MIND framework suggested the effective use of a single wearable device integrating several sensors (or modalities). Crucially, all children in the study were willing to wear the sensor at the optimized location, highlighting its practicality. MIND achieved 100% accuracy in detecting motor RRBs in new subjects with unfamiliar behavior types and 92.2% accuracy in forecasting (2 sec. in advance) motor RRBs. Cross-validation using various sampling methods showcased that MIND has the potential to generalize to a broader sample of children with ASD. MIND provides an advancement in the automated detection and forecasting of motor RRBs.