An indoor fall monitoring system: Robust, multistatic radar sensing and explainable, feature-resonated deep neural network

Indoor fall monitoring is challenging for community-dwelling older adults due to the need for high accuracy and privacy concerns. Doppler radar is promising, given its low cost and contactless sensing mechanism. However, the line-of-sight restriction limits the application of radar sensing in practice, as the Doppler signature will vary when the sensing angle changes, and signal strength will be substantially degraded with large aspect angles. Additionally, the similarity of the Doppler signatures among different fall types makes it extremely challenging for classification. To address these problems, in this paper we first present a comprehensive experimental study to obtain Doppler radar signals under large and arbitrary aspect angles for diverse types of simulated falls and daily living activities. We then develop a novel, explainable, multi-stream, feature-resonated neural network (eMSFRNet) that achieves fall detection and a pioneering study of classifying seven fall types. eMSFRNet is robust to both radar sensing angles and subjects. It is also the first method that can resonate and enhance feature information from noisy/weak Doppler signatures. The multiple feature extractors - including partial pre-trained layers from ResNet, DenseNet, and VGGNet - extracts diverse feature information with various spatial abstractions from a pair of Doppler signals. The feature-resonated-fusion design translates the multi-stream features to a single salient feature that is critical to fall detection and classification. eMSFRNet achieved 99.3% accuracy detecting falls and 76.8% accuracy for classifying seven fall types. Our work is the first effective multistatic robust sensing system that overcomes the challenges associated with Doppler signatures under large and arbitrary aspect angles, via our comprehensible feature-resonated deep neural network. Our work also demonstrates the potential to accommodate different radar monitoring tasks that demand precise and robust sensing.