Wireless Sensing and Fusion using Deep Neural Networks

Deep Neural Networks (DNNs) have been proposed to solve many difficult problems within the context of wireless sensing. Indoor localization and human activity recognition (HAR) are two major applications of wireless sensing. However, current fingerprint-based localization methods require massive amounts of labeled data and suffer severe performance degradation in NLOS environments. To address this challenge, we first apply DNNs to multi-modal wireless signals, including Wi-Fi, an inertial measurement unit (IMU), and ultra-wideband (UWB). By formulating localization as a multi-modal sequence regression problem, a multi- stream recurrent fusion method is developed to combine the current hidden state of each modality. This is done in the context of recurrent neural networks while accounting for the modality uncertainty directly learned from its immediate past states. The proposed method was evaluated on a large-scale open dataset and compared with a wide range of baseline methods. It is shown that the proposed approach has an average error below 20 centimeters, which is nearly three times better than classic methods. Second, in the context of activity recognition, we propose a multi-band WiFi fusion frame- work that hierarchically combines the features of sub-6 GHz channel state information (CSI) and the beam signal-to-noise ratio (SNR) at 60 GHz at different granularity levels. Specifically, we introduce three fusion methods: simple input fusion, feature fusion, and a more customized feature permutation that accounts for the granularity correspondence between the CSI and beam SNR measurements for task-specific sensing. To mitigate the problem of limited labeled training data, we further propose an autoencoder-based unsupervised fusion network consisting of separate encoders and decoders for the CSI and beam SNR. The effectiveness of the framework is thoroughly validated using an in-house experimental platform which includes indoor localization, pose recognition, and occupancy sensing. Finally, in the context of array processing, we solve the Model order estimation (MOE) problem, a prerequisite for Direction of Arrival (DoA) estimation in the presence of correlated multipath, a well-known difficult problem. Due to the limits imposed by array geometry, it is not possible to estimate spatial parameters for an arbitrary number of sources; an estimate of the signal model is required. While classic methods fail at MOE in the presence of correlated multi-path interference, we show that data-driven supervised learning models can meet this challenge. In particular, we propose the application of Residual Neural Net- works (ResNets), with grouped symmetric kernel filters to provide an accuracy over 95%, and a weighted loss function to eliminate the underestimation error of model order. The improved MOE is shown improve subsequent array processing tasks such as reducing the overhead needed for temporal smoothing, reducing the search space for signal association, and improving DoA estimation.