Learning Schemes for Adaptive Spectrum Sharing Radar

Society's newfound dependence on wireless transmission systems has driven demand for access to the electromagnetic (EM) spectrum to an all-time high. In particular, wireless applications related to the fifth generation (5G) of cellular technology along with statically allocated radar systems have contributed to the increasing scarcity of the sub 6 GHz frequency bands. As a result, development of Dynamic Spectrum Access (DSA) techniques for sharing these frequencies has become a critical research area for the greater wireless community. Since among incumbent systems, radars are the largest consumers of spectrum in the sub 6 GHz regime, and are being used increasingly for civilian applications such as traffic control, adaptive cruise control, and collision avoidance, the need for radars which can adaptively tune specific transmission parameters in an intelligent manner to promote coexistence with other systems has arisen. Thus, fully-aware, dynamic, cognitive radar has been proposed as target for radars to evolve towards.

In this thesis, we extend current research thrusts towards cognitive radar to utilize Reinforcement Learning (RL) techniques which allow a radar system to learn desired behavior using information obtained from past transmissions. Since radar systems inherently interact with their electromagnetic environment, it is natural to view the use of reinforcement learning techniques as a straightforward extension to previous adaptive techniques. However, in designing learning algorithms for radar systems, we must carefully define goal-driven rewards, formalize the learning process, and consider an appropriate amount of environmental information. In this thesis, we apply well-established and emerging reinforcement learning approaches to meet the demands of modern radar coexistence problems. In particular, function estimation using deep neural networks is examined, as Deep RL presents a scalable learning framework which allows many environmental states to be considered in the decision-making process. We then show how these techniques can be used to improve traditional radar performance metrics, such as interference avoidance, spectral efficiency, and target detectibility with simulated and experimental results. We also compare the learning techniques to each other and naive approaches, such as fixed bandwidth radar and avoiding interference reactively. Finally, online learning strategies are considered which aim to balance the fundamental learning trade-off between exploration and exploitation. We show that online learning techniques can be used to select individual waveforms or applied as a high-level controller in a hierarchical learning scheme based on the biologically inspired concept of metacognition.

The general use of RL techniques provides a robust framework for decision making under uncertainty that is more flexible than previously proposed cognitive radar strategies. Further, the wide array of RL models and algorithms allow the underlying structure to be applied to both small and large-scale radar scenarios.