Real-Time Processing and Visualization of High-Volume Smart Infrastructure Data Using Open-Source Technologies
Vipond, Natasha M.
MetadataShow full item record
Smart infrastructure has become increasingly prevalent in recent decades due to the emergence of sophisticated and affordable sensing technologies. As sensors are deployed more widely and higher sampling rates are feasible, managing the massive scale of real-time data collected by these systems has become fundamental to providing relevant and timely information to decision-makers. To address this task, a novel open-source framework has been developed to manage and intuitively present high-volume data in near real-time. This design is centered around the goals of making data accessible, supporting decision-making, and providing flexibility to modify and reuse this framework in the future. In this work, the framework is tailored to vibration-based structural health monitoring, which can be used in near real-time to screen building condition. To promote timely intervention, distributed computing technologies are employed to accelerate the processing, storage, and visualization of data. Vibration data is processed in parallel using a publish-subscribe messaging queue and then inserted into a NoSQL database that stores heterogeneous data across several nodes. A REST-based web application allows interaction with this stored data via customizable visualization interfaces. To illustrate the utility of this framework design, it has been implemented to support a frequency domain monitoring dashboard for a 5-story classroom building instrumented with 224 accelerometers. A simulated scenario is presented to capture how the dashboard can aid decisions about occupant safety and structural maintenance.
General Audience Abstract
Advances in technology have made it affordable and accessible to collect information about the world around us using sensors. When sensors are used to aid decision-making about structures, it is frequently referred to as Structural Health Monitoring (SHM). SHM can be used to monitor long-term structural health, inform maintenance decisions, and rapidly screen structural conditions following extreme events. Accelerometers can be used in SHM to capture vibration data that give insight into deflection patterns and natural frequencies in a structure. The challenge with vibration-based SHM and many other applications that leverage sensors is that the amount of data collected has the potential to grow to massive scales. To communicate relevant information to decision-makers, data must be processed quickly and presented intuitively. To facilitate this process, a novel open-source framework was developed for processing, storing, and visualizing high-volume data in near real-time. This framework combines multiple computers to extend the processing and storage capacity of our system. Data is processed in parallel and stored in a database that supports efficient data retrieval. A web application enables interaction with stored data via customizable visualization interfaces. To demonstrate the framework functionality, it was implemented in a 5-story classroom building instrumented with 224 accelerometers. A frequency-domain dashboard was developed for the building, and a simulated scenario was conducted to capture how the dashboard can aid decisions about occupant safety and structural maintenance.
- Masters Theses