Real-Time Processing and Visualization of High-Volume Smart Infrastructure Data Using Open-Source Technologies

Smart infrastructure has become increasingly prevalent in recent decades due to the emergence of sophisticated and affordable sensing technologies. As sensors are deployed more widely and higher sampling rates are feasible, managing the massive scale of real-time data collected by these systems has become fundamental to providing relevant and timely information to decision-makers. To address this task, a novel open-source framework has been developed to manage and intuitively present high-volume data in near real-time. This design is centered around the goals of making data accessible, supporting decision-making, and providing flexibility to modify and reuse this framework in the future. In this work, the framework is tailored to vibration-based structural health monitoring, which can be used in near real-time to screen building condition. To promote timely intervention, distributed computing technologies are employed to accelerate the processing, storage, and visualization of data. Vibration data is processed in parallel using a publish-subscribe messaging queue and then inserted into a NoSQL database that stores heterogeneous data across several nodes. A REST-based web application allows interaction with this stored data via customizable visualization interfaces. To illustrate the utility of this framework design, it has been implemented to support a frequency domain monitoring dashboard for a 5-story classroom building instrumented with 224 accelerometers. A simulated scenario is presented to capture how the dashboard can aid decisions about occupant safety and structural maintenance.