Human-Interactions with Robotic Cyber-Physical Systems (CPS) for Facilitating Construction Progress Monitoring

Progress monitoring in construction involves a set of inspection tasks with repetitive in-person observations on the site. The current manual inspection process is time-consuming, inefficient, inconsistent, and has many safety risks to project inspectors. Cyber-Physical Systems (CPS) are networks of integrated physical and cyber components, such as robots, sensors, actuators, cloud computing, artificial intelligence, and the building itself. Introducing CPS for construction progress monitoring can reduce risks involved in the process, improve efficiency, and enable remote progress monitoring. A robotic CPS uses a robot as the core component of the CPS. But human interaction with technology plays an important role in the successful implementation of any technology. This research studied the human-centered design of a CPS from a human-computer interaction perspective for facilitating construction progress monitoring that puts the needs and abilities of humans at the center of the development process. User experience and interactions play an important role in human-centered design. This study first develops a CPS framework to autonomously collect visual data and facilitate remote construction progress monitoring. The two types of interactions occur between the human and the CPS – the human provides input for the CPS to collect data referred to as mission planning, and CPS provides visual data to enable the human to perform progress analysis. The interaction may occur through different modalities, such as visual, tactile, auditory, and immersive. The goal of this research is to understand the role of human interactions with CPS for construction progress monitoring. The study answers five research questions – a) What robotic CPS framework can be applied in construction progress monitoring? b) To what extent is the proposed CPS framework acceptable as an alternative to traditional construction progress monitoring? c) How can natural interaction modalities like hand gestures and voice commands be used as human-CPS interaction modalities for the proposed CPS? d) How does the human interaction modality between the proposed CPS and its user affect the usability of the proposed CPS? e) How does the human interaction modality between CPS and its user affect the performance of the proposed CPS?. To answer the research questions, a mixed-method-based methodology is used in this study. First, a systematic literature review is performed on the use of robots in inspection and monitoring of the built environment. Second, a CPS framework for remote progress monitoring is developed and evaluated in lab conditions. Third, a set of industry experts experienced with construction progress monitoring are interviewed to measure their acceptance of the developed CPS and to collect feedback for the evaluation of the CPS. Fourth, two methodologies are developed to use hand gesture and voice command recognition for human-CPS interaction in progress monitoring. Fifth, the usability and performance of the CPS are measured for identified interaction modalities through a human subject study. The human subjects are also interviewed post-experiment to identify the challenges they faced in their interactions with the CPS. The study makes the following contributions to the body of knowledge – a) key research areas and gaps were identified for robots in inspection and monitoring of the built environment, b) a fundamental framework for a robotic CPS was developed to automate reality capture and visualization using quadruped robots to facilitate remote construction progress monitoring, c) factors affecting the acceptance of the proposed robotic CPS for construction progress monitoring were identified by interviewing construction experts, d) two methodologies for using hand gestures and voice commands were developed for human-CPS interaction in construction progress monitoring, e) the effect of human interaction modalities on the usability and performance of the proposed CPS was assessed in construction progress monitoring through user studies, f) factors affecting the usability and performance of the proposed CPS with different interaction modalities were identified by conducting semi-structured interviews with users.