A Cyber-Physical System (CPS) Approach to Support Worker Productivity based on Voice-Based Intelligent Virtual Agents

The Architecture, Engineering, and Construction (AEC) industry is currently challenged by low productivity trends and labor shortages. Efforts in academia and industry alike invested in developing solutions to this pressing issue. The majority of such efforts moved towards modernization of the industry, making use of digitalization approaches such as cyber-physical systems (CPS). In this direction, various research works have developed methods to capture information from construction environments and elements and provide monitoring capabilities to measure construction productivity at multiple levels. At the root of construction productivity, the productivity at the worker level is deemed critical. As a result, previous works explored monitoring the productivity of construction workers and resources to address the industry's productivity problems. However, productivity trends are not promising and show a need to more rigorously address productivity issues. Labor shortages also exacerbated the need for increasing the productivity of the current labor workers. Active means to address productivity have been explored as a solution in recent years. As a result, previous research took advantage of CPS and developed systems that sense construction workers' actions and environment and enable interaction with workers to render productivity improvements. One viable solution to this problem is providing on-demand activity-related information to the workers while at work, to decrease the need for manually seeking information from different sources, including supervisors, thereby improving their productivity. Especially, construction workers whose activities involve visual and manual limitations need to receive more attention, as seeking information can jeopardize their safety. Multiple labor trades such as plumbing, steel work, or carpenters are considered within this worker classification. These workers rely on knowledge gathered from the construction project documentation and databases, but have difficulties accessing this information while doing their work. Research works have explored the use of knowledge retrieval systems to give access to construction project data sources to construction workers through multiple methods, including information booths, mobile devices, and augmented reality (AR). However, these solutions do not address the need of this category of workers in receiving on-demand activity related information during their work, without negatively impacting their safety. This research focuses on voice, as an effective modality most appropriate for construction workers whose activities impose visual and manual limit actions. to this end, first, a voice-based solution is developed that supports workers' productivity through providing access to project knowledge available in Building Information Modeling (BIM) data sources. The effect of the selected modality on these workers' productivity is then evaluated using multiple user studies. The work presented in this dissertation is structured as follows: First, in chapter 2, a literature review was conducted to identify means to support construction workers and how integration with BIM has been done in previous research. This chapter identified challenges in incorporating human factors in previous systems and opportunities for seamless integration of workers into BIM practices. In chapter 3, voice-based assistance was explored as the most appropriate means to provide knowledge to workers while performing their activities. As such, Chapter 3 presents the first prototype of a voice-based intelligent virtual agent, aka VIVA, and focuses on evaluating the human factors and testing performance of voice as a modality for worker support. VIVA was tested using a user study involving a simulated construction scenario and the results of the performance achieved through VIVA were compared with the baseline currently used in construction projects for receiving activity-related information, i.e., blueprints. Results from this assessment evidenced productivity performance improvements of users using VIVA over the baseline. Finally, chapter 4 presents an updated version of VIVA that provides automatic real-time link to BIM project data and provides knowledge to the workers through voice. This system was developed based on web platforms, allowing easier development and deployment and access to more devices for future deployment. This study contributes to the productivity improvements in the AEC industry by empowering construction workers through providing on-demand access to project information. This is done through voice as a method that does not jeopardize workers' safety or interrupt their activities. This research contributes to the body of knowledge by developing an in-depth study of the effect of voice-based support systems on worker productivity, enabling real-time BIM-worker integration, and developing a working worker-level productivity support solution for construction workers whose activities limit them in manually accessing project knowledge.