Browsing by Author "Akanmu, Abiola Abosede"

Now showing 1 - 7 of 7
  • Thumbnail Image
    Analysis and Management of UAV-Captured Images towards Automation of Building Facade Inspections
    Chen, Kaiwen (Virginia Tech, 2020-08-27)
    Building facades, serving mainly to protect occupants and structural components from natural forces, require periodic inspections for the detection and assessment of building façade anomalies. Over the past years, a growing trend of utilizing camera-equipped drones for periodical building facade inspection has emerged. Building façade anomalies, such as cracks and erosion, can be detected through analyzing drone-captured video, photographs, and infrared images. Such anomalies are known to have an impact on various building performance aspects, e.g., thermal, energy, moisture control issues. Current research efforts mainly focus on the design of drone flight schema for building inspection, 3D building model reconstruction through drone-captured images, and the detection of specific façade anomalies with these images. However, there are several research gaps impeding the improvement of automation level during the processes of building façade inspection with UAV (Unmanned Aerial Vehicle). These gaps are (1) lack effective ways to store multi-type data captured by drones with the connection to the spatial information of building facades, (2) lack high-performance tools for UAV-image analysis for the automated detection of building façade anomalies, and (3) lack a comprehensive management (i.e., storage, retrieval, analysis, and display) of large amounts and multi-media information for cyclic façade inspection. When seeking inspirations from nature, the process of drone-based facade inspection can be compared with caching birds' foraging food through spatial memory, visual sensing, and remarkable memories. This dissertation aims at investigating ways to improve the management of UAV-captured data and the automation level of drone-based façade anomaly inspection with inspirations from caching birds' foraging behavior. Firstly, a 2D spatial model of building façades was created in the geographic information system (GIS) for the registration and storage of UAV-images to assign façade spatial information to each image. Secondly, computational methods like computer vision and deep learning neural networks were applied to develop algorithms for automated extraction of visual features of façade anomalies within UAV-captured images. Thirdly, a GIS-based database was designed for the comprehensive management of heterogeneous inspection data, such as the spatial, multi-spectral, and temporal data. This research will improve the automation level of storage, retrieval, analysis, and documentation of drone-captured images to support façade inspection during a building's service lifecycle. It has promising potential for supporting the decision-making of early-intervention or maintenance strategies to prevent façade failures and improve building performance.
  • Thumbnail Image
    Evaluating Mental Workload for AR Head-Mounted Display Use in Construction Assembly Tasks
    Qin, Yimin (Virginia Tech, 2023-06-14)
    Augmented Reality (AR) head-mounted display (HMD) provides users with an immersive virtual experience in the real world. The portability of this technology affords various information display options for construction workers that are not possible otherwise. The information delivered via an interactive user interface provides an innovative method to display complex building instructions, which is more intuitive and accessible compared with traditional paper documentations. However, there are still challenges hindering the practical usage of this technology at the construction jobsite. As a technical restriction, current AR HMD products have a limited field of view (FOV) compared to the human vision range. It leads to an uncertainty of how the obstructed view of display will affect construction workers' perception of hazards in their surrounding area. Similarly, the information displayed to workers requires rigorous testing and evaluation to make sure that it does not lead to information overload. Therefore, it is essential to comprehensively evaluate the impacts of using AR HMD from both perspectives of task performance and cognitive performance. This dissertation aims to bridge the gap in understanding the cognitive impacts of using AR HMD in construction assembly tasks. Specifically, it focuses on answering the following two questions: (1) How are task performance and cognitive skills affected by AR displays under complex working conditions? (2) How are moment-to-moment changes of mental workload captured and evaluated during construction assembly tasks? To answer these questions, this dissertation proposed two experiments. The first study tests two AR displays (conformal and tag-along) and paper instruction under complex working conditions, involving different framing scales and interference settings. Subjective responses are collected and analyzed to evaluate overall mental workload and situation awareness. The second study focuses on exploring an electroencephalogram (EEG) based approach for moment-to-moment capture and evaluation of mental workload. It uncovers the cognitive change on the time domain and provides room for further quantitative analyzing on mental workload. Especially, two frameworks of mental workload prediction are proposed by using (1) Long Short-Term Memory (LSTM) and (2) one-dimensional Convolutional Neural Network (1D CNN)-LSTM for forecasting EEG signal and, classifying task conditions and mental workload levels respectively. The approaches are tested to be effective and reliable for predicting and recognizing subjects' mental workload during assembly. In brief, this research contributes to the existing knowledge with an assessment of AR HMD use in construction assembly, including task performance evaluation and both subjective and physiological measurements for cognitive skills.
  • Thumbnail Image
    Human-Interactions with Robotic Cyber-Physical Systems (CPS) for Facilitating Construction Progress Monitoring
    Halder, Srijeet (Virginia Tech, 2023-08-23)
    Progress monitoring in construction involves a set of inspection tasks with repetitive in-person observations on the site. The current manual inspection process is time-consuming, inefficient, inconsistent, and has many safety risks to project inspectors. Cyber-Physical Systems (CPS) are networks of integrated physical and cyber components, such as robots, sensors, actuators, cloud computing, artificial intelligence, and the building itself. Introducing CPS for construction progress monitoring can reduce risks involved in the process, improve efficiency, and enable remote progress monitoring. A robotic CPS uses a robot as the core component of the CPS. But human interaction with technology plays an important role in the successful implementation of any technology. This research studied the human-centered design of a CPS from a human-computer interaction perspective for facilitating construction progress monitoring that puts the needs and abilities of humans at the center of the development process. User experience and interactions play an important role in human-centered design. This study first develops a CPS framework to autonomously collect visual data and facilitate remote construction progress monitoring. The two types of interactions occur between the human and the CPS – the human provides input for the CPS to collect data referred to as mission planning, and CPS provides visual data to enable the human to perform progress analysis. The interaction may occur through different modalities, such as visual, tactile, auditory, and immersive. The goal of this research is to understand the role of human interactions with CPS for construction progress monitoring. The study answers five research questions – a) What robotic CPS framework can be applied in construction progress monitoring? b) To what extent is the proposed CPS framework acceptable as an alternative to traditional construction progress monitoring? c) How can natural interaction modalities like hand gestures and voice commands be used as human-CPS interaction modalities for the proposed CPS? d) How does the human interaction modality between the proposed CPS and its user affect the usability of the proposed CPS? e) How does the human interaction modality between CPS and its user affect the performance of the proposed CPS?. To answer the research questions, a mixed-method-based methodology is used in this study. First, a systematic literature review is performed on the use of robots in inspection and monitoring of the built environment. Second, a CPS framework for remote progress monitoring is developed and evaluated in lab conditions. Third, a set of industry experts experienced with construction progress monitoring are interviewed to measure their acceptance of the developed CPS and to collect feedback for the evaluation of the CPS. Fourth, two methodologies are developed to use hand gesture and voice command recognition for human-CPS interaction in progress monitoring. Fifth, the usability and performance of the CPS are measured for identified interaction modalities through a human subject study. The human subjects are also interviewed post-experiment to identify the challenges they faced in their interactions with the CPS. The study makes the following contributions to the body of knowledge – a) key research areas and gaps were identified for robots in inspection and monitoring of the built environment, b) a fundamental framework for a robotic CPS was developed to automate reality capture and visualization using quadruped robots to facilitate remote construction progress monitoring, c) factors affecting the acceptance of the proposed robotic CPS for construction progress monitoring were identified by interviewing construction experts, d) two methodologies for using hand gestures and voice commands were developed for human-CPS interaction in construction progress monitoring, e) the effect of human interaction modalities on the usability and performance of the proposed CPS was assessed in construction progress monitoring through user studies, f) factors affecting the usability and performance of the proposed CPS with different interaction modalities were identified by conducting semi-structured interviews with users.
  • Thumbnail Image
    Impact of Interactive Holographic Learning Environment for bridging Technical Skill Gaps of Future Smart Construction Engineering and Management Students
    Ogunseiju, Omobolanle Ruth (Virginia Tech, 2022-07-25)
    The growth in the adoption of sensing technologies in the construction industry has triggered the need for graduating construction engineering students equipped with the necessary skills for deploying the technologies. For construction engineering students to acquire technical skills for implementing sensing technologies, it is pertinent to engage them in hands-on learning with the technologies. However, limited opportunities for hands-on learning experiences on construction sites and in some cases, high upfront costs of acquiring sensing technologies are encumbrances to equipping construction engineering students with the required technical skills. Inspired by opportunities offered by mixed reality, this study presents an interactive holographic learning environment that can afford learners an experiential opportunity to acquire competencies for implementing sensing systems on construction projects. Firstly, this study explores the required competencies for deploying sensing technologies on construction projects. The current state of sensing technologies in the industry and sensing technology education in construction engineering and management programs were investigated. The learning contents of the holographic learning environment were then driven by the identified competencies. Afterwards, a learnability study was conducted with industry practitioners already adopting sensing technologies to assess the learning environment. Feedback from the learnability study was implemented to further improve the learning environment after which a usability evaluation was conducted. To investigate the pedagogical value of the learning environment in construction education, a summative evaluation was conducted with construction engineering students. This research contributes to the definition of the domain-specific skills required of the future workforce for implementing sensing technologies in the construction industry and how such skills can be developed and enhanced within a mixed reality learning environment. Through concise outline and sequential design of the user interface, this study further revealed that knowledge scaffolding can improve task performance in a holographic learning environment. This study contributes to the body of knowledge by advancing immersive experiential learning discourses previously confined by technology. It opens a new avenue for both researchers and practitioners to further investigate the opportunities offered by mixed reality for future workforce development.
  • Thumbnail Image
    Leveraging Artificial Intelligence for Improving Students' Noticing of Practice during Virtual Site Visits
    Olayiwola, Johnson Tumininu (Virginia Tech, 2023-01-11)
    Complementing the theoretical concepts taught in the classroom with practice has been known to enhance students' contextual understanding of the subject matter. Exposing students to practical knowledge is crucial as employers are expressing discontent with the skills of newly hired graduates. In construction education, site visits have been identified as one of the most effective tools to support theory with practice. While site visits allow students to observe construction projects and engage with field personnel, numerous barriers limit its use as an effective educational tool. For instance, there are safety, cost, schedule, and weather constraints, in addition to the logistics of accommodating large class sizes. As a result, instructors employ videos of construction projects as an alternative to physical site visits. However, videos alone are insufficient to draw students' attention to essential practice concepts. Annotations can be used to attract students' attention to practical knowledge while reducing distractions and assumptions. Leveraging on the recent progress in computer vision techniques, this study presents an AI-annotated video learning tool that instructors can utilize to equip students with practice knowledge when there is limited access to physical construction sites. First, this study investigated the construction practice concepts that industry practitioners would want students to know when engaging them in site visits. Afterward, the design and development of the AI-annotated learning tool were guided by the identified practice concepts, cognitive theory of multimedia learning, and dual coding theory. To determine if the learning tool can call students' attention to annotated practice concepts in videos, a usability evaluation was conducted. Finally, this research investigated the influence of individual differences that could contribute to how learners notice practice concepts in videos. This study contributes to the body of knowledge by identifying what construction professionals notice about their work and what they would like students to notice about construction practice. This study reveals that annotations of learning contents in construction videos can direct students' focus to the annotated contents, thereby contributing to the cognitive theory of multimedia learning and dual coding theory. By leveraging machine learning classification algorithms, this research identified the extent to which individual differences such as gender, academic program, and cognitive load can be detected from the ways students notice information in construction videos. Results from this research provide opportunities for researchers to further advance the potential of annotated videos in the construction domain and other fields that employ video as a learning tool.
  • Thumbnail Image
    Understanding Underlying Risks and Socio-technical Challenges of Human-Wearable Robot Interaction in the Construction Industry
    Gonsalves, Nihar James (Virginia Tech, 2023-07-06)
    The construction industry, one of the largest employers of labor in the United States, has long suffered from health and safety issues relating to work-related musculoskeletal disorders. Back-related injuries are one of the most prevalent of all musculoskeletal disorders in the construction industry. Due to advancements in the field of wearable technologies, wearable robots such as passive back-support exoskeletons have emerged as a possible solution. Exoskeletons have the potential to augment human capacity, support non-neutral work positions, and reduce muscle fatigue and physical exertion. Current research efforts to evaluate the potential of exoskeletons in other industry sectors have been focused on outcome measures such as muscle activity, productivity, perceived discomfort and exertion, usability, and stakeholders' perspectives. However, there is scarce evidence regarding the efficacy of using exoskeletons for construction work. Furthermore, the risks and sociotechnical challenges of employing exoskeletons on construction sites are not well documented. Thus, through the lens of human-centric and socio-technical considerations, this study explores the prospects of adopting back-support exoskeletons in the construction industry. Firstly, a laboratory experiment was conducted to quantify the impact of using a passive exoskeleton for construction work in terms of muscle activity, perceived discomfort, and productivity. In order to investigate the acceptance of exoskeletons among construction workers and the challenges of adopting exoskeletons on construction sites, field explorations evaluating usability, perceived discomfort and exertion, social influence, and workers user perceptions were executed. Using sequential mixed methods approach, the stakeholders and factors (i.e., facilitators and barriers) critical for the adoption of exoskeletons on construction sites were investigated. Thereafter, by employing the factors and leveraging the constructs of the normalization process theory, an implementation plan to facilitate the adoption of passive exoskeletons was developed. The study contributes to the scarce body of knowledge regarding the extent to which exoskeletons can reduce ergonomic exposures associated with construction work. This study provides evidence of the perceptions of the contextual use of wearable robots, and workers' interaction with wearable robots on construction sites. The study contributes to the normalization process theory by showing its efficacy for the development and evaluation of implementation frameworks for construction industry. Furthermore, this study advances the socio-technical systems theory by incorporating all its subsystems (i.e., human, technology, organization and social) for investigating the potential of using a passive back support exoskeleton in the construction industry.
  • Thumbnail Image
    User-centered evaluations of multi-modal building interfaces
    Kianpour rad, Simin (Virginia Tech, 2025-01-31)
    In the evolving landscape of building systems and human-building interaction (HBI), the complexity of building interfaces has significantly increased, posing both challenges and opportunities for enhancing energy consumption, indoor environmental quality (IEQ), and building services. This dissertation, titled "User-centered Evaluation of Multi-modal Building Interfaces," delves into the realm of HBI by focusing on the user's experience and perception of multimodal building control interfaces, particularly the various visual modalities of Connected Thermostats (CTs). This body of work aims to support CTs' ongoing adoption, expansion, and performance through a user-centered perspective. The research is motivated by the observation that the design process in the current building industry often overlooks a human-centered approach, leading to a disconnection between occupants' needs and building interface design. This misalignment not only results in user dissatisfaction but also leads to a missed opportunity in leveraging smart building technologies to enhance building performance for achieving climate change mitigation goals. This research attempts to address the main identified gaps in the literature and AEC industry concerning 1) human interaction and perception of multimodal CT interfaces,2) the scarcity of knowledge in the field of human-computer-building interaction (HCBI) regarding the user study methods, 3) the exiting highly non-standard practices in the design of building interfaces. This research highlights 1) the necessity of a multimodal interaction approach, 2) robust mixed-methods User Experience (UX) summative evaluation studies, and 3) the need for standardization in HCBI. This body of work is grounded in the Technology Acceptance Model (TAM) and Human Information Processing (HIP) theories, aiming to foster the adoption of connected building controls with a special focus on usability by suggesting best practices in design and research. The methodology comprised three-step mixed-methods summative evaluation studies designed using a funnel approach to answer the general question: "How do users interact with connected thermostats, and how do these interactions inform our understanding of human-building interaction?": 1) The first and broadest study leveraged texting mining big data of user reviews to identify the general themes and patterns that affect the UX and acceptance of CTs. 2) The second study employed mixed-methods lab experiments to further focus on usability, being recognized as the most determining factor in the adoption of CTs in the first study. This study investigated human interaction with three of the most prevalent modalities of CTs: the Fixed Visual Display (FVD), the phone app, and the web portal. 3) The third study investigated human interaction with a specific visual aspect of UI of FVD and phone app modalities, the interface icons, with the goal of providing some data-driven guidelines for their standardization. Throughout the three studies, the dissertation employed and evaluated some novel and established HCI summative user evaluation methods, including a grounded theory approach for text mining and analyzing user-generated content, eye-tracking think-aloud protocol and contextual inquiry, A/B testing and NASA TLX and SUS surveys to evaluate UX, usability and mental workload. The dissertation outlined three discrete contributions: 1) It bridged some of the well-established UX research methods into HCBI and highlighted the potential of knowledge in the HCI field, 2) Provided guidance for human-centered design of multimodal building interfaces through identifying the main strengths, weaknesses, opportunities, and threats in UX of CTs, 3) Informed the standardization of UI of multimodal building interfaces.