Browsing by Author "Davis, Kirsten A."
Now showing 1 - 6 of 6
Results Per Page
Sort Options
- Assessing learning processes rather than outcomes: using critical incidents to explore student learning abroadDavis, Kirsten A.; Knight, David B. (Springer Nature, 2022-03-18)There is an increasing emphasis on assessing student learning outcomes from study abroad experiences, but this assessment often focuses on a limited range of outcomes and assessment methods. We argue for shifting to assessing student learning processes in study abroad and present the critical incident technique as one approach to achieve this goal. We demonstrate this approach in interviews with 79 students across a range of global engineering programs, through which we identified 173 incidents which were analyzed to identify common themes. This analysis revealed that students described a wide range of experiences and outcomes from their time abroad. Students’ experiences were messy and complex, making them challenging to understand through typical assessment approaches. Our findings emphasize the importance of using a range of assessment approaches and suggest that exploring students’ learning processes in addition to learning outcomes could provide new insights to inform the design of study abroad programs.
- Assessing the Impact of IRES on Researchers, Research Outcomes, and Students: A Case Study ApproachKnight, David B.; Sanderlin, Nicole; Davis, Kirsten A. (Virginia Tech, 2020-09)IRES programs seek to develop globally connected future researchers while also facilitating broader long-term collaborations between U.S.-based and international research groups. Faculty and student exchanges are expected to result in international linkages between the researchers that bring new insights and methods to U.S.-based research projects, leading to transformative research. In this project, we conducted a comprehensive evaluation of IRES programs through a case study analysis. The nine selected cases focused specifically on engineering-focused IRES experiences for ease of comparison. Case studies included interviews with all U.S. PI’s, a selection of additional international and U.S. faculty researchers, and select student alumni for a total of 59 interviews across the nine sites. Our analysis highlights the variety of approaches different IRES programs have taken, which we hope will be helpful for institutions proposing new IRES programs as well as for NSF program officers who work with IRES PI teams. We highlight best practices in developing research exchanges, the return on investment of such programs, as well as ideas from program leaders and participants in how to structure such experiences for students as well as faculty and institutions both on the U.S. and international sides to achieve a range of outcomes.
- Comparing Self-Report Assessments and Scenario-Based Assessments of Systems Thinking CompetenceDavis, Kirsten A.; Grote, Dustin; Mahmoudi, Hesam; Perry, Logan; Ghaffarzadegan, Navid; Grohs, Jacob; Hosseinichimeh, Niyousha; Knight, David B.; Triantis, Konstantinos (Springer, 2023-03)Self-report assessments are used frequently in higher education to assess a variety of constructs, including attitudes, opinions, knowledge, and competence. Systems thinking is an example of one competence often measured using self-report assessments where individuals answer several questions about their perceptions of their own skills, habits, or daily decisions. In this study, we define systems thinking as the ability to see the world as a complex interconnected system where different parts can influence each other, and the interrelationships determine system outcomes. An alternative, less-common, assessment approach is to measure skills directly by providing a scenario about an unstructured problem and evaluating respondents' judgment or analysis of the scenario (scenario-based assessment). This study explored the relationships between engineering students' performance on self-report assessments and scenario-based assessments of systems thinking, finding that there were no significant relationships between the two assessment techniques. These results suggest that there may be limitations to using self-report assessments as a method to assess systems thinking and other competencies in educational research and evaluation, which could be addressed by incorporating alternative formats for assessing competence. Future work should explore these findings further and support the development of alternative assessment approaches.
- Information Technology Change in the Architecture, Engineering, and Construction Industry: An Investigation of Individuals' ResistanceDavis, Kirsten A. (Virginia Tech, 2004-04-23)This research project investigates individuals' resistance to change brought about by new information technology implementation in the Architecture, Engineering, and Construction (AEC) industry. By understanding how individual participants resist and adapt to change, their resistance can be better accommodated by the organization in the adoption of new information technology within the AEC industry. This enables researchers and practitioners to understand how new technologies should be introduced within organizations. A social architecture factor model associated with impeding/promoting use of information technologies was created based on a literature review of change management theory on resistance to change and attitude-behavior connections. In Phase I of the research, the personality traits and behavioral characteristics individuals included in the original model were reduced to a smaller number of variables indicative of resistance to information technology change. A revised social architecture factor model was created after this reduction. The variable reduction and revised model were based on data collected from a 50-person sample of the AEC population. At the conclusion of Phase I, a Resistance to Change Index (RTCI) was created, enabling estimations of the intensity of resistance an individual is likely to exhibit using the personality traits and behavioral characteristics kept in the revised social architecture factor model. Phase II of the research investigated relationships between the RTCI and demographics of the individual using a 156-person sample of the AEC population. This phase of the research determined whether different demographic groups within the AEC population exhibited differences in their RTCI. The data analysis found several demographic groups that were different in their likelihood of resistance, including profession, gender, computer understanding and experience, and awareness of past or future changes occurring in their company. Age and education level were expected to have relationships with RTCI, based on industry stereotypes. The data analysis found that these stereotypes have no scientific basis. Two other stereotypes, gender and computer understanding and experience, were supported by the data analysis, however.
- Pursuing Intentional Design of Global Engineering Programs: Understanding Student Experiences and Learning OutcomesDavis, Kirsten A. (Virginia Tech, 2020-05-05)As engineering work becomes more globalized, it is important to prepare students to succeed in a global engineering workforce. Prior research has explored different factors important to the design of global programs generally, but less is known about the experiences and learning outcomes of engineering students while abroad. Because this population has historically been underrepresented in global programs, few research-based practices have been identified to support the intentional design of global experiences for engineering students. This dissertation seeks to provide such insights through three studies that address the following components of global engineering program design: 1) the assessment of Global Engineering Competency (GEC) through use of a series of scenarios based in the Chinese context; 2) the relationship between program destination and students’ experiences and reported learning outcomes; and 3) the experiences that engineering students identify as most significant during their time abroad. I used a combination of quantitative, qualitative, and mixed research methods to explore students’ learning and experiences in global engineering programs. The results of these studies revealed that engineering students have a wide variety of learning outcomes and experiences as a result of global programs, that great care should be exercised in choosing and interpreting assessment methods for global programs, and that the design of global engineering programs should be informed by characteristics of both the student participants and the program destination. Overall, the results of this dissertation highlight the importance of intentionality when designing global engineering programs to better prepare students for success in a global work environment.
- Resistance to IT Change in the AEC Industry: An Individual Assessment ToolDavis, Kirsten A.; Songer, Anthony D. (2008)Numerous IT adoption studies within the AEC industry identify issues with individuals resisting IT changes. Current change models often only look at organizations and tasks and frequently neglect the individuals involved. The limitations in existing change models and the criticality of people issues in the successful implementation of change necessitates the investigation of individual resistance to IT change. Change management theory and attitude-behavior connections provide a framework to study variables associated with impeding/promoting the use of technologies. Data collected from a 50-person sample of the AEC population allowed reductions of the attitudes, fears, and beliefs variables. Reducing the variables indicative of resistance to information technology change facilitated the creation of a detailed social architecture factor model. Subsequently, a Resistance to Change Index (RTCI) was created, enabling estimations of the intensity of resistance an individual is likely to exhibit using the personality traits and behavioral characteristics identified in the revised social architecture factor model. The RTCI assists practitioners in developing new technology implementation plans. The RTCI also enables researchers to understand how individual participants resist and adapt to change allowing the development of enhanced organizational adoption models for new technology implementation within the building industry.