Development of an Instrument to Evidence Knowledge Abstractions in Technological/Engineering Design-Based Activities

This document outlines the development of a Design Log Instrument (DLI) intended for use in identifying moments of abstraction as evidence of STEM content knowledge transfer. Many theoretical approaches to explaining knowledge transfer are rooted in a belief that transfer occurs through knowledge abstraction (Reed, Ernst, & Banerji, 1974; Gick & Holyoak, 1980, 1983). The DLI prompts participants to be reflective during technological/engineering design activities. During the development of this instrument, a three-phase multiple case: embedded design was used. Three distinct Phases accommodated the collection and analysis of data necessary for this investigation: Phase 1: Pilot Case Study, Phase 2: Establishing Content Validity, and Phase 3: Establishing Construct Validity. During Phase 3, data from the DLI was collected at each of seven work sessions from two design teams each working through different engineering problems. At the end of Phase 3, a comparison of abstractions found in DLI responses and observation data (Audio/Video transcripts) indicated the extent to which the DLI independently reflected those abstractions revealed in observations (Audio/Video transcripts). Results of this comparison showed that the DLI has the potential to be 68% reliable to reveal abstracted knowledge. Further analysis of these findings showed ancillary correlations between the percent abstractions found per DLI reflective prompt and the percent abstractions found per T/E design phase. Specifically, DLI Reflective Prompts 2 and 3 correlate with T/E Design Phases 3 and 4 (58% and 76% respectively of the total abstractions) which deal with design issues related to investigating the problem and developing alternate solutions. DLI Reflective Prompts 4 and 5 correlate with T/E Design Phases 5 and 6 (22% and 24% respectively of total abstractions) which deal with design issues related to choosing a solution and developing a prototype. Findings also indicate that there are highs and lows of abstraction throughout the T/E design process. The implications of these highs and lows are that specific phases of the T/E design process can be targeted for research and instruction. By targeting specific T/E design phases, a researcher or instructor can increase the likelihood of fostering abstractions as evidence of STEM content knowledge transfer.