Empirical Evaluation of a Technology-rich Learning Environment
MetadataShow full item record
In the fall of 1996, the Computer Science Department at Virginia Tech initiated a joint project with a local school district, to determine how ready access to networked computing in the fifth grade would affect students. Called the PCs for Families (PCF) project, its goal was to learn what could be achieved if technology access, support, and curriculum integration could be eliminated as obstacles or constraints in the classroom and at home. A technology-rich classroom was created, with the classroom teacher trained in constructivist teaching practices and technology integration by a master teacher. Network computers were found on every desktop, with scanners, digital cameras, and other technologies scattered throughout the room. A computer was sent home with each child and teacher, and as much support as necessary was provided to all program participants, including parents. As part of this research, a yearlong field experiment was undertaken to explore the effects of the PCF intervention on the third cohort of students participating in the project. Macroergonomics served as the theoretical framework for the experiment, which focused on the in-depth, systematic assessment of those quantitative changes that resulted from exposure to the PCF fifth-grade network classroom. Students participating in the field research were randomly selected from the larger pool of students eligible for the PCF project at the school. Selected students were randomly assigned to either to the PCF fifth-grade classroom or the standard fifth-grade classroom, which served as a control group. To first-time visitors walking into the PCF network classroom, the classroom bore little resemblance to its more traditional counterparts. However, the functioning of the PCF classroom was in many ways indistinguishable from that of its traditional counterparts. The yearly average for computer use in the PCF classroom was 4.275 hours, with computer use in the PCF classroom exceeding the three hours of computer laboratory time allotted to the control class only during the last 12 weeks of school. When used, the technology functioned as an electronic replacement for materials commonly found in traditional settings. Observers reported the pedagogy remained steadfastly teacher-centered and didactic. Despite limited utilization of the computer during classroom hours, analysis of individual, academic measures indicated PCF students made significantly greater gains than control students only on standardized writing tests. PCF students also performed significantly better than control students on measures related to technology skills. Boys in the PCF classroom also made greater improvements in their attitudes towards school than boys in the control classroom. At home, PCF students were found to interact with computer technology more often than their control counterparts. Despite lower overall home use, control students reported spending more time playing computer games than PCF students. Correlational analyses indicated significant linear relationships between changes in student performance, student entry characteristics, and home computer use variables. Student previous achievement was by far the strongest predictor of student SOL test performance, with computer use only linked to student standardized test performance on the writing and mathematics sections. As the number of email messages sent by the student increased, their writing performance increased with email usage accounting for almost ten percent of the total variance in the writing score. The only other computer use measure significantly associated with test performance was student self-reports of computer use, which accounted for less than four percent of the total variance in mathematics test performance. Computer use was associated more strongly with changes in student motivation. Student self-reports of home computer use accounted for fully 30 percent of the variance in changes on the school motivation survey. Analyses of data from the PCF proxy server suggest that student web browsing overshadows other home Internet activities, with email taking precedence over chat. Further, unlike chat or email, family web usage was sustained long after students left the PCF classroom. Over 68 percent of family web usage each week was attributable to student, not family, characteristics suggesting students play a large role in determining family usage. Academic information finding provides a plausible explanation for these results, with family web usage declining somewhat during summer months when students were not in school. Stability of both web and email use was relatively high among students. In keeping with critical mass theory, student email use increased when other students used email. However, social variables were not found to have a significant effect on web usage. Girls were found to make greater use of email than boys, with this research suggesting highly visual students used email more often. The field research also found a significant increase in student self-reports of musculoskeletal problems among the PCF students. A year-end examination of workstation fit found seat and monitor heights an average of two inches higher than the corresponding student dimensions. A participatory design study was used to elicit conceptions of computer workstations from PCF students, teachers, and parents. Children were interested in gaining greater control over the workstation, both in terms of individual technology and adjustability of furniture. Parents, however, focused on improving the richness of an individual student's workspace and de-emphasized collaborative work. Teacher opinions diverged more than other groups with designs strongly influenced by pedagogic beliefs. Results from the field study provide evidence that macroergonomic methodologies for analysis and design of work systems are extensible to classroom systems, and provide a systematic framework for examining issues related to the introduction of classroom computing technology. A critical element of any successful effort to integrate technology into the curriculum is access to adequate classroom technology and support; however, as this research illustrates, they are not sufficient to ensure successful integration. This research demonstrates other forces are at work, and in keeping with macroergonomic theory, key to the success of such an effort is the "fit" between the new technology and the characteristics of the classroom system, especially those of the teacher who effectively functions as the gatekeeper for the technology.
- Doctoral Dissertations