Empirical Methods for Evaluating Video-Mediated Collaborative Work

Advancements in computer technology are making video conferencing a viable communication medium for desktop computers. These same advancements are changing the structure and means by which information workers conduct business. From a human factors perspective, however, the study of new communication technologies and their relationships with end users presents a challenging research domain. This study employed two diverse research approaches to the problem of reduced video frame rate in desktop video conferencing. In the first study, a psychophysical method was used to evaluate video image quality as a function of frame rate for a series of different scenes. Scenes varied in terms of level of detail, velocity of panning, and content. Results indicate that for most scenes, differences in frame rate become less detectable above approximately 10 frames per second (fps), suggesting a curvilinear relationship between image quality and frame rate. For a traditional conferencing scene, however, a linear increase in frame rate produced a linear improvement in perceived image quality. High detail scenes were perceived to be of lower quality than the low detail scenes, while panning velocity had no effect. In the second study, a collection of research methods known as ethnography was used to examine long-term use of desktop video by collaborators in a real work situation. Participants from a graduate course met each week for seven weeks and worked on a class project under one of four communication conditions: face-to-face, 1 fps, 10 fps, and 25 fps. Dependent measures included interviews, questionnaires, interaction analysis measures, and ethnomethodology. Recommendations are made regarding the utility and expense of each method with respect to uncovering human factors issues in video-mediated collaboration. It is believed that this research has filled a significant gap in the human factors literature of advanced telecommunications and research methodology.