A formative evaluation of CU-SeeMe

Abstract

CU-SeeMe is a video conferencing software package that was designed and programmed at Cornell University. The program works with the TCP/IP network protocol and allows two or more parties to conduct a real-time video conference with full audio support. In this paper we evaluate CU-SeeMe through the process of Formative Evaluation. [3] [9] [16] [24] We first perform a Critical Review of the software using a subset of the Smith and Mosier Guidelines for HumanComputer Interaction. [23] Next, we empirically review the software interface through a series of benchmark tests [3] that are derived directly from a set of scenarios. The scenarios attempt to model real world situations that might be encountered by an individual in the target user class.

Designing benchmark tasks becomes a natural and straightforward process when they are derived from the scenario set. Empirical measures are taken for each task, including completion times and error counts. These measures are accompanied by critical incident analysis [2] [7] [13] which serves to identify problems with the interface and the cognitive roots of those problems.

The critical incidents reported by participants are accompanied by explanations of what caused the problem and why. This helps in the process of formulating solutions for observed usability problems. All the testing results are combined in the Appendix in an illustrated partial redesign of the CU-SeeMe interface.