The effects of 16 variables on a telephone information system which uses synthetic speech

Abstract

Information systems that employ synthetic speech are emerging daily in the consumer market. However, many of these systems are being developed without first investigating the numerous factors that affect the design and usability of these systems. This study investigated the effects of 16 variables on a telephone information system which uses synthetic speech as the display modality. The information system was for a fictitious department store. Subjects telephoned the system and searched for information messages on specific store items. Upon hearing the message, subjects transcribed what they heard and rated their perceived difficulty in understanding the message, their confidence in correctly remembering the message, and their perceived difficulty in finding the store item in the system. Subject search performance measures were recorded during each search, and system evaluation subjective ratings were collected at the end of each experimental session.

A Hadamard 32x32 matrix design was used in this screening study to test efficiently the main effects of the 16 variables on 23 measures of user performance. Only 32 data points were required to evaluate the variables in the screening study. The analyses identified 8 variables (speech rate, menu organization, number of targets, wallet guide, menu feedback, background music, subject age, and subject gender) as having a significant effect in at least two tests; 4 variables (voice type, pause/resume, repeat keyword, and command feedback) as having a significant effect in one test; and 4 variables (input timeout, system response time, selection feedback, and spell-out keyword) that did not have a significant effect in any test. The analyses also assessed the worth of the 12 dependent measures in providing meaningful test results.