A behavioral evaluation of command-selection aids for inexperienced computer users/

Two experiments were conducted to determine the feasibility of providing online command-selection aids to novice users of an information retrieval system. The results of the first experiment revealed a difference in the mean and variability of search performance between novice and expert computer users. Half of the novices were performing much like experts, while the rest of the sample was extremely slow. These slower novices were using inefficient scrolling strategies and appeared to be unfamiliar with the structure of the database.

The second experiment evaluated whether novices could be assisted or trained with command-selection aids developed from the behavior of experts. The command-selection aids were defined in a 3 X 3 mixed factor design with type of model (frequency, sequence, or plan-based) as the between-subjects variable and dialogue initiative (user, computer, or mixed) as the within-subjects variable. The frequency and sequence models presented and ranked search procedures based on a command-usage profile and a command-transition matrix, respectively. The plan-based model presented an ordered set of search procedures with verbal explanations. All models were constructed for groups of homogeneous search problems selected by a sorting and cluster analysis. The three dialogue-initiatives determined whether the user, the computer, or both the user and computer controlled presentation of advice. Administration of the dialogue initiatives was completely counterbalanced and was followed by a final unaided transfer session.

As a result of receiving online aiding, the wide ranging search performance of novice subjects was improved both during assistance and transfer. Performance of aided novices was superior to the slow novices and equal to the fast novices and experts. All three command-selection models were equally effective, with exception of the sequence model which sometimes presented frequent and complicated advice. Of the dialogues, mixed-initiated advice was ineffective during the first aiding session possibly due to the difficulties novices faced deciding whether to receive the suggested assistance. The conclusion of the study was that online command—selection aids can be effective if providing appropriate feedback and minimizing the amount of dialogue in aiding.