Usability evaluation of a fault tree software user documentation

To incorporate users' opinions into the evaluation phase early in the software documentation development stage, the critical incident technique was used to identify usability problems in a fault free software user document. The critical incidents were used to modify the original document to improve its usability. To test whether the modified document was better in usability than the original document, an experiment was conducted to compare objective measures and subjective ratings.

Four objective measures included number of errors, task completion time, document reading time, and number of personal helps requested. In addition, subjective ratings on ease of use, accuracy of information, inconsistencies, ease of learning, completeness, helpfulness of figures, and ease of understanding were compared between the two documents.

The analyses showed that for 3 of 4 objective measures and 7 out of 9 subjective ratings, the new document was better and easier to use. In some cases, this difference was task specific. Generally, easier tasks accounted for better objective measures and more favorable subjective ratings.