Development and validation of a methodology for comprehensive performance assessment of complex tasks

Abstract

A new task analysis methodology was developed to provide objective information on complex tasks. A complex task was broken down into observable elements and unobservable elements that were inferred to have taken place in support of the observable actions. Subject matter experts (SMEs) were used to assist in this breakdown. Additionally, guidelines for specifying the level of detail in the task analysis breakdown were developed to help objectify the analysis. A simulation model framework then was built of the task elements. Personnel proficient in the task were observed during work, and objective data on their observable actions were collected. These data then were used to provide numeric input to a simulation model. The simulation was run, and the results of the model of performance compared to the observed performance data. The model was altered at that point to reflect lessons learned during data collection. The process yielded a model that accurately reflects human performance on the task. Variations on the model based on a conceptual understanding of operators strategies also correlated well with observed performance, indicating the value of the methodology for building an understanding of the motivations critical to successful task performance.