Holistic versus Decomposed Rating Scales: Which causes higher levels of cognitive load?
Watt, Alisa H.
MetadataShow full item record
The purpose of this study was to help explore the assumptions in job analysis literature involving whether holistic or decomposed job analysis items lead to a higher level of cognitive load for raters. The main study, involving 303 undergraduate university students, was a 2 (Type of measure: holistic or decomposed) X 2 (Level of extraneous cognitive load: additional load or no additional load) within-subjects design. The 160 decomposed items analyzed in this study were pilot tested to ensure that they would correlate with the 17 holistic items. Under the additional cognitive load condition, participants memorized an 8-digit number, and then were asked to recall and recognize this number upon the completion of the rating task (this manipulation was performed for both the holistic and decomposed measures). Stability of ratings across conditions and interrater agreement were used as dependent measures. Results indicated that the holistic items (r=.74) had higher levels of stability across cognitive load conditions than did the decomposed items (r=.66). The levels of interrater agreement were not significantly different between three of the four conditions. In partial support of Butler and Harvey (1988), the level of interrater agreement for the Holistic additional cognitive load condition (r*wg=.33) was significantly lower than the interrater agreement for the remaining three conditions. The pattern of results supported prior research (Cornelius & Lyness, 1980; Lyness & Cornelius, 1982) indicating that, depending on the criteria being used, holistic items do not necessarily cause a higher level of cognitive load for raters than do decomposed items.
- Masters Theses