Revisiting Rating Format Research: Computer-Based Rating Formats and Components of Accuracy

Abstract

Prior to 1980, most research on performance appraisal focused on rating formats. Since then, most performance appraisal research has focused on the internal processes of raters. This study redirects the focus back onto rating format with a critical eye towards rating accuracy. Ninety subjects read several hypothetical descriptions of teacher behavior and then rated the teachers on different dimensions of teaching performance using computer-based rating formats. It was found that rating format does affect some measures of rating accuracy. In addition, support was found for the viability of a new rating format. Graphic rating scales with no anchors received higher accuracy scores on certain measures of accuracy, higher ratings for liking of the rating format, higher levels of comfort with the rating format, and higher levels of interrater reliability than either BARS or graphic rating scales with numerical anchors. This study supports the ideas that rating format research should be reexamined with a focus on rating accuracy and that computer-based graphic scales with no anchors should be considered as an alternative to more traditional rating methods.