Interrater Agreement of Incumbent Job Specification Importance Ratings: Rater, Occupation, and Item Effects

Simple item page
dc.contributor.authorBurnkrant, Steven Richarden
dc.contributor.committeechairHauenstein, Neil M. A.en
dc.contributor.committeememberFoti, Roseanne J.en
dc.contributor.committeememberHarvey, Robert J.en
dc.contributor.committeememberCarlson, Kevin D.en
dc.contributor.committeememberStephens, Robert S.en
dc.contributor.departmentPsychologyen
dc.date.accessioned2014-03-14T20:17:38Zen
dc.date.adate2003-10-27en
dc.date.available2014-03-14T20:17:38Zen
dc.date.issued2003-10-23en
dc.date.rdate2003-10-27en
dc.date.sdate2003-10-25en
dc.description.abstractDespite the importance of job specifications to much of industrial and organizational psychology, little is known of their reliability or validity. Because job specifications are developed based on input from subject matter experts, interrater agreement is a necessary condition for their validity. The purpose of the present research is to examine the validity of job specifications by assessing the level of agreement in ratings and the effects of occupational tenure, occupational complexity, and the abstractness of rated worker requirements. Based on the existing literature, it was hypothesized that (1) agreement will be worse than acceptable levels, (2) agreement will be higher among those with longer tenure, (3) agreement will be lower in more complex occupations, (4) the effect of occupational tenure will be more pronounced in complex than simple occupations, (5) agreement will be higher on more abstract items, and (6) agreement will be lowest for concrete KSAOs in complex occupations. These hypotheses were tested using ratings from 38,041 incumbents in 61 diverse occupations in the Federal government. Consistent with Hypothesis 1, agreement failed to reach acceptable levels in nearly every case, whether measured with the awg or various forms of the rwg agreement indices. However, tenure, occupational complexity, and item abstractness had little effect on ratings, whether agreement was measured with rwg or awg. The most likely explanation for these null findings is that the disagreement reflected a coarse classification system that overshadowed the effects of tenure, complexity, and abstractness. The existence of meaningful subgroups within a single title threatens the content validity of job specifications: the extent to which they include all relevant and predictive KSAOs. Future research must focus on the existence of such subgroups, their consequences, and ways of identifying them.en
dc.description.degreePh. D.en
dc.identifier.otheretd-10252003-132129en
dc.identifier.sourceurlhttp://scholar.lib.vt.edu/theses/available/etd-10252003-132129/en
dc.identifier.urihttp://hdl.handle.net/10919/29368en
dc.publisherVirginia Techen
dc.relation.haspartBurnkrant.pdfen
dc.rightsIn Copyrighten
dc.rights.urihttp://rightsstatements.org/vocab/InC/1.0/en
dc.subjectJob specificationen
dc.subjectCompetencyen
dc.subjectValidityen
dc.subjectKSAOen
dc.subjectJob analysisen
dc.subjectInterrater agreementen
dc.titleInterrater Agreement of Incumbent Job Specification Importance Ratings: Rater, Occupation, and Item Effectsen
dc.typeDissertationen
thesis.degree.disciplinePsychologyen
thesis.degree.grantorVirginia Polytechnic Institute and State Universityen
thesis.degree.leveldoctoralen
thesis.degree.namePh. D.en

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Burnkrant.pdf
Size:
655.2 KB
Format:
Adobe Portable Document Format
Download

Collections

Doctoral Dissertations