An Empirical Analysis of Rating Effectiveness for a State Quality Award

Simple item page
dc.contributor.authorSienknecht, Ronald Theodore Jr.en
dc.contributor.committeechairVan Aken, Eileen M.en
dc.contributor.committeememberKoelling, C. Patricken
dc.contributor.committeememberColeman, Garry D.en
dc.contributor.departmentIndustrial and Systems Engineeringen
dc.date.accessioned2014-03-14T20:41:50Zen
dc.date.adate1999-07-27en
dc.date.available2014-03-14T20:41:50Zen
dc.date.issued1999-06-28en
dc.date.rdate2000-07-27en
dc.date.sdate1999-07-23en
dc.description.abstractThis research clarified existing inconsistencies in self-assessment literature, and added to the body of knowledge for rating effectiveness of organizational assessments by defining relationships among rating effectiveness criteria (ratee, rater, rating scale, rating process) and measures (interrater reliability, halo error, leniency and severity, range restriction) based on extensive literature review. A research framework was developed from this review, and was employed in computing rating effectiveness measures at the individual (i.e., examiner or eight rating scale dimensions) and sector (e.g., Private Manufacturing Sector, Private Service Sector, Public Local Sector, Public State & Federal Sector) levels for a State Quality Award (SQA) using data from the 1998 applications. Interrater reliability (measured by intraclass correlations for each rating scale dimension) was low to moderate, and differed by dimension. Halo error (measured by the determinant of the dimension intercorrelation matrices for each examiner) was present for all examiners. Leniency and severity (measured by presence of statistically significant Rater main effect for each dimension) was present for 11 of 32 cases, and differed by dimension. Range restriction (measured by variance analysis for each dimension) was present for 22 of 32 cases, and differed by dimension. A post-hoc principle component analysis indicated poor internal reliability for the rating scale. To improve, the SQA should replace the existing rating scale and provide in-depth training on all elements of the rating process. The importance of the SQA using boxplots, histograms, and rating effectiveness measures to make fully informed decisions was discussed.en
dc.description.degreeMaster of Scienceen
dc.identifier.otheretd-072399-154928en
dc.identifier.sourceurlhttp://scholar.lib.vt.edu/theses/available/etd-072399-154928/en
dc.identifier.urihttp://hdl.handle.net/10919/34129en
dc.publisherVirginia Techen
dc.relation.haspartthesis.pdfen
dc.rightsIn Copyrighten
dc.rights.urihttp://rightsstatements.org/vocab/InC/1.0/en
dc.subjectHalo Erroren
dc.subjectOrganizational Assessmenten
dc.subjectQuality Awarden
dc.subjectRating Effectivenessen
dc.subjectInterrater Reliabilityen
dc.subjectLeniency and Severityen
dc.subjectRange Restrictionen
dc.titleAn Empirical Analysis of Rating Effectiveness for a State Quality Awarden
dc.typeThesisen
thesis.degree.disciplineIndustrial and Systems Engineeringen
thesis.degree.grantorVirginia Polytechnic Institute and State Universityen
thesis.degree.levelmastersen
thesis.degree.nameMaster of Scienceen

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
thesis.pdf
Size:
1.74 MB
Format:
Adobe Portable Document Format
Download

Collections

Masters Theses