Show simple item record

dc.contributor.authorSienknecht, Jr., Ronald Theodoreen_US
dc.date.accessioned2014-03-14T20:41:50Z
dc.date.available2014-03-14T20:41:50Z
dc.date.issued1999-06-28en_US
dc.identifier.otheretd-072399-154928en_US
dc.identifier.urihttp://hdl.handle.net/10919/34129
dc.description.abstractThis research clarified existing inconsistencies in self-assessment literature, and added to the body of knowledge for rating effectiveness of organizational assessments by defining relationships among rating effectiveness criteria (ratee, rater, rating scale, rating process) and measures (interrater reliability, halo error, leniency and severity, range restriction) based on extensive literature review. A research framework was developed from this review, and was employed in computing rating effectiveness measures at the individual (i.e., examiner or eight rating scale dimensions) and sector (e.g., Private Manufacturing Sector, Private Service Sector, Public Local Sector, Public State & Federal Sector) levels for a State Quality Award (SQA) using data from the 1998 applications. Interrater reliability (measured by intraclass correlations for each rating scale dimension) was low to moderate, and differed by dimension. Halo error (measured by the determinant of the dimension intercorrelation matrices for each examiner) was present for all examiners. Leniency and severity (measured by presence of statistically significant Rater main effect for each dimension) was present for 11 of 32 cases, and differed by dimension. Range restriction (measured by variance analysis for each dimension) was present for 22 of 32 cases, and differed by dimension. A post-hoc principle component analysis indicated poor internal reliability for the rating scale. To improve, the SQA should replace the existing rating scale and provide in-depth training on all elements of the rating process. The importance of the SQA using boxplots, histograms, and rating effectiveness measures to make fully informed decisions was discussed.en_US
dc.publisherVirginia Techen_US
dc.relation.haspartthesis.pdfen_US
dc.rightsI hereby grant to Virginia Tech or its agents the right to archive and to make available my thesis or dissertation in whole or in part in the University Libraries in all forms of media, now or hereafter known. I retain all proprietary rights, such as patent rights. I also retain the right to use in future works (such as articles or books) all or part of this thesis or dissertation.en_US
dc.subjectHalo Erroren_US
dc.subjectOrganizational Assessmenten_US
dc.subjectQuality Awarden_US
dc.subjectRating Effectivenessen_US
dc.subjectInterrater Reliabilityen_US
dc.subjectLeniency and Severityen_US
dc.subjectRange Restrictionen_US
dc.titleAn Empirical Analysis of Rating Effectiveness for a State Quality Awarden_US
dc.typeThesisen_US
dc.contributor.departmentIndustrial and Systems Engineeringen_US
dc.description.degreeMaster of Scienceen_US
thesis.degree.nameMaster of Scienceen_US
thesis.degree.levelmastersen_US
thesis.degree.grantorVirginia Polytechnic Institute and State Universityen_US
thesis.degree.disciplineIndustrial and Systems Engineeringen_US
dc.contributor.committeechairVan Aken, Eileen M.en_US
dc.contributor.committeememberKoelling, Charles Patricken_US
dc.contributor.committeememberColeman, Garry D.en_US
dc.identifier.sourceurlhttp://scholar.lib.vt.edu/theses/available/etd-072399-154928/en_US
dc.date.sdate1999-07-23en_US
dc.date.rdate2000-07-27
dc.date.adate1999-07-27en_US


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record