Effectiveness of a program-specific assessment instrument for a department of clothing and textiles

Abstract

The purposes of this study were (1) to determine the validity, reliability, and item quality of a program-specific, assessment instrument designed to measure student mastery of core knowledge in the discipline, (2) to evaluate the effectiveness of the instrument for determining student outcomes from the value-added perspective, and (3) to determine whether performance on the instrument and other measures of achievement was affected by student involvement characteristics.

The instrument contained 100 multiple-choice items relating to five core courses. For portions of the study, the instrument was divided into five subtests and administered as a pretest on the first day of classes in the five core courses and as a post test on the last day of classes. The instrument in its full form was administered to upper class clothing and textiles majors and to under class clothing and textiles majors. Interviews related to the instrument were conducted with a subset of students who had taken the subtests and the full exam and with faculty who taught the courses.

The split-half and the KR-20 reliability estimates for internal consistency of the full test were above .70. The increase in student performance from pretest to posttest and the correlation between information obtained from the interviews with the results of the item analyses demonstrated the instrument's content validity. The significant difference between scores received by upper class majors and scores received by underclass majors on the instrument, and the correlations between course grades with scores on the instrument and final exam grades with scores on the instrument demonstrated the construct validity of the instrument. The ability of the instrument to measure value-added knowledge was ascertained by the increase in student performance from pretest to post test.

Further data analyses revealed a pattern regarding factors which affect student achievement. Student involvement in organizations and employment had a positive and negative effect on various measures of student achievement.

Although the validity and reliability of the instrument were demonstrated, improvements in item quality and representation of course objectives on the core knowledge exam are needed.