Item Exposure issues in the Wagner Assessment Test: A Computer Adaptive Testing Approach
Wagner, Teresa Ann
MetadataShow full item record
The performance of the Wagner Assessment Test (WAT; Wagner & Harvey, in press) when administered using computer adaptive testing (CAT) methods was examined using the original 103-item WAT pool and an expanded 263-item pool. Although previous research (e.g., Wagner & Harvey, in press) has shown that the WAT outperforms the Watson Glaser Critical Thinking Appraisal (WGCTA; Watson & Glaser, 1980) in terms of test information, standard error, and susceptibility to successful guessing, it was hypothesized that a significant lengthening of the WAT item pool would be necessary in order to achieve acceptable CAT item-exposure rates. Using item parameters derived in an N = 423 college-student sample, Monte Carlo methods were used to conduct CAT sessions in an N = 5,000 sample of simulated raters; theta estimation accuracy and item-exposure levels at test lengths of 20, 40, and 60 items were examined. Results indicated that both b-blocking a-stratified (BAS) and Fisher maximum-information (FMI) methods of item selection exhibited strong convergence with the true theta scores; however, only BAS yielded acceptable item-exposure rates in CAT tests of these lengths.
- Doctoral Dissertations