On comparing different tests of the same hypothesis

Abstract

This thesis presents two alternative procedures for comparing standard and quick tests of a null hypothesis H_o. This comparison is usually made by plotting the power curves of each test for a fixed Type I error. However, the power curves give only an indication of the individual performances of each test and not of the extent to which they agree when applied to the same problem. The procedures discussed in this paper deal with determining this degree of agreement.

The first method determines the probability, P, that the quick test leads to a significant result at a level α given that the standard test is just significant at level α. If the standard and quick tests are based on the statistics u₁ and u₂ , respectively, the second approach determines the level of significance corresponding to the expected value of u₂ given that u₁ is just significant at level α. This level of significance is termed the "equivalent Type I error" of the quick test and denoted by γ.

Both methods are applied to compare tests of location, dispersion, and the paired t-test with the sign test, all in samples taken from a normal population. In the first two cases, values of P and γ are given for different sample sizes, and in the third case only the “equivalent Type I error" of the sign test is given, P being rather difficult to evaluate.