Restrictive ranking

TR Number



Journal Title

Journal ISSN

Volume Title


Virginia Polytechnic Institute


This dissertation is a study of certain aspects of restricted ranking, a method intended for use by a panel of m judges evaluating the relative merits of N subjects, candidates tor scholarships, awards, etc. Each judge divides the N subjects into R classes, so that n₁ individuals receive a grade i (i = 1, 2, R; Σnᵢ = N) where the R numbers nᵢ are close to N/R (nᵢ = N/R when N is divisible by R) and are preassigned and the same for all judges. When this method is used, all subjects are treated alike, the grading system is the same for all judges and the grades of each judge are given equal weight. Equally important, the meaning of a particular grade is clear to each judge and the same for each judge.

Under the null hypothesis that all nR = N subjects are of equal merit, tests of significance are developed to determine whether (1) a particular individual is superior or inferior to the rest of the subjects; (2) two particular subjects are of equal merit; (3) the individuals with the highest and lowest scores are respectively superior and interior to the rest of the subjects and (4) the nR subjects form a homogeneous group. The critical values of the test statistics for (1), (2) and (3) are tabled for small to moderate values of m, an approximation based on the asymptotic normality of the appropriate test statistic proving suitable for large m. The test of homogeneity (4) employs a sum of squares of subjects’ scores which is shown to be asymptotically distributed for m→∞ as chi-square with nR-1 degrees of freedom. For the special case of complete ranking (R=N), this statistic is identical to one proposed by Friedman (1937) form rankings.

The behavior of two of these tests is theoretically investigated for the non-null case of nR-1 subjects having equal merit and one "outlying" subject whose merit exceeds the others. The assumption is made that each judge j assigns a grade to every subject i on the basis of a "subjective random variable" xᵢⱼ with mean equal to the "true" merit of subject i and that the distribution of xᵢⱼ is the same for all j. The probability, P(δ), that subject #1 with true mean differing from the others by an amount δ would receive a significantly high score according to the test for outliers is obtained and presented graphically as a function of for xᵢⱼ distributed as (1/2) sech² (x-δ) and also as N ( δ, 1). Using a result due to Hannan (1956), an expression for the asymptotic relative efficiency of the chi-squared homogeneity test for restricted vs. complete ranking for the aforementioned non-null case is obtained and values of this A.R.E. for 2 ≤n≤10 and. 2≤R≤8 are tabled. This A.R.E. is found to be at least 0.9 for all cases where n≤10 and R≥4.

A further comparison of the performances of restricted (R) and complete (C) ranking is made by way of some simulation studies performed on a high speed digital computer tor the non- null ease where xᵢⱼ is normally distributed with unit variance and a mean δ₁ having as many as three different possible values. The complete and restricted ranks assigned by the jth judge to the ith subject are assigned on the basis of the value of xᵢⱼ obtained by experimental sampling using a random normal number generator in the computer program. A group of Nₛ subjects with the highest rank sums for (R) and for (C) are then selected in each study. The observed difference in true means between selected and remaining groups is then used as a measure of goodness of the two selection procedures. The results of these studies are presented graphically, displaying a very close agreement between (R) and (C) in all instances.