The Orange County Transportation Authority (OCTA) conducts periodic pre-qualification studies to evaluate the proficiency of pavement inspectors in performing pavement distress surveys per ASTM Standard D6433. The results of two such studies were used to evaluate the variability of reported PCI values and to establish qualification criteria for future prequalification efforts. A linear trend between the standard deviation of the reported PCI values and the average PCI was observed. Sections with a higher PCI (better condition) showed less variance than those with lower PCI values (worse condition). The linear relationship was used to normalize deviations from the baseline PCI and the root mean square error (RMSE) of the normal deviates was proposed for use as qualification criteria. In comparison with the original OCTA criteria, the RMSE of the normal deviates is a better measure of accuracy and precision and it is less dependent on the number of control sections used in the prequalification study.