The Design and Implementation of the Facial Recognition Vendor Test 2000 Evaluation Methodology

Abstract

The biggest change in the facial recognition community since the completion of the FacE REcognition Technology (FERET) program has been the introduction of facial recognition products to the commercial market. Open market competitiveness has driven numerous technological advances in automated face recognition since the FERET program and significantly lowered system costs. Today there are dozens of facial recognition systems available that have the potential to meet performance requirements for numerous applications. But which of these systems best meet the performance requirements for given applications? Repeated inquiries from numerous government agencies on the current state of facial recognition technology prompted the DoD Counterdrug Technology Development Program Office to establish a new set of evaluations. The Facial Recognition Vendor Test 2000 (FRVT 2000), was co-sponsored by the DoD Counterdrug Technology Development Program Office, the National Institute of Justice, and the Defense Advanced Research Projects Agency, and was administered in May-June 2000. The sponsors of the FRVT 2000 had two major goals for the evaluation. The first was a technical assessment of the capabilities of commercially available facial recognition systems. The sponsors wanted to know the strengths and weaknesses of each individual system, as well as obtain an understanding of the current state of the art for facial recognition. The second goal of the evaluation was to educate the biometrics community and the general public on how to present and analyze results. The sponsors have seen vendors and would-be customers quoting outstanding performance specifications without understanding that these specifications are virtually useless without first knowing the details of the test that was used to produce the quoted results. The Facial Recognition Vendor Test 2000 was a worthwhile endeavor. It will help numerous readers evaluate facial recognition systems for their own uses and will serve as a benchmark for all future evaluations of biometric technologies. The FRVT 2000 evaluations were not designed, and the FRVT 2000 Evaluation Report was not written, to be a buyer's guide for facial recognition. No one will be able to open the report to a specific page to determine which facial recognition system is best because there is not one system for all applications. The only way to determine the best facial recognition system for any application is to follow the three-step evaluation methodology described in the FRVT 2000 Evaluation Report and analyze the data as it pertains to each individual application. This thesis explains the design and implementation of the FRVT 2000 evaluations, and discusses how the FRVT 2000 Evaluation Report met the author's objectives for the evaluation.