Application of the Analytic Hierarchy Process to Complex System Design Evaluation

This paper examines the use of the Analytic Hierarchy Process (AHP) to weight indicators used in the evaluation of complex systems designs which involve software, hardware, and humanware. Since such a comprehensive easily include hundreds of system quality indicators, evaluators need a technique to ensure the identification and emphasis of salient indicators in the determination of the quality of the design. The AHP is a popular technique for determining relative worth among a set of elements. In the present work, we introduce AHP with a simple example, then illustrate the application of the AHP to design evaluation using a subset of indicators from the human component of a system. We note in some detail issues which require added attention when applying AHP to this domain. The issues include indicator selection, dealing with large numbers of indicators, incorporating group judgements, and conflict resolution. We found AHP to be an effective tool for use in assigning weights criticality in indicator-based design evaluation, and propose elements of an environment in which the use of AHP is easily incorporated.