An evaluation methodology to ensure the success of decision support tools

Abstract

Motivated by the need for an evaluation technique to help decision makers ensure the success of their computer-based decision support tools, this research explores the evaluation of decision aids from a broad organizational and managerial perspective. A review of current research identifies the need for theoretical and practical developments emphasizing: (1) evaluation techniques which can work with partial knowledge about the effect of a decision support tool on management processes, (2) a systematic way to prescribe evaluation techniques for different assessment situations, (3) evaluation techniques which provide a way to transition from one assessment situation to another, and (4) evaluation techniques which recognize that the performance of one decision support tool may depend on other decision aids used by the manager. This study complements existing theoretical research by developing seven conceptual models which identify essential evaluation parameters and their relationships. The first model explores parameters affecting the decision to evaluate. The second and third models examine the role of evaluation in ensuring success. The fourth and fifth models analyze how conclusions about success are made. The sixth model identifies components of an evaluation technique. Finally, the seventh model presents a framework for prescribing evaluation approaches. Using the seven conceptual models and previous research as its theoretical foundation, Evalu-Action, a step-by-step practical technique to ensure the success of computer-based decision support tools is developed. The technique is pilot tested and improved. Recommendations for further work are presented.