Impediments to Effective Safety Risk Assessment of Safety Critical Systems: An Insight into SRM Processes and Expert Aggregation

Safety risk assessment forms an integral part of the design and development of Safety Critical Systems. Conventionally in these systems, standards and policies have been developed to prescribe processes for safety risk assessment. These standards provide guidelines, references and structure to personnel involved in the risk assessment process. However, in some of these standards, the prescribed methods for safety decision making were found to be deficient in some respects. Two such deficiencies have been addressed in this thesis.

First, when different safety metrics are required to be combined for a safety related decision, the current practices of using safety risk matrices were found to be inconsistent with the axioms of decision theory. Second, in the safety risk assessment process, when multiple experts are consulted to provide their judgment on the severity and/or likelihood of hazards, the standards were lacking detailed guidelines for aggregating experts' judgements. Such deficiencies could lead to misconceptions pertaining to the safety risk level of critical hazards. These misconceptions potentially give rise to inconsistent safety decisions that might ultimately result in catastrophic outcomes.

This thesis addresses both these concerns present in SRM processes. For the problem of combining safety metrics, three potential approaches have been proposed. Normative Decision Analysis tools such as Utility Theory and Multi-attribute Utility Theory were proposed in the first and second approaches. The third approach proposes the use of a Multi-Objective Optimization technique - Pareto Analysis. For problems in Expert Aggregation, behavioral and mathematical solutions have been explored and the implications of using these methods for Safety Risk Assessment have been discussed. Two standard documents that contain the Safety Risk Management Processes of the Federal Aviation Agency (FAA) and the U.S. Navy were used to structure the case studies.

This thesis has two main contributions. First, it evaluates the use of decision analysis in safety decision process of Safety Critical Systems. It provides guidelines to decision makers on how to meaningfully use and/or combine different safety metrics in the decision process. Second, it identifies the best practices and methods of aggregating expert assessments pertaining to safety decision making.