Browsing by Author "Stephen, Cynthia"

Now showing 1 - 2 of 2
  • Thumbnail Image
    Formal Inconsistencies of Expertise Aggregation Techniques Commonly Employed in Engineering Teams
    Stephen, Cynthia; Kannan, Hanumanthrao; Salado, Alejandro (MDPI, 2024-05-18)
    Engineering managers leverage the expertise of engineers in their teams to inform decisions. Engineers may convey their expertise in the form of opinions and/or judgements. Given a decision, it is common to elicit and aggregate the expertise from various engineers to capture a broader set of experiences and knowledge. Establishing an internally and externally consistent aggregation framework is therefore paramount to yield a meaningful aggregation, that is, to make sure that the expertise of each engineer is accounted for reasonably. However, we contend that most de facto aggregation techniques lack such consistency and lead to the inadequate use and aggregation of engineering expertise. In this paper, we investigate the consistency or lack thereof of various expertise aggregation techniques. We derive implications of such inconsistencies and provide recommendations about how they may be overcome. We illustrate our discussion using safety decisions in engineering as a notional case.
  • Thumbnail Image
    Impediments to Effective Safety Risk Assessment of Safety Critical Systems: An Insight into SRM Processes and Expert Aggregation
    Stephen, Cynthia (Virginia Tech, 2020-06-25)
    Safety risk assessment forms an integral part of the design and development of Safety Critical Systems. Conventionally in these systems, standards and policies have been developed to prescribe processes for safety risk assessment. These standards provide guidelines, references and structure to personnel involved in the risk assessment process. However, in some of these standards, the prescribed methods for safety decision making were found to be deficient in some respects. Two such deficiencies have been addressed in this thesis. First, when different safety metrics are required to be combined for a safety related decision, the current practices of using safety risk matrices were found to be inconsistent with the axioms of decision theory. Second, in the safety risk assessment process, when multiple experts are consulted to provide their judgment on the severity and/or likelihood of hazards, the standards were lacking detailed guidelines for aggregating experts' judgements. Such deficiencies could lead to misconceptions pertaining to the safety risk level of critical hazards. These misconceptions potentially give rise to inconsistent safety decisions that might ultimately result in catastrophic outcomes. This thesis addresses both these concerns present in SRM processes. For the problem of combining safety metrics, three potential approaches have been proposed. Normative Decision Analysis tools such as Utility Theory and Multi-attribute Utility Theory were proposed in the first and second approaches. The third approach proposes the use of a Multi-Objective Optimization technique - Pareto Analysis. For problems in Expert Aggregation, behavioral and mathematical solutions have been explored and the implications of using these methods for Safety Risk Assessment have been discussed. Two standard documents that contain the Safety Risk Management Processes of the Federal Aviation Agency (FAA) and the U.S. Navy were used to structure the case studies. This thesis has two main contributions. First, it evaluates the use of decision analysis in safety decision process of Safety Critical Systems. It provides guidelines to decision makers on how to meaningfully use and/or combine different safety metrics in the decision process. Second, it identifies the best practices and methods of aggregating expert assessments pertaining to safety decision making.