Formal Inconsistencies of Expertise Aggregation Techniques Commonly Employed in Engineering Teams

Engineering managers leverage the expertise of engineers in their teams to inform decisions. Engineers may convey their expertise in the form of opinions and/or judgements. Given a decision, it is common to elicit and aggregate the expertise from various engineers to capture a broader set of experiences and knowledge. Establishing an internally and externally consistent aggregation framework is therefore paramount to yield a meaningful aggregation, that is, to make sure that the expertise of each engineer is accounted for reasonably. However, we contend that most de facto aggregation techniques lack such consistency and lead to the inadequate use and aggregation of engineering expertise. In this paper, we investigate the consistency or lack thereof of various expertise aggregation techniques. We derive implications of such inconsistencies and provide recommendations about how they may be overcome. We illustrate our discussion using safety decisions in engineering as a notional case.