Playing the Bad Guy: How Organizations Design, Develop, and Measure Red Teams

Abstract

The study is a descriptive analysis using a case-study methodology that identifies the critical elements (methods, tools, processes, personnel, and practices) of adversary analysis identified as a red team and red-teaming. A red team is the adversary element of the analytic method of red-teaming. The study incorporates interview data with organization leadership, subject matter experts, and red-team developers from Department of Defense (DoD), Intelligence Community (IC), and Federally Funded Research and Development Centers (FFRDC) organizations. The study also includes red-team governance documents, red-team briefings, and discussions to first identify the concepts, analyze the critical design elements of the concept(s), and develop a fundamental taxonomy or classification of red-team approaches based on these artifacts. The study compares and contrasts four red teams that utilize groups of adversary subject-matter experts for common themes, differences, and best practices. The data collection builds on grounded theory—i.e., identification of the methods, tools, processes, and personnel as the organizations understand and develop their red teams as part of their red-teaming analyses to address gaps in understanding possible adversaries. The four organizations studied are the U.S. Army, Training and Doctrine Command, University of Foreign Military and Cultural Studies; a Department of Defense unified combatant command; the U.S. Naval War College (NWC) and its red-team detachment; and a Sandia National Laboratories (SNL) Homeland Security and Defense, National Infrastructure Simulation and Analysis Center (NISAC). Two basic types of red teams are identified from the data with a hybrid between the two among the variations of the red-teaming concept. Some of the other findings from the four red teams include a need to develop common terms and standards; a need to explain the benefits of alternative analysis to decisionmakers; a need to develop trend analyses on types of red teams requested by sponsors; a need to design methods to capture non-state actors; a need to include more coalition and foreign partners; and a need to immerse red teams more fully into the culture to be understood.