Smokowski, Cesar James2024-06-052024-06-052024-06-04vt_gsexam:40438https://hdl.handle.net/10919/119294The remarkable surge of artificial intelligence has placed a new emphasis on energy optimization in high-performance computing systems (HPCs). Energy consumption has become a critical consideration and limiting factor for the utility of machine learning applications, with each iteration consuming significantly more energy than the last. Understanding the distribution of energy consumption across different system components is crucial for enhancing the efficiency of systems that drive machine learning applications. This understanding allows us to identify optimal system configurations for machine learning-dedicated machines and minimize the substantial energy costs associated with training and deploying machine learning models. In this thesis, we present our findings from measuring power consumption for individual system components, including the CPU, GPU, and disk drive, using the PowerPack framework and on-chip power estimates for the FT and MLPerf Inference benchmarks. The direct power consumption measurements taken by the PowerPack framework offer a level of accuracy, granularity, and synchronization across multiple system components that is not feasible with on-chip power estimates alone. PowerPack provides fine-grain application and component-level profiling without the overhead cost of simulation. The PowerPack framework consists of software tools, including the NI-DAQmx library, and hardware components, such as the NI cDAQ-9172 CompactDAQ Chassis and three NI Analog Input Modules, designed to take physical direct current (DC) power measurements with component-level granularity. PowerPack's physical power measurements were used to assess the accuracy of two on-chip power estimates: AMDuProf, a power and performance analysis tool specifically designed for AMD processors, and NVIDIA-SMI, a software power analysis tool developed for NVIDIA GPUs. The AC power draw of our system under test was measured using a HOBO Plug Load Data Logger to determine the energy consumption distribution among the CPU, GPU, and disk drive. Our findings reveal the power consumption characteristics for the seven major functions of the MLPerf Inference benchmark. The major functions that comprise MLPerf Inference are ONNX Runtime model prediction, system check, post-processing ONNX model output, image retrieval, backend initialization, and loading annotations using the COCO API. Defining the power consumption characteristics of each function with component-level granularity provides valuable insight for function execution scheduling and identifying bottlenecks in the machine learning inference phase.ETDenIn CopyrightMachine LearningArtificial IntelligencePowerPackPowerEnergyEnergy OptimizationPower MeasurementThesis