MLPerf PowerPack: Defining MLPerf Inference Benchmark's Power Profile

Simple item page
dc.contributor.authorSmokowski, Cesar Jamesen
dc.contributor.committeechairCameron, Kirk W.en
dc.contributor.committeememberEllis, Margaret O.'Neilen
dc.contributor.committeememberBack, Godmar Volkeren
dc.contributor.departmentComputer Science and#38; Applicationsen
dc.date.accessioned2024-06-05T08:04:01Zen
dc.date.available2024-06-05T08:04:01Zen
dc.date.issued2024-06-04en
dc.description.abstractThe remarkable surge of artificial intelligence has placed a new emphasis on energy optimization in high-performance computing systems (HPCs). Energy consumption has become a critical consideration and limiting factor for the utility of machine learning applications, with each iteration consuming significantly more energy than the last. Understanding the distribution of energy consumption across different system components is crucial for enhancing the efficiency of systems that drive machine learning applications. This understanding allows us to identify optimal system configurations for machine learning-dedicated machines and minimize the substantial energy costs associated with training and deploying machine learning models. In this thesis, we present our findings from measuring power consumption for individual system components, including the CPU, GPU, and disk drive, using the PowerPack framework and on-chip power estimates for the FT and MLPerf Inference benchmarks. The direct power consumption measurements taken by the PowerPack framework offer a level of accuracy, granularity, and synchronization across multiple system components that is not feasible with on-chip power estimates alone. PowerPack provides fine-grain application and component-level profiling without the overhead cost of simulation. The PowerPack framework consists of software tools, including the NI-DAQmx library, and hardware components, such as the NI cDAQ-9172 CompactDAQ Chassis and three NI Analog Input Modules, designed to take physical direct current (DC) power measurements with component-level granularity. PowerPack's physical power measurements were used to assess the accuracy of two on-chip power estimates: AMDuProf, a power and performance analysis tool specifically designed for AMD processors, and NVIDIA-SMI, a software power analysis tool developed for NVIDIA GPUs. The AC power draw of our system under test was measured using a HOBO Plug Load Data Logger to determine the energy consumption distribution among the CPU, GPU, and disk drive. Our findings reveal the power consumption characteristics for the seven major functions of the MLPerf Inference benchmark. The major functions that comprise MLPerf Inference are ONNX Runtime model prediction, system check, post-processing ONNX model output, image retrieval, backend initialization, and loading annotations using the COCO API. Defining the power consumption characteristics of each function with component-level granularity provides valuable insight for function execution scheduling and identifying bottlenecks in the machine learning inference phase.en
dc.description.abstractgeneralThe popularity of services that use machine learning, such as OpenAI's ChatGPT and Dall-E, Amazon Alexa, and content curation on social media, has made minimizing the cost of using machine learning incredibly important for businesses across all industries. Machine learning is a powerful tool that enables businesses to optimize production and improve their customer experience, but it can be tremendously expensive to develop and run these services. Training and running machine learning models consume large amounts of energy, which is a major contributor to their cost. The work presented in this thesis aims to identify how much energy is consumed by individual computer components while running machine learning applications. Understanding how energy consumption is divided among a computer's components is a crucial step toward reducing the energy required to run these applications, thereby making them more affordable.en
dc.description.degreeMaster of Scienceen
dc.format.mediumETDen
dc.identifier.othervt_gsexam:40438en
dc.identifier.urihttps://hdl.handle.net/10919/119294en
dc.language.isoenen
dc.publisherVirginia Techen
dc.rightsIn Copyrighten
dc.rights.urihttp://rightsstatements.org/vocab/InC/1.0/en
dc.subjectMachine Learningen
dc.subjectArtificial Intelligenceen
dc.subjectPowerPacken
dc.subjectPoweren
dc.subjectEnergyen
dc.subjectEnergy Optimizationen
dc.subjectPower Measurementen
dc.titleMLPerf PowerPack: Defining MLPerf Inference Benchmark's Power Profileen
dc.typeThesisen
thesis.degree.disciplineComputer Science & Applicationsen
thesis.degree.grantorVirginia Polytechnic Institute and State Universityen
thesis.degree.levelmastersen
thesis.degree.nameMaster of Scienceen

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Smokowski_CJ_T_2024.pdf
Size:
32.74 MB
Format:
Adobe Portable Document Format
Download

Collections

Masters Theses