Synthesizing a Hybrid Benchmark Suite with BenchPrime
| dc.contributor.author | Wu, Xiaolong | en |
| dc.contributor.committeechair | Jung, Changhee | en |
| dc.contributor.committeemember | Jian, Xun | en |
| dc.contributor.committeemember | Lee, Dongyoon | en |
| dc.contributor.department | Computer Science | en |
| dc.date.accessioned | 2018-10-10T08:00:38Z | en |
| dc.date.available | 2018-10-10T08:00:38Z | en |
| dc.date.issued | 2018-10-09 | en |
| dc.description.abstract | This paper presents BenchPrime, an automated benchmark analysis toolset that is systematic and extensible to analyze the similarity and diversity of benchmark suites. BenchPrime takes multiple benchmark suites and their evaluation metrics as inputs and generates a hybrid benchmark suite comprising only essential applications. Unlike prior work, BenchPrime uses linear discriminant analysis rather than principal component analysis, as well as selects the best clustering algorithm and the optimized number of clusters in an automated and metric-tailored way, thereby achieving high accuracy. In addition, BenchPrime ranks the benchmark suites in terms of their application set diversity and estimates how unique each benchmark suite is compared to other suites. As a case study, this work for the first time compares the DenBench with the MediaBench and MiBench using four different metrics to provide a multi-dimensional understanding of the benchmark suites. For each metric, BenchPrime measures to what degree DenBench applications are irreplaceable with those in MediaBench and MiBench. This provides means for identifying an essential subset from the three benchmark suites without compromising the application balance of the full set. The experimental results show that the necessity of including DenBench applications varies across the target metrics and that significant redundancy exists among the three benchmark suites. | en |
| dc.description.abstractgeneral | Representative benchmarks are widely used in the research area to achieve an accurate and fair evaluation of hardware and software techniques. However, the redundant applications in the benchmark set can skew the average towards redundant characteristics overestimating the benefit of any proposed research. This work proposes a machine learning-based framework BenchPrime to generates a hybrid benchmark suite comprising only essential applications. In addition, BenchPrime ranks the benchmark suites in terms of their application set diversity and estimates how unique each benchmark suite is compared to other suites. | en |
| dc.description.degree | Master of Science | en |
| dc.format.medium | ETD | en |
| dc.identifier.other | vt_gsexam:17413 | en |
| dc.identifier.uri | http://hdl.handle.net/10919/85332 | en |
| dc.publisher | Virginia Tech | en |
| dc.rights | In Copyright | en |
| dc.rights.uri | http://rightsstatements.org/vocab/InC/1.0/ | en |
| dc.subject | Algorithm | en |
| dc.subject | Machine learning | en |
| dc.subject | BenchPrime | en |
| dc.subject | Benchmark subseting | en |
| dc.title | Synthesizing a Hybrid Benchmark Suite with BenchPrime | en |
| dc.type | Thesis | en |
| thesis.degree.discipline | Computer Science and Applications | en |
| thesis.degree.grantor | Virginia Polytechnic Institute and State University | en |
| thesis.degree.level | masters | en |
| thesis.degree.name | Master of Science | en |
Files
Original bundle
1 - 1 of 1