Multi-program benchmark definition

Loading...
Thumbnail Image

Date

2015-01-01

Journal Title

Journal ISSN

Volume Title

Repository Usage Stats

179
views
267
downloads

Citation Stats

Abstract

© 2015 IEEE.Although definition of single-program benchmarks is relatively straight-forward-a benchmark is a program plus a specific input-definition of multi-program benchmarks is more complex. Each program may have a different runtime and they may have different interactions depending on how they align with each other. While prior work has focused on sampling multiprogram benchmarks, little attention has been paid to defining the benchmarks in their entirety. In this work, we propose a four-tuple that formally defines multi-program benchmarks in a well-defined way. We then examine how four different classes of benchmarks created by varying the elements of this tuple align with real-world use-cases. We evaluate the impact of these variations on real hardware, and see drastic variations in results between different benchmarks constructed from the same programs. Notable differences include significant speedups versus slowdowns (e.g., +57% vs -5% or +26% vs -18%), and large differences in magnitude even when the results are in the same direction (e.g., 67% versus 11%).

Department

Description

Provenance

Subjects

Citation

Published Version (Please cite this version)

10.1109/ISPASS.2015.7095786

Scholars@Duke

Hilton

Andrew Douglas Hilton

Professor of the Practice in the Department of Electrical and Computer Engineering

Drew Hilton is a Professor of the Practice in Electrical and Computer Engineering, as well as Pratt’s Director of Innovation in Computing Education.

His main focus is on teaching professional-level programming skills to ECE’s master's students to prepare them for software engineering careers.

Professor Hilton also teaches a 3-week introduction to Programming Python for Duke's Master in Interdisciplinary Data Science, and Duke's Center for Computational Thinking.

He has two Coursera specializations, one in Java, and another in C.

Sorin

Daniel J. Sorin

Professor of Electrical and Computer Engineering

Dr. Daniel Sorin is a professor of Electrical and Computer Engineering at Duke University. His research interests are primarily in computer architecture and dependability.


Unless otherwise indicated, scholarly articles published by Duke faculty members are made available here with a CC-BY-NC (Creative Commons Attribution Non-Commercial) license, as enabled by the Duke Open Access Policy. If you wish to use the materials in ways not already permitted under CC-BY-NC, please consult the copyright owner. Other materials are made available here through the author’s grant of a non-exclusive license to make their work openly accessible.