Skip to main content
Duke University Libraries
DukeSpace Scholarship by Duke Authors
  • Login
  • Ask
  • Menu
  • Login
  • Ask a Librarian
  • Search & Find
  • Using the Library
  • Research Support
  • Course Support
  • Libraries
  • About
View Item 
  •   DukeSpace
  • Theses and Dissertations
  • Duke Dissertations
  • View Item
  •   DukeSpace
  • Theses and Dissertations
  • Duke Dissertations
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

Output Performance of Petascale File Systems

Thumbnail
View / Download
3.2 Mb
Date
2017
Author
Xie, Bing
Advisor
Chase, Jeff S
Repository Usage Stats
270
views
244
downloads
Abstract

HPC applications generate periodic output bursts for intermediate results, checkpointing and visualization. For a typical HPC application, when writing a burst its entire execution is stalled until all data reaches disks. In general, since cores must idle during a burst, a supercomputer and its I/O system must absorb the output burst fast to use compute cores efficiently.

This thesis studies the performance of file writes in supercomputer file systems under production load, including quantitative behavior analysis, component performance profiling and performance prediction modeling. Our target environment is Titan-the 4th fastest supercomputer in the world-and its Lustre parallel file stores. The results of behavior analysis and performance profiling can inform file system configuration choices and the design of I/O software in the application, operating system, and adaptive I/O middleware systems. Moreover, the predictive model we build is useful for output performance prediction of supercomputer file systems in live use.

To quantify the performance behavior of production supercomputer file systems, we introduce a statistical benchmarking methodology to measure the impact of parameter choices on burst absorption rates. Our approach combines many samples of their impacts over time to filter out interference caused by the transient congestion from competing workloads in the production setting. These samples are also used to characterize the performance of individual stages and components in the multi-stage write pipelines, and their variations over time.

We find that Titan's I/O system is variable across the machine at fine time scales. This variability has two major implications. First, stragglers lessen the benefit of coupled I/O parallelism (striping). Peak median output bandwidths are obtained with parallel writes to many independent files, with no striping or write-sharing of files across clients (compute nodes). I/O parallelism is most effective when the application-or its I/O middleware system-distributes the I/O load so that each target stores files for multiple clients and each client writes files on multiple targets, in a balanced way with minimal contention. Second, our results suggest that the potential benefit of dynamic adaptation is limited. In particular, it is not fruitful to attempt to identify "good locations" in the machine or in the file system: component performance is driven by transient load conditions, and past performance is not a useful predictor of future performance. For example, we do not observe diurnal load patterns that are predictable.

Beyond the observation of performance variability of Titan, we also notice that: 1. The mean performance is stable and consistent over typical application run times; 2. Output performance is non-linearly related to its correlated parameters due to interference and saturation on individual stages on the path. These observations enable us to build a predictive model of expected write times of output patterns and I/O configurations, using feature transformations to capture non-linear relationships. We identify the candidate features based on the structure of the Lustre/Titan write path, and use feature transformation functions to produce a linear model space with 135,000 candidate models. By searching for the minimal mean square error in this space we identify a good model and show that it is effective.

Type
Dissertation
Department
Computer Science
Subject
Computer science
Permalink
https://hdl.handle.net/10161/16317
Citation
Xie, Bing (2017). Output Performance of Petascale File Systems. Dissertation, Duke University. Retrieved from https://hdl.handle.net/10161/16317.
Collections
  • Duke Dissertations
More Info
Show full item record
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.

Rights for Collection: Duke Dissertations


Works are deposited here by their authors, and represent their research and opinions, not that of Duke University. Some materials and descriptions may include offensive content. More info

Make Your Work Available Here

How to Deposit

Browse

All of DukeSpaceCommunities & CollectionsAuthorsTitlesTypesBy Issue DateDepartmentsAffiliations of Duke Author(s)SubjectsBy Submit DateThis CollectionAuthorsTitlesTypesBy Issue DateDepartmentsAffiliations of Duke Author(s)SubjectsBy Submit Date

My Account

LoginRegister

Statistics

View Usage Statistics
Duke University Libraries

Contact Us

411 Chapel Drive
Durham, NC 27708
(919) 660-5870
Perkins Library Service Desk

Digital Repositories at Duke

  • Report a problem with the repositories
  • About digital repositories at Duke
  • Accessibility Policy
  • Deaccession and DMCA Takedown Policy

TwitterFacebookYouTubeFlickrInstagramBlogs

Sign Up for Our Newsletter
  • Re-use & Attribution / Privacy
  • Harmful Language Statement
  • Support the Libraries
Duke University