Show simple item record

dc.contributor.advisor Babu, Shivnath en_US
dc.contributor.author Herodotou, Herodotos en_US
dc.date.accessioned 2012-05-25T20:07:41Z
dc.date.available 2012-05-25T20:07:41Z
dc.date.issued 2012 en_US
dc.identifier.uri http://hdl.handle.net/10161/5415
dc.description Dissertation en_US
dc.description.abstract <p>Modern industrial, government, and academic organizations are collecting massive amounts of data ("Big Data") at an unprecedented scale and pace. The ability to perform timely and cost-effective analytical processing of such large datasets in order to extract deep insights is now a key ingredient for success. These insights can drive automated processes for advertisement placement, improve customer relationship management, and lead to major scientific breakthroughs.</p><p>Existing database systems are adapting to the new status quo while large-scale dataflow systems (like Dryad and MapReduce) are becoming popular for executing analytical workloads on Big Data. Ensuring good and robust performance automatically on such systems poses several challenges. First, workloads often analyze a hybrid mix of structured and unstructured datasets stored in nontraditional data layouts. The structure and properties of the data may not be known upfront, and will evolve over time. Complex analysis techniques and rapid development needs necessitate the use of both declarative and procedural programming languages for workload specification. Finally, the space of workload tuning choices is very large and high-dimensional, spanning configuration parameter settings, cluster resource provisioning (spurred by recent innovations in cloud computing), and data layouts.</p><p>We have developed a novel dynamic optimization approach that can form the basis for tuning workload performance automatically across different tuning scenarios and systems. Our solution is based on (i) collecting monitoring information in order to learn the run-time behavior of workloads, (ii) deploying appropriate models to predict the impact of hypothetical tuning choices on workload behavior, and (iii) using efficient search strategies to find tuning choices that give good workload performance. The dynamic nature enables our solution to overcome the new challenges posed by Big Data, and also makes our solution applicable to both MapReduce and Database systems. We have developed the first cost-based optimization framework for MapReduce systems for determining the cluster resources and configuration parameter settings to meet desired requirements on execution time and cost for a given analytic workload. We have also developed a novel tuning-based optimizer in Database systems to collect targeted run-time information, perform optimization, and repeat as needed to perform fine-grained tuning of SQL queries.</p> en_US
dc.subject Computer science en_US
dc.subject cost-based optimization en_US
dc.subject Database systems en_US
dc.subject MapReduce systems en_US
dc.subject self-tuning systems en_US
dc.title Automatic Tuning of Data-Intensive Analytical Workloads en_US
dc.type Dissertation en_US
dc.department Computer Science en_US

Files in this item

This item appears in the following Collection(s)

Show simple item record