Handling Resource Constraints and Scalability in Continuous Query Processing

Abstract

Recent years have witnessed a rapid rise of a new class of data-intensive applications in which data arrive as transient, high-volume streams. Financial data processing, network monitoring, and sensor networks are all examples of such applications. Traditional relational database systems model data as persistent relations, but for this new class of applications, it is more appropriate to model data as unbounded streams with continuously arriving tuples. The stream data model necessitates a new style of queries called continuous queries. Unlike a one-time query executed over a single finite and static database state, a continuous query continuously generates new result tuples as new stream tuples arrive. This dissertation tackles a range of challenges that arise in processing continuous queries. Specifically, for resource-constrained settings, this dissertation proposes techniques for coping with response-time and memory constraints. To scale to a large number of continuous queries running concurrently, this dissertation proposes techniques for indexing continuous queries as data, and processing and optimizing incoming stream tuples as queries over such data. A common theme underlying most of these techniques is exploiting the characteristics of the data and the continuous queries, e.g., asymmetry in the costs of processing different streams, temporal trends in the values of stream attributes, and clusteredness that arises in a large number of continuous queries.