Efficient Query Processing for Dynamically Changing Datasets

Idris, Muhammad; Ugarte, Martin; Vansummeren, Stijn; Voigt, Hannes; Lehner, Wolfgang

Abstract

The ability to efficiently analyze changing data is a key requirement of many real-time analytics applications. Traditional approaches to this problem were developed around the notion of Incremental View Maintenance (IVM), and are based either on the materialization of subresults (to avoid their recomputation) or on the recomputation of subresults (to avoid the space overhead of materialization). Both techniques are suboptimal: instead of materializing results and subresults, one may also maintain a data structure that supports efficient maintenance under updates and from which the full query result can quickly be enumerated. In two previous articles, we have presented algorithms for dynamically evaluating queries that are easy to implement, efficient, and can be naturally extended to evaluate queries from a wide range of application domains. In this paper, we discuss our algorithm and its complexity, explaining the main components behind its efficiency. Finally, we show experiments that compare our algorithm to a state-of-the-art (Higher-order) IVM engine, as well as to a prominent complex event recognition engine. Our approach outperforms the competitor systems by up to two orders of magnitude in processing time, and one order in memory consumption.

Más información

Título según WOS: Efficient Query Processing for Dynamically Changing Datasets
Título según SCOPUS: Efficient query processing for dynamically changing datasets
Título de la Revista: SIGMOD RECORD
Volumen: 48
Número: 1
Editorial: ASSOC COMPUTING MACHINERY
Fecha de publicación: 2019
Página de inicio: 33
Página final: 40
Idioma: English
Notas: ISI, SCOPUS