-
Towards application-specific query processing systems
Authors:
Dimitrios Vasilas,
Marc Shapiro,
Bradley King,
Sara Hamouda
Abstract:
Database systems use query processing subsystems for enabling efficient query-based data retrieval. An essential aspect of designing any query-intensive application is tuning the query system to fit the application's requirements and workload characteristics. However, the configuration parameters provided by traditional database systems do not cover the design decisions and trade-offs that arise f…
▽ More
Database systems use query processing subsystems for enabling efficient query-based data retrieval. An essential aspect of designing any query-intensive application is tuning the query system to fit the application's requirements and workload characteristics. However, the configuration parameters provided by traditional database systems do not cover the design decisions and trade-offs that arise from the geo-distribution of users and data. In this paper, we present a vision towards a new type of query system architecture that addresses this challenge by enabling query systems to be designed and deployed in a per use case basis. We propose a distributed abstraction called Query Processing Unit that encapsulates primitive query processing tasks, and show how it can be used as a building block for assembling query systems. Using this approach, application architects can construct query systems specialized to their use cases, by controlling the query system's architecture and the placement of its state. We demonstrate the expressiveness of this approach by applying it to the design of a query system that can flexibly place its state in the data center or at the edge, and show that state placement decisions affect the trade-off between query response time and query result freshness.
△ Less
Submitted 21 September, 2020;
originally announced September 2020.
-
A Modular Design for Geo-Distributed Querying
Authors:
Dimitrios Vasilas,
Marc Shapiro,
Bradley King
Abstract:
Most distributed storage systems provide limited abilities for querying data by attributes other than their primary keys. Supporting efficient search on secondary attributes is challenging as applications pose varying requirements to query processing systems, and no single system design can be suitable for all needs. In this paper, we show how to overcome these challenges in order to extend distri…
▽ More
Most distributed storage systems provide limited abilities for querying data by attributes other than their primary keys. Supporting efficient search on secondary attributes is challenging as applications pose varying requirements to query processing systems, and no single system design can be suitable for all needs. In this paper, we show how to overcome these challenges in order to extend distributed data stores to support queries on secondary attributes. We propose a modular architecture that is flexible and allows query processing systems to make trade-offs according to different use case requirements. We describe adap-tive mechanisms that make use of this flexibility to enable query processing systems to dynamically adjust to query and write operation workloads.
△ Less
Submitted 12 March, 2018;
originally announced March 2018.
-
Search on Secondary Attributes in Geo-Distributed Systems
Authors:
Dimitrios Vasilas
Abstract:
In the age of big data, more and more applications need to query and analyse large volumes of continuously updated data in real-time. In response, cloud-scale storage systems can extend their interface that allows fast lookups on the primary key with the ability to retrieve data based on non-primary attributes. However, the need to ingest content rapidly and make it searchable immediately while su…
▽ More
In the age of big data, more and more applications need to query and analyse large volumes of continuously updated data in real-time. In response, cloud-scale storage systems can extend their interface that allows fast lookups on the primary key with the ability to retrieve data based on non-primary attributes. However, the need to ingest content rapidly and make it searchable immediately while supporting low-latency, high-throughput query evaluation, as well as the geo-distributed nature and weak consistency guarantees of modern storage systems pose several challenges to the implementation of indexing and search systems. We present our early-stage work on the design and implementation of an indexing and query processing system that enables realtime queries on secondary attributes of data stored in geo-distributed, weakly consistent storage systems.
△ Less
Submitted 9 January, 2018;
originally announced January 2018.
