Skip to main content
We gratefully acknowledge support from the Simons Foundation, member institutions, and all contributors. Donate
arXiv logo
Cornell University Logo

Computer Science > Databases

arXiv:2304.01659 (cs)
[Submitted on 4 Apr 2023 (v1), last revised 27 Aug 2024 (this version, v2)]

Title:Diba: A Re-configurable Stream Processor

Authors:Mohammadreza Najafi, Thamir M. Qadah, Mohammad Sadoghi, Hans-Arno Jacobsen
View PDF
Abstract:Stream processing acceleration is driven by the continuously increasing volume and velocity of data generated on the Web and the limitations of storage, computation, and power consumption. Hardware solutions provide better performance and power consumption, but they are hindered by the high research and development costs and the long time to market. In this work, we propose our re-configurable stream processor (Diba), a complete rethinking of a previously proposed customized and flexible query processor that targets real-time stream processing. Diba uses a unidirectional dataflow not dedicated to any specific type of query (operator) on streams, allowing a straightforward placement of processing components on a general data path that facilitates query mapping. In Diba, the concepts of the distribution network and processing components are implemented as two separate entities connected using generic interfaces. This approach allows the adoption of a versatile architecture for a family of queries rather than forcing a rigid chain of processing components to implement such queries. Our experimental evaluations of representative queries from TPC-H yielded processing times of 300, 1220, and 3520 milliseconds for data streams with scale factor sizes of one, four, and ten gigabytes, respectively.
Subjects: Databases (cs.DB); Hardware Architecture (cs.AR)
Cite as: arXiv:2304.01659 [cs.DB]
  (or arXiv:2304.01659v2 [cs.DB] for this version)
  https://doi.org/10.48550/arXiv.2304.01659
Journal reference: in IEEE Transactions on Knowledge and Data Engineering, vol. 36, no. 9, pp. 4550-4566, Sept. 2024
Related DOI: https://doi.org/10.1109/TKDE.2024.3381192

Submission history

From: Thamir Qadah [view email]
[v1] Tue, 4 Apr 2023 09:29:01 UTC (5,339 KB)
[v2] Tue, 27 Aug 2024 21:27:21 UTC (6,407 KB)
Full-text links:

Access Paper:

license icon view license
Current browse context:
cs.DB
< prev   |   next >
new | recent | 2023-04
Change to browse by:
cs
cs.AR

References & Citations

export BibTeX citation

Bookmark

BibSonomy logo Reddit logo

Bibliographic and Citation Tools

Bibliographic Explorer (What is the Explorer?)
Connected Papers (What is Connected Papers?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)

Code, Data and Media Associated with this Article

alphaXiv (What is alphaXiv?)
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Hugging Face (What is Huggingface?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)

Demos

Replicate (What is Replicate?)
Hugging Face Spaces (What is Spaces?)
TXYZ.AI (What is TXYZ.AI?)

Recommenders and Search Tools

Influence Flower (What are Influence Flowers?)
CORE Recommender (What is CORE?)

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)