-
DHT-based Edge and Fog Computing Systems: Infrastructures and Applications
Authors:
Yahya Hassanzadeh-Nazarabadi,
Sanaz Taheri-Boshrooyeh,
Öznur Özkasap
Abstract:
Intending to support new emerging applications with latency requirements below what can be offered by the cloud data centers, the edge and fog computing paradigms have reared. In such systems, the real-time instant data is processed closer to the edge of the network, instead of the remote data centers. With the advances in edge and fog computing systems, novel and efficient solutions based on Dist…
▽ More
Intending to support new emerging applications with latency requirements below what can be offered by the cloud data centers, the edge and fog computing paradigms have reared. In such systems, the real-time instant data is processed closer to the edge of the network, instead of the remote data centers. With the advances in edge and fog computing systems, novel and efficient solutions based on Distributed Hash Tables (DHTs) emerge and play critical roles in system design. Several DHT-based solutions have been proposed to either augment the scalability and efficiency of edge and fog computing infrastructures or to enable application-specific functionalities such as task and resource management. This paper presents the first comprehensive study on the state-of-the-art DHT-based architectures in edge and fog computing systems from the lenses of infrastructure and application. Moreover, the paper details the open problems and discusses future research guidelines for the DHT-based edge and fog computing systems.
△ Less
Submitted 10 November, 2022;
originally announced November 2022.
-
DHT-based Communications Survey: Architectures and Use Cases
Authors:
Yahya Hassanzadeh-Nazarabadi,
Sanaz Taheri-Boshrooyeh,
Safa Otoum,
Seyhan Ucar,
Öznur Özkasap
Abstract:
Several distributed system paradigms utilize Distributed Hash Tables (DHTs) to realize structured peer-to-peer (P2P) overlays. DHT structures arise as the most commonly used organizations for peers that can efficiently perform crucial services such as data storage, replication, query resolution, and load balancing. With the advances in various distributed system technologies, novel and efficient s…
▽ More
Several distributed system paradigms utilize Distributed Hash Tables (DHTs) to realize structured peer-to-peer (P2P) overlays. DHT structures arise as the most commonly used organizations for peers that can efficiently perform crucial services such as data storage, replication, query resolution, and load balancing. With the advances in various distributed system technologies, novel and efficient solutions based on DHTs emerge and play critical roles in system design. DHT-based methods and communications have been proposed to address challenges such as scalability, availability, reliability and performance, by considering unique characteristics of these technologies. In this article, we propose a classification of the state-of-the-art DHT-based methods focusing on their system architecture, communication, routing and technological aspects across various system domains. To the best of our knowledge, there is no comprehensive survey on DHT-based applications from system architecture and communication perspectives that spans various domains of recent distributed system technologies. We investigate the recently emerged DHT-based solutions in the seven key domains of edge and fog computing, cloud computing, blockchain, the Internet of Things (IoT), Online Social Networks (OSNs), Mobile Ad Hoc Networks (MANETs), and Vehicular Ad Hoc Networks (VANETs). In contrast to the existing surveys, our study goes beyond the commonly known DHT methods such as storage, routing, and lookup, and identifies diverse DHT-based solutions including but not limited to aggregation, task scheduling, resource management and discovery, clustering and group management, federation, data dependency management, and data transmission. Furthermore, we identify open problems and discuss future research guidelines for each domain.
△ Less
Submitted 22 September, 2021;
originally announced September 2021.
-
LightChain: Scalable DHT-Based Blockchain
Authors:
Yahya Hassanzadeh-Nazarabadi,
Alptekin Küpçü,
Öznur Özkasap
Abstract:
As an append-only distributed database, blockchain is utilized in a vast variety of applications including the cryptocurrency and Internet-of-Things (IoT). The existing blockchain solutions show downsides in communication and storage scalability, as well as decentralization. In this article, we propose LightChain , which is the first blockchain architecture that operates over a Distributed Hash Ta…
▽ More
As an append-only distributed database, blockchain is utilized in a vast variety of applications including the cryptocurrency and Internet-of-Things (IoT). The existing blockchain solutions show downsides in communication and storage scalability, as well as decentralization. In this article, we propose LightChain , which is the first blockchain architecture that operates over a Distributed Hash Table (DHT) of participating peers. LightChain is a permissionless blockchain that provides addressable blocks and transactions within the network, which makes them efficiently accessible by all peers. Each block and transaction is replicated within the DHT of peers and is retrieved in an on-demand manner. Hence, peers in LightChain are not required to retrieve or keep the entire ledger. LightChain is fair as all of the participating peers have a uniform chance of being involved in the consensus regardless of their influence such as hashing power or stake. We provide formal mathematical analysis and experimental results (simulations and cloud deployment) to demonstrate the security, efficiency, and fairness of LightChain , and show that LightChain is the only existing blockchain that can provide integrity under the corrupted majority power of peers. As we experimentally demonstrate, compared to the mainstream blockchains such as Bitcoin and Ethereum, LightChain requires around 66 times smaller per node storage, and is around 380 times faster on bootstrapping a new node to the system, and each LightChain node is rewarded equally likely for participating in the protocol.
△ Less
Submitted 1 September, 2021;
originally announced September 2021.
-
Demo: A Proof-of-Concept Implementation of Guard Secure Routing Protocol
Authors:
Sanaz Taheri-Boshrooyeh,
Ali Utkan Şahin,
Yahya Hassanzadeh-Nazarabadi,
Öznur Özkasap
Abstract:
Skip Graphs belong to the family of Distributed Hash Table (DHT) structures that are utilized as routing overlays in various peer-to-peer applications including blockchains, cloud storage, and social networks. In a Skip Graph overlay, any misbehavior of peers during the routing of a query compromises the system functionality. Guard is the first authenticated search mechanism for Skip Graphs, enabl…
▽ More
Skip Graphs belong to the family of Distributed Hash Table (DHT) structures that are utilized as routing overlays in various peer-to-peer applications including blockchains, cloud storage, and social networks. In a Skip Graph overlay, any misbehavior of peers during the routing of a query compromises the system functionality. Guard is the first authenticated search mechanism for Skip Graphs, enables reliable search operation in a fully decentralized manner. In this demo paper, we present a proof-of-concept implementation of Guard on Skip Graph nodes as well as a deployment demo scenario.
△ Less
Submitted 26 July, 2020;
originally announced July 2020.
-
A containerized proof-of-concept implementation of LightChain system
Authors:
Yahya Hassanzadeh-Nazarabadi,
Nazir Nayal,
Shadi Sameh Hamdan,
Öznur Özkasap,
Alptekin Küpçü
Abstract:
LightChain is the first Distributed Hash Table (DHT)-based blockchain with a logarithmic asymptotic message and memory complexity. In this demo paper, we present the software architecture of our open-source implementation of LightChain, as well as a novel deployment scenario of the entire LightChain system on a single machine aiming at results reproducibility.
LightChain is the first Distributed Hash Table (DHT)-based blockchain with a logarithmic asymptotic message and memory complexity. In this demo paper, we present the software architecture of our open-source implementation of LightChain, as well as a novel deployment scenario of the entire LightChain system on a single machine aiming at results reproducibility.
△ Less
Submitted 26 July, 2020;
originally announced July 2020.
-
SkipSim: Scalable Skip Graph Simulator
Authors:
Yahya Hassanzadeh-Nazarabadi,
Ali Utkan Şahin,
Öznur Özkasap,
Alptekin Küpçü
Abstract:
SkipSim is an offline Skip Graph simulator that enables Skip Graph-based algorithms including blockchains and P2P cloud storage to be simulated while preserving their scalability and decentralized nature. To the best of our knowledge, it is the first Skip Graph simulator that provides several features for experimentation on Skip Graph-based overlay networks. In this demo paper, we present SkipSim…
▽ More
SkipSim is an offline Skip Graph simulator that enables Skip Graph-based algorithms including blockchains and P2P cloud storage to be simulated while preserving their scalability and decentralized nature. To the best of our knowledge, it is the first Skip Graph simulator that provides several features for experimentation on Skip Graph-based overlay networks. In this demo paper, we present SkipSim features, its architecture, as well as a sample blockchain demo scenario.
△ Less
Submitted 26 July, 2020;
originally announced July 2020.
-
Flow: Separating Consensus and Compute -- Block Formation and Execution
Authors:
Alexander Hentschel,
Yahya Hassanzadeh-Nazarabadi,
Ramtin Seraj,
Dieter Shirley,
Layne Lafrance
Abstract:
Most current blockchains require all full nodes to execute all tasks limits the throughput of existing blockchains, which are well documented and among the most significant hurdles for the widespread adoption of decentralized technology.
This paper extends out presentation of Flow, a pipelined blockchain architecture, which separates the process of consensus on the transaction order from transac…
▽ More
Most current blockchains require all full nodes to execute all tasks limits the throughput of existing blockchains, which are well documented and among the most significant hurdles for the widespread adoption of decentralized technology.
This paper extends out presentation of Flow, a pipelined blockchain architecture, which separates the process of consensus on the transaction order from transaction computation. As we experimentally showed in our previous white paper, our architecture provides a significant throughput improvement while preserving the security of the system. Flow exploits the heterogeneity offered by the nodes, in terms of bandwidth, storage, and computational capacity, and defines the roles for the nodes based on their tasks in the pipeline, i.e., Collector, Consensus, Execution, and Verification. While transaction collection from the user agents is completed through the bandwidth-optimized Collector Nodes, the execution of them is done by the compute-optimized Execution Nodes. Checking the execution result is then distributed among a more extensive set of Verification Nodes, which confirm the result is correct in a distributed and parallel manner. In contrast to more traditional blockchain architectures, Flow's Consensus Nodes do not execute the transaction. Instead, Verification Nodes report observed faulty executions to the Consensus Nodes, which adjudicate the received challenges and slash malicious actors.
In this paper, we detail the lifecycle of the transactions from the submission to the system until they are getting executed. The paper covers the Collector, Consensus, and Execution role. We provide a protocol specification of collecting the transactions, forming a block, and executing the resulting block. Moreover, we elaborate on the safety and liveness of the system concerning these processes.
△ Less
Submitted 18 February, 2020;
originally announced February 2020.
-
Decentralized utility- and locality-aware replication for heterogeneous DHT-based P2P cloud storage systems
Authors:
Yahya Hassanzadeh-Nazarabadi,
Alptekin Küpçü,
Öznur Özkasap
Abstract:
As a Distributed Hash Table (DHT), Skip Graph routing overlays are exploited in several peer-to-peer (P2P) services, including P2P cloud storage. The fully decentralized replication algorithms that are applicable to the Skip Graph-based P2P cloud storage fail on improving the performance of the system with respect to both the availability of replicas as well as their response time. Additionally, t…
▽ More
As a Distributed Hash Table (DHT), Skip Graph routing overlays are exploited in several peer-to-peer (P2P) services, including P2P cloud storage. The fully decentralized replication algorithms that are applicable to the Skip Graph-based P2P cloud storage fail on improving the performance of the system with respect to both the availability of replicas as well as their response time. Additionally, they presume the system as homogeneous with respect to the nodes' latency distribution, availability behavior, bandwidth, or storage. In this paper, we propose Pyramid, which is the first fully decentralized utility- and locality-aware replication approach for Skip Graph-based P2P cloud storage systems. Pyramid considers the nodes as heterogeneous with respect to their latency distribution, availability behavior, bandwidth, and storage. Pyramid is utility-aware as it maximizes the average available bandwidth of replicas per time slot (e.g., per hour). Additionally, Pyramid is locality-aware as it minimizes the average latency between nodes and their closest replica. Our simulation results show that compared to the state-of-the-art solutions that either perform good in utility-awareness, or in locality-awareness, our proposed Pyramid improves both the utility- and locality-awareness of replicas with a gain of about 1.2 and 1.1 times at the same time, respectively.
△ Less
Submitted 27 July, 2019;
originally announced July 2019.
-
LightChain: A DHT-based Blockchain for Resource Constrained Environments
Authors:
Yahya Hassanzadeh-Nazarabadi,
Alptekin Küpçü,
Öznur Özkasap
Abstract:
As an append-only distributed database, blockchain is utilized in a vast variety of applications including the cryptocurrency and Internet-of-Things (IoT). The existing blockchain solutions have downsides in communication and storage efficiency, convergence to centralization, and consistency problems. In this paper, we propose LightChain, which is the first blockchain architecture that operates ov…
▽ More
As an append-only distributed database, blockchain is utilized in a vast variety of applications including the cryptocurrency and Internet-of-Things (IoT). The existing blockchain solutions have downsides in communication and storage efficiency, convergence to centralization, and consistency problems. In this paper, we propose LightChain, which is the first blockchain architecture that operates over a Distributed Hash Table (DHT) of participating peers. LightChain is a permissionless blockchain that provides addressable blocks and transactions within the network, which makes them efficiently accessible by all the peers. Each block and transaction is replicated within the DHT of peers and is retrieved in an on-demand manner. Hence, peers in LightChain are not required to retrieve or keep the entire blockchain. LightChain is fair as all of the participating peers have a uniform chance of being involved in the consensus regardless of their influence such as hashing power or stake. LightChain provides a deterministic fork-resolving strategy as well as a blacklisting mechanism, and it is secure against colluding adversarial peers attacking the availability and integrity of the system. We provide mathematical analysis and experimental results on scenarios involving 10K nodes to demonstrate the security and fairness of LightChain. As we experimentally show in this paper, compared to the mainstream blockchains like Bitcoin and Ethereum, LightChain requires around 66 times less per node storage, and is around 380 times faster on bootstrapping a new node to the system, while each LightChain node is rewarded equally likely for participating in the protocol.
△ Less
Submitted 20 December, 2020; v1 submitted 31 March, 2019;
originally announced April 2019.
-
Interlaced: Fully decentralized churn stabilization for Skip Graph-based DHTs
Authors:
Yahya Hassanzadeh-Nazarabadi,
Alptekin Küpçü,
Öznur Özkasap
Abstract:
As a distributed hash table (DHT) routing overlay, Skip Graph is used in a variety of peer-to-peer (P2P) systems including cloud storage, social networks, and search engines. The overlay connectivity of P2P systems is negatively affected by the arrivals and departures of nodes to and from the system that is known as churn. Preserving connectivity of the overlay network (i.e., the reachability of e…
▽ More
As a distributed hash table (DHT) routing overlay, Skip Graph is used in a variety of peer-to-peer (P2P) systems including cloud storage, social networks, and search engines. The overlay connectivity of P2P systems is negatively affected by the arrivals and departures of nodes to and from the system that is known as churn. Preserving connectivity of the overlay network (i.e., the reachability of every pair of nodes) under churn is a performance challenge in every P2P system including the Skip Graph-based ones. The existing decentralized churn stabilization solutions that are applicable on Skip Graphs have intensive communication complexities, which leave them unable to provide a strong overlay connectivity, especially under high rates of churn.
In this paper, we propose Interlaced, a fully decentralized churn stabilization mechanism for Skip Graphs that provides drastically stronger overlay connectivity without changing the asymptotic complexity of the Skip Graph in terms of storage, computation, and communication. We also propose the Sliding Window De Bruijn Graph (SW-DBG) as a tool to predict the availability of nodes with high accuracy. Our simulation results show that in comparison to the best existing DHT-based solutions, Interlaced improves the overlay connectivity of Skip Graph under churn with the gain of about 1.81 times. A Skip Graph that benefits from Interlaced and SW-DBG is about 2.47 times faster on average in routing the queries under churn compared to the best existing solutions. We also present an adaptive extension of Interlaced to be applied on other DHTs, for example Kademlia.
△ Less
Submitted 18 March, 2019;
originally announced March 2019.
