-
A Review of Deep Reinforcement Learning in Serverless Computing: Function Scheduling and Resource Auto-Scaling
Authors:
Amjad Yousef Majid,
Eduard Marin
Abstract:
In the rapidly evolving field of serverless computing, efficient function scheduling and resource scaling are critical for optimizing performance and cost. This paper presents a comprehensive review of the application of Deep Reinforcement Learning (DRL) techniques in these areas. We begin by providing an overview of serverless computing, highlighting its benefits and challenges, with a particular…
▽ More
In the rapidly evolving field of serverless computing, efficient function scheduling and resource scaling are critical for optimizing performance and cost. This paper presents a comprehensive review of the application of Deep Reinforcement Learning (DRL) techniques in these areas. We begin by providing an overview of serverless computing, highlighting its benefits and challenges, with a particular focus on function scheduling and resource scaling. We then delve into the principles of deep reinforcement learning (DRL) and its potential for addressing these challenges. A systematic review of recent studies applying DRL to serverless computing is presented, covering various algorithms, models, and performances. Our analysis reveals that DRL, with its ability to learn and adapt from an environment, shows promising results in improving the efficiency of function scheduling and resource scaling in serverless computing. However, several challenges remain, including the need for more realistic simulation environments, handling of cold starts, and the trade-off between learning time and scheduling performance. We conclude by discussing potential future directions for this research area, emphasizing the need for more robust DRL models, better benchmarking methods, and the exploration of multi-agent reinforcement learning for more complex serverless architectures. This review serves as a valuable resource for researchers and practitioners aiming to understand and advance the application of DRL in serverless computing.
△ Less
Submitted 5 October, 2023;
originally announced November 2023.
-
Covy: An AI-powered Robot with a Compound Vision System for Detecting Breaches in Social Distancing
Authors:
Serge Saaybi,
Amjad Yousef Majid,
R Venkatesha Prasad,
Anis Koubaa,
Chris Verhoeven
Abstract:
This paper introduces a compound vision system that enables robots to localize people up to 15m away using a cheap camera. And, it proposes a robust navigation stack that combines Deep Reinforcement Learning (DRL) and a probabilistic localization method. To test the efficacy of these systems, we prototyped a low-cost mobile robot that we call Covy. Covy can be used for applications such as promoti…
▽ More
This paper introduces a compound vision system that enables robots to localize people up to 15m away using a cheap camera. And, it proposes a robust navigation stack that combines Deep Reinforcement Learning (DRL) and a probabilistic localization method. To test the efficacy of these systems, we prototyped a low-cost mobile robot that we call Covy. Covy can be used for applications such as promoting social distancing during pandemics or estimating the density of a crowd. We evaluated Covy's performance through extensive sets of experiments both in simulated and realistic environments. Our results show that Covy's compound vision algorithm doubles the range of the used depth camera, and its hybrid navigation stack is more robust than a pure DRL-based one.
△ Less
Submitted 23 August, 2022; v1 submitted 14 July, 2022;
originally announced July 2022.
-
Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey
Authors:
Amjad Yousef Majid,
Serge Saaybi,
Tomas van Rietbergen,
Vincent Francois-Lavet,
R Venkatesha Prasad,
Chris Verhoeven
Abstract:
Deep Reinforcement Learning (DRL) and Evolution Strategies (ESs) have surpassed human-level control in many sequential decision-making problems, yet many open challenges still exist. To get insights into the strengths and weaknesses of DRL versus ESs, an analysis of their respective capabilities and limitations is provided. After presenting their fundamental concepts and algorithms, a comparison i…
▽ More
Deep Reinforcement Learning (DRL) and Evolution Strategies (ESs) have surpassed human-level control in many sequential decision-making problems, yet many open challenges still exist. To get insights into the strengths and weaknesses of DRL versus ESs, an analysis of their respective capabilities and limitations is provided. After presenting their fundamental concepts and algorithms, a comparison is provided on key aspects such as scalability, exploration, adaptation to dynamic environments, and multi-agent learning. Then, the benefits of hybrid algorithms that combine concepts from DRL and ESs are highlighted. Finally, to have an indication about how they compare in real-world applications, a survey of the literature for the set of applications they support is provided.
△ Less
Submitted 28 September, 2021;
originally announced October 2021.
-
Multi-hop Backscatter Tag-to-Tag Networks
Authors:
Amjad Yousef Majid,
Michel Jansen,
Guillermo Ortas Delgado,
Kasım Sinan Yıldırım,
Przemysław Pawełczak
Abstract:
We characterize the performance of a backscatter tag-to-tag (T2T) multi-hop network. For this, we developed a discrete component-based backscatter T2T transceiver and a communication protocol suite. The protocol composed of a novel (i) flooding-based link control tailored towards backscatter transmission, and (ii) low-power listening MAC. The MAC design is based on the new insight that backscatter…
▽ More
We characterize the performance of a backscatter tag-to-tag (T2T) multi-hop network. For this, we developed a discrete component-based backscatter T2T transceiver and a communication protocol suite. The protocol composed of a novel (i) flooding-based link control tailored towards backscatter transmission, and (ii) low-power listening MAC. The MAC design is based on the new insight that backscatter reception is more energy costly than transmission. Our experiments show that multi-hopping extends the coverage of backscatter networks by enabling longer backward T2T links (tag far from the exciter sending to the tag close to the exciter). Four hops, for example, extend the communication range by a factor of two. Furthermore, we show that dead spots in multi-hop T2T networks are far less significant than those in the single-hop T2T networks.
△ Less
Submitted 29 January, 2019;
originally announced January 2019.
-
A Testbed for Transiently Powered Computers
Authors:
Henko Aantjes,
Amjad Y. Majid,
Przemysław Pawełczak
Abstract:
Transiently Powered Computers (TPCs) are novel devices that are battery-less and operate using only ambient energy. Therefore TPCs are prone to frequent power interruptions, and such the need for developing TPC-centric algorithms is a necessity. We advocate that only through a common experimental environment, accessible to everyone, a proper comparison of newly developed algorithms for TPCs can be…
▽ More
Transiently Powered Computers (TPCs) are novel devices that are battery-less and operate using only ambient energy. Therefore TPCs are prone to frequent power interruptions, and such the need for developing TPC-centric algorithms is a necessity. We advocate that only through a common experimental environment, accessible to everyone, a proper comparison of newly developed algorithms for TPCs can be realized. Moreover, only through access to various TPC testbeds---distributed geographically throughout the world---a proper applications testing and validation are possible. We enlist properties and features that any TPC testbed should have, calling for more coordinated action in this domain of TPC research. Finally, we present (to the best of our knowledge) world's first Internet-accessible testbed for TPCs.
△ Less
Submitted 24 June, 2016;
originally announced June 2016.
-
On the Synchronization of Intermittently Powered Wireless Embedded Systems
Authors:
Kasım Sinan Yıldırım,
Henko Aantjes,
Amjad Yousef Majid,
Przemysław Pawełczak
Abstract:
Battery-free computational RFID platforms, such as WISP (Wireless Identification and Sensing Platform), are emerging intermittently powered devices designed for replacing existing battery-powered sensor networks. As their applications become increasingly complex, we anticipate that synchronization (among others) to appear as one of crucial building blocks for collaborative and coordinated actions.…
▽ More
Battery-free computational RFID platforms, such as WISP (Wireless Identification and Sensing Platform), are emerging intermittently powered devices designed for replacing existing battery-powered sensor networks. As their applications become increasingly complex, we anticipate that synchronization (among others) to appear as one of crucial building blocks for collaborative and coordinated actions. With this paper we aim at providing initial observations regarding the synchronization of intermittently powered systems. In particular, we design and implement the first and very initial synchronization protocol for the WISP platform that provides explicit synchronization among individual WISPs that reside inside the communication range of a common RFID reader. Evaluations in our testbed showed that with our mechanism a synchronization error of approximately 1.5 milliseconds can be ensured between the RFID reader and a WISP tag.
△ Less
Submitted 6 June, 2016;
originally announced June 2016.
