Showing 1–7 of 7 results for author: Nagib, M

SafeSlice: Enabling SLA-Compliant O-RAN Slicing via Safe Deep Reinforcement Learning

Authors: Ahmad M. Nagib, Hatem Abou-Zeid, Hossam S. Hassanein

Abstract: Deep reinforcement learning (DRL)-based slicing policies have shown significant success in simulated environments but face challenges in physical systems such as open radio access networks (O-RANs) due to simulation-to-reality gaps. These policies often lack safety guarantees to ensure compliance with service level agreements (SLAs), such as the strict latency requirements of immersive application… ▽ More Deep reinforcement learning (DRL)-based slicing policies have shown significant success in simulated environments but face challenges in physical systems such as open radio access networks (O-RANs) due to simulation-to-reality gaps. These policies often lack safety guarantees to ensure compliance with service level agreements (SLAs), such as the strict latency requirements of immersive applications. As a result, a deployed DRL slicing agent may make resource allocation (RA) decisions that degrade system performance, particularly in previously unseen scenarios. Real-world immersive applications require maintaining SLA constraints throughout deployment to prevent risky DRL exploration. In this paper, we propose SafeSlice to address both the cumulative (trajectory-wise) and instantaneous (state-wise) latency constraints of O-RAN slices. We incorporate the cumulative constraints by designing a sigmoid-based risk-sensitive reward function that reflects the slices' latency requirements. Moreover, we build a supervised learning cost model as part of a safety layer that projects the slicing agent's RA actions to the nearest safe actions, fulfilling instantaneous constraints. We conduct an exhaustive experiment that supports multiple services, including real virtual reality (VR) gaming traffic, to investigate the performance of SafeSlice under extreme and changing deployment conditions. SafeSlice achieves reductions of up to 83.23% in average cumulative latency, 93.24% in instantaneous latency violations, and 22.13% in resource consumption compared to the baselines. The results also indicate SafeSlice's robustness to changing the threshold configurations of latency constraints, a vital deployment scenario that will be realized by the O-RAN paradigm to empower mobile network operators (MNOs). △ Less

Submitted 16 March, 2025; originally announced March 2025.

Comments: This article has been accepted for presentation in the IEEE International Conference on Machine Learning for Communication and Networking (ICMLCN) 2025

Predicting Human Impressions of Robot Performance During Navigation Tasks

Authors: Qiping Zhang, Nathan Tsoi, Mofeed Nagib, Booyeon Choi, Jie Tan, Hao-Tien Lewis Chiang, Marynel Vázquez

Abstract: Human impressions of robot performance are often measured through surveys. As a more scalable and cost-effective alternative, we investigate the possibility of predicting people's impressions of robot behavior using non-verbal behavioral cues and machine learning techniques. To this end, we first contribute the SEAN TOGETHER Dataset consisting of observations of an interaction between a person and… ▽ More Human impressions of robot performance are often measured through surveys. As a more scalable and cost-effective alternative, we investigate the possibility of predicting people's impressions of robot behavior using non-verbal behavioral cues and machine learning techniques. To this end, we first contribute the SEAN TOGETHER Dataset consisting of observations of an interaction between a person and a mobile robot in a VR simulation, together with impressions of robot performance provided by users on a 5-point scale. Second, we contribute analyses of how well humans and supervised learning techniques can predict perceived robot performance based on different observation types (like facial expression features, and features that describe the navigation behavior of the robot and pedestrians). Our results suggest that facial expressions alone provide useful information about human impressions of robot performance; but in the navigation scenarios that we considered, reasoning about spatial features in context is critical for the prediction task. Also, supervised learning techniques showed promise because they outperformed humans' predictions of robot performance in most cases. Further, when predicting robot performance as a binary classification task on unseen users' data, the F1 Score of machine learning models more than doubled in comparison to predicting performance on a 5-point scale. This suggested that the models can have good generalization capabilities, although they are better at telling the directionality of robot performance than predicting exact performance ratings. Based on our findings in simulation, we conducted a real-world demonstration in which a mobile robot uses a machine learning model to predict how a human that follows it perceives it. Finally, we discuss the implications of our results for implementing such supervised learning models in real-world navigation scenarios. △ Less

Submitted 4 November, 2024; v1 submitted 17 October, 2023; originally announced October 2023.

Safe and Accelerated Deep Reinforcement Learning-based O-RAN Slicing: A Hybrid Transfer Learning Approach

Authors: Ahmad M. Nagib, Hatem Abou-Zeid, Hossam S. Hassanein

Abstract: The open radio access network (O-RAN) architecture supports intelligent network control algorithms as one of its core capabilities. Data-driven applications incorporate such algorithms to optimize radio access network (RAN) functions via RAN intelligent controllers (RICs). Deep reinforcement learning (DRL) algorithms are among the main approaches adopted in the O-RAN literature to solve dynamic ra… ▽ More The open radio access network (O-RAN) architecture supports intelligent network control algorithms as one of its core capabilities. Data-driven applications incorporate such algorithms to optimize radio access network (RAN) functions via RAN intelligent controllers (RICs). Deep reinforcement learning (DRL) algorithms are among the main approaches adopted in the O-RAN literature to solve dynamic radio resource management problems. However, despite the benefits introduced by the O-RAN RICs, the practical adoption of DRL algorithms in real network deployments falls behind. This is primarily due to the slow convergence and unstable performance exhibited by DRL agents upon deployment and when encountering previously unseen network conditions. In this paper, we address these challenges by proposing transfer learning (TL) as a core component of the training and deployment workflows for the DRL-based closed-loop control of O-RAN functionalities. To this end, we propose and design a hybrid TL-aided approach that leverages the advantages of both policy reuse and distillation TL methods to provide safe and accelerated convergence in DRL-based O-RAN slicing. We conduct a thorough experiment that accommodates multiple services, including real VR gaming traffic to reflect practical scenarios of O-RAN slicing. We also propose and implement policy reuse and distillation-aided DRL and non-TL-aided DRL as three separate baselines. The proposed hybrid approach shows at least: 7.7% and 20.7% improvements in the average initial reward value and the percentage of converged scenarios, and a 64.6% decrease in reward variance while maintaining fast convergence and enhancing the generalizability compared with the baselines. △ Less

Submitted 18 September, 2023; v1 submitted 13 September, 2023; originally announced September 2023.

Comments: This paper has been accepted for publication in a future issue of IEEE Journal on Selected Areas in Communications (JSAC)

How Does Forecasting Affect the Convergence of DRL Techniques in O-RAN Slicing?

Authors: Ahmad M. Nagib, Hatem Abou-Zeid, Hossam S. Hassanein

Abstract: The success of immersive applications such as virtual reality (VR) gaming and metaverse services depends on low latency and reliable connectivity. To provide seamless user experiences, the open radio access network (O-RAN) architecture and 6G networks are expected to play a crucial role. RAN slicing, a critical component of the O-RAN paradigm, enables network resources to be allocated based on the… ▽ More The success of immersive applications such as virtual reality (VR) gaming and metaverse services depends on low latency and reliable connectivity. To provide seamless user experiences, the open radio access network (O-RAN) architecture and 6G networks are expected to play a crucial role. RAN slicing, a critical component of the O-RAN paradigm, enables network resources to be allocated based on the needs of immersive services, creating multiple virtual networks on a single physical infrastructure. In the O-RAN literature, deep reinforcement learning (DRL) algorithms are commonly used to optimize resource allocation. However, the practical adoption of DRL in live deployments has been sluggish. This is primarily due to the slow convergence and performance instabilities suffered by the DRL agents both upon initial deployment and when there are significant changes in network conditions. In this paper, we investigate the impact of time series forecasting of traffic demands on the convergence of the DRL-based slicing agents. For that, we conduct an exhaustive experiment that supports multiple services including real VR gaming traffic. We then propose a novel forecasting-aided DRL approach and its respective O-RAN practical deployment workflow to enhance DRL convergence. Our approach shows up to 22.8%, 86.3%, and 300% improvements in the average initial reward value, convergence rate, and number of converged scenarios respectively, enhancing the generalizability of the DRL agents compared with the implemented baselines. The results also indicate that our approach is robust against forecasting errors and that forecasting models do not have to be ideal. △ Less

Submitted 1 September, 2023; originally announced September 2023.

Comments: This article has been accepted for presentation in IEEE GLOBECOM 2023

The hunt for the Kármán "constant'' revisited

Authors: Peter A. Monkewitz, Hassan M. Nagib

Abstract: The logarithmic law of the wall, joining the inner, near-wall mean velocity profile (abbreviated MVP) in wall-bounded turbulent flows to the outer region, has been a permanent fixture of turbulence research for over hundred years, but there is still no general agreement on the value of the pre-factor, the inverse of the Kármán ``constant'' $κ$, or on its universality. The choice diagnostic tool to… ▽ More The logarithmic law of the wall, joining the inner, near-wall mean velocity profile (abbreviated MVP) in wall-bounded turbulent flows to the outer region, has been a permanent fixture of turbulence research for over hundred years, but there is still no general agreement on the value of the pre-factor, the inverse of the Kármán ``constant'' $κ$, or on its universality. The choice diagnostic tool to locate logarithmic parts of the MVP is to look for regions where the indicator function $Ξ$ (equal to the wall-normal coordinate $y^+$ times the mean velocity derivative $\dd U^+/\dd y^+$) is constant. In pressure driven flows however, such as channel and pipe flows, $Ξ$ is significantly affected by a term proportional to the wall-normal coordinate, of order $\mathcal{O}(\Reytau^{-1})$ in the inner expansion, but moving up across the overlap to the leading $\mathcal{O}(1)$ in the outer expansion. Here we show that, due to this linear overlap term, $\Reytau$'s well beyond $10^5$ are required to produce one decade of near constant $Ξ$ in channels and pipes. The problem is resolved by considering the common part of the inner asymptotic expansion carried to $\mathcal{O}(\Reytau^{-1})$, and the leading order of the outer expansion. This common part contains a \textit{superposition} of the log law and a linear term $S_0 \,y^+\Reytau^{-1}$, and corresponds to the linear part of $Ξ$, which, in channel and pipe, is concealed up to $y^+ \approx 500-1000$ by terms of the inner expansion. A new and robust method is devised to simultaneously determine $κ$ and $S_0$ in pressure driven flows at currently accessible $\Reytau$'s, yielding $κ$'s which are consistent with the $κ$'s deduced from the Reynolds number dependence of centerline velocities. A comparison with the zero-pressure-gradient turbulent boundary layer further clarifies the issues. △ Less

Submitted 12 July, 2023; v1 submitted 14 March, 2023; originally announced March 2023.

Toward Safe and Accelerated Deep Reinforcement Learning for Next-Generation Wireless Networks

Authors: Ahmad M. Nagib, Hatem Abou-zeid, Hossam S. Hassanein

Abstract: Deep reinforcement learning (DRL) algorithms have recently gained wide attention in the wireless networks domain. They are considered promising approaches for solving dynamic radio resource management (RRM) problems in next-generation networks. Given their capabilities to build an approximate and continuously updated model of the wireless network environments, DRL algorithms can deal with the mult… ▽ More Deep reinforcement learning (DRL) algorithms have recently gained wide attention in the wireless networks domain. They are considered promising approaches for solving dynamic radio resource management (RRM) problems in next-generation networks. Given their capabilities to build an approximate and continuously updated model of the wireless network environments, DRL algorithms can deal with the multifaceted complexity of such environments. Nevertheless, several challenges hinder the practical adoption of DRL in commercial networks. In this article, we first discuss two key practical challenges that are faced but rarely tackled when developing DRL-based RRM solutions. We argue that it is inevitable to address these DRL-related challenges for DRL to find its way to RRM commercial solutions. In particular, we discuss the need to have safe and accelerated DRL-based RRM solutions that mitigate the slow convergence and performance instability exhibited by DRL algorithms. We then review and categorize the main approaches used in the RRM domain to develop safe and accelerated DRL-based solutions. Finally, a case study is conducted to demonstrate the importance of having safe and accelerated DRL-based RRM solutions. We employ multiple variants of transfer learning (TL) techniques to accelerate the convergence of intelligent radio access network (RAN) slicing DRL-based controllers. We also propose a hybrid TL-based approach and sigmoid function-based rewards as examples of safe exploration in DRL-based RAN slicing. △ Less

Submitted 16 September, 2022; originally announced September 2022.

Comments: This article has been accepted for publication in a future issue of IEEE Network

A mechanistic model of separation bubble

Authors: R. Krechetnikov, J. E. Marsden, H. M. Nagib

Abstract: This work uncovers the low-dimensional nature the complex dynamics of actuated separated flows. Namely, motivated by the problem of model-based predictive control of separated flows, we identify the requirements on a model-based observer and the key variables and propose a prototype model in the case of thick airfoils as motivated by practical applications. The approach in this paper differs f… ▽ More This work uncovers the low-dimensional nature the complex dynamics of actuated separated flows. Namely, motivated by the problem of model-based predictive control of separated flows, we identify the requirements on a model-based observer and the key variables and propose a prototype model in the case of thick airfoils as motivated by practical applications. The approach in this paper differs fundamentally from the logic behind known models, which are either linear or based on POD-truncations and are unable to reflect even the crucial bifurcation and hysteresis inherent in separation phenomena. This new look at the problem naturally leads to several important implications, such as, firstly, uncovering the physical mechanisms for hysteresis, secondly, predicting a finite amplitude instability of the bubble, and thirdly to new issues to be studied theoretically and tested experimentally. More importantly, by employing systematic reasoning, the low-dimensional nature of these complex phenomena at the coarse level is revealed. △ Less

Submitted 8 October, 2007; originally announced October 2007.