-
In Search of a Lost Metric: Human Empowerment as a Pillar of Socially Conscious Navigation
Authors:
Vasanth Reddy Baddam,
Behdad Chalaki,
Vaishnav Tadiparthi,
Hossein Nourkhiz Mahjoub,
Ehsan Moradi-Pari,
Hoda Eldardiry,
Almuatazbellah Boker
Abstract:
In social robot navigation, traditional metrics like proxemics and behavior naturalness emphasize human comfort and adherence to social norms but often fail to capture an agent's autonomy and adaptability in dynamic environments. This paper introduces human empowerment, an information-theoretic concept that measures a human's ability to influence their future states and observe those changes, as a…
▽ More
In social robot navigation, traditional metrics like proxemics and behavior naturalness emphasize human comfort and adherence to social norms but often fail to capture an agent's autonomy and adaptability in dynamic environments. This paper introduces human empowerment, an information-theoretic concept that measures a human's ability to influence their future states and observe those changes, as a complementary metric for evaluating social compliance. This metric reveals how robot navigation policies can indirectly impact human empowerment. We present a framework that integrates human empowerment into the evaluation of social performance in navigation tasks. Through numerical simulations, we demonstrate that human empowerment as a metric not only aligns with intuitive social behavior, but also shows statistically significant differences across various robot navigation policies. These results provide a deeper understanding of how different policies affect social compliance, highlighting the potential of human empowerment as a complementary metric for future research in social navigation.
△ Less
Submitted 2 January, 2025;
originally announced January 2025.
-
Learning Emergence of Interaction Patterns across Independent RL Agents in Multi-Agent Environments
Authors:
Vasanth Reddy Baddam,
Suat Gumussoy,
Almuatazbellah Boker,
Hoda Eldardiry
Abstract:
Many real-world problems, such as controlling swarms of drones and urban traffic, naturally lend themselves to modeling as multi-agent reinforcement learning (RL) problems. However, existing multi-agent RL methods often suffer from scalability challenges, primarily due to the introduction of communication among agents. Consequently, a key challenge lies in adapting the success of deep learning in…
▽ More
Many real-world problems, such as controlling swarms of drones and urban traffic, naturally lend themselves to modeling as multi-agent reinforcement learning (RL) problems. However, existing multi-agent RL methods often suffer from scalability challenges, primarily due to the introduction of communication among agents. Consequently, a key challenge lies in adapting the success of deep learning in single-agent RL to the multi-agent setting. In response to this challenge, we propose an approach that fundamentally reimagines multi-agent environments. Unlike conventional methods that model each agent individually with separate networks, our approach, the Bottom Up Network (BUN), adopts a unique perspective. BUN treats the collective of multi-agents as a unified entity while employing a specialized weight initialization strategy that promotes independent learning. Furthermore, we dynamically establish connections among agents using gradient information, enabling coordination when necessary while maintaining these connections as limited and sparse to effectively manage the computational budget. Our extensive empirical evaluations across a variety of cooperative multi-agent scenarios, including tasks such as cooperative navigation and traffic control, consistently demonstrate BUN's superiority over baseline methods with substantially reduced computational costs.
△ Less
Submitted 3 October, 2024;
originally announced October 2024.
-
Data-Driven Near-Optimal Control of Nonlinear Systems Over Finite Horizon
Authors:
Vasanth Reddy,
Hoda Eldardiry,
Almuatazbellah Boker
Abstract:
We examine the problem of two-point boundary optimal control of nonlinear systems over finite-horizon time periods with unknown model dynamics by employing reinforcement learning. We use techniques from singular perturbation theory to decompose the control problem over the finite horizon into two sub-problems, each solved over an infinite horizon. In the process, we avoid the need to solve the tim…
▽ More
We examine the problem of two-point boundary optimal control of nonlinear systems over finite-horizon time periods with unknown model dynamics by employing reinforcement learning. We use techniques from singular perturbation theory to decompose the control problem over the finite horizon into two sub-problems, each solved over an infinite horizon. In the process, we avoid the need to solve the time-varying Hamilton-Jacobi-Bellman equation. Using a policy iteration method, which is made feasible as a result of this decomposition, it is now possible to learn the controller gains of both sub-problems. The overall control is then formed by piecing together the solutions to the two sub-problems. We show that the performance of the proposed closed-loop system approaches that of the model-based optimal performance as the time horizon gets long. Finally, we provide three simulation scenarios to support the paper's claims.
△ Less
Submitted 8 June, 2023;
originally announced June 2023.
-
Identification of Power System Oscillation Modes using Blind Source Separation based on Copula Statistic
Authors:
Pooja Algikar,
Lamine Mili,
Mohsen Ben Hassine,
Somayeh Yarahmadi,
Almuatazbellah,
Boker
Abstract:
The dynamics of a power system with large penetration of renewable energy resources are becoming more nonlinear due to the intermittence of these resources and the switching of their power electronic devices. Therefore, it is crucial to accurately identify the dynamical modes of oscillation of such a power system when it is subject to disturbances to initiate appropriate preventive or corrective c…
▽ More
The dynamics of a power system with large penetration of renewable energy resources are becoming more nonlinear due to the intermittence of these resources and the switching of their power electronic devices. Therefore, it is crucial to accurately identify the dynamical modes of oscillation of such a power system when it is subject to disturbances to initiate appropriate preventive or corrective control actions. In this paper, we propose a high-order blind source identification (HOBI) algorithm based on the copula statistic to address these non-linear dynamics in modal analysis. The method combined with Hilbert transform (HOBI-HT) and iteration procedure (HOBMI) can identify all the modes as well as the model order from the observation signals obtained from the number of channels as low as one. We access the performance of the proposed method on numerical simulation signals and recorded data from a simulation of time domain analysis on the classical 11-Bus 4-Machine test system. Our simulation results outperform the state-of-the-art method in accuracy and effectiveness.
△ Less
Submitted 7 February, 2023;
originally announced February 2023.
-
Singular Perturbation-based Reinforcement Learning of Two-Point Boundary Optimal Control Systems
Authors:
Vasanth Reddy,
Hoda Eldardiry,
Almuatazbellah Boker
Abstract:
This work presents a technique for learning systems, where the learning process is guided by knowledge of the physics of the system. In particular, we solve the problem of the two-point boundary optimal control problem of linear time-varying systems with unknown model dynamics using reinforcement learning. Borrowing techniques from singular perturbation theory, we transform the time-varying optima…
▽ More
This work presents a technique for learning systems, where the learning process is guided by knowledge of the physics of the system. In particular, we solve the problem of the two-point boundary optimal control problem of linear time-varying systems with unknown model dynamics using reinforcement learning. Borrowing techniques from singular perturbation theory, we transform the time-varying optimal control problem into a couple of time-invariant subproblems. This allows the utilization of an off-policy iteration method to learn the controller gains. We show that the performance of the learning-based controller approximates that of the model-based optimal controller and the accuracy of the approximation improves as the time horizon of the control problem increases. Finally, we provide a simulation example to verify the results of the paper.
△ Less
Submitted 29 April, 2021; v1 submitted 19 April, 2021;
originally announced April 2021.
-
Convergence Rates of Distributed Consensus over Cluster Networks: A Two-Time-Scale Approach
Authors:
Amit Dutta,
Almuatazbellah M. Boker,
Thinh T. Doan
Abstract:
We study the popular distributed consensus method over networks composed of a number of densely connected clusters with a sparse connection between them. In these cluster networks, the method often constitutes two-time-scale dynamics, where the internal nodes within each cluster reach consensus quickly relative to the aggregate nodes across clusters. Our main contribution is to provide the rate of…
▽ More
We study the popular distributed consensus method over networks composed of a number of densely connected clusters with a sparse connection between them. In these cluster networks, the method often constitutes two-time-scale dynamics, where the internal nodes within each cluster reach consensus quickly relative to the aggregate nodes across clusters. Our main contribution is to provide the rate of the distributed consensus method, which characterize explicitly the impacts of the internal and external graphs on the performance of this method. Our main result shows that this rate converges exponentially and only scales with a few number of nodes, which is relatively small to the size of the network.
The key technique in our analysis is to consider a Lyapunov function which captures the impacts of different time-scale dynamics on the convergence of the method. Our approach avoids using model reduction, which is the typical way according to singular perturbation theory and relies on relatively simple definitions of the slow and fast variables. In addition, Lyapunov analysis allows us to derive the rate of distributed consensus methods over cluster networks, which is missing from the existing works using singular perturbation theory. We illustrate our theoretical results by a number of numerical simulations over different cluster networks.
△ Less
Submitted 12 September, 2022; v1 submitted 15 April, 2021;
originally announced April 2021.
-
Semi-global Output Feedback Stabilization of Non-Minimum Phase Nonlinear Systems
Authors:
Almuatazbellah M. Boker,
Hassan K. Khalil
Abstract:
We solve the problem of output feedback stabilization of a class of nonlinear systems, which may have unstable zero dynamics. We allow for any globally stabilizing full state feedback control scheme to be used as long as it satisfies a particular ISS condition. We show semi-global stability of the origin of the closed-loop system and also the recovery of the performance of an auxiliary system usin…
▽ More
We solve the problem of output feedback stabilization of a class of nonlinear systems, which may have unstable zero dynamics. We allow for any globally stabilizing full state feedback control scheme to be used as long as it satisfies a particular ISS condition. We show semi-global stability of the origin of the closed-loop system and also the recovery of the performance of an auxiliary system using a full-order observer. This observer is based on the use of an extended high-gain observer to provide estimates of the output and its derivatives plus a signal used by an extended Kalman filter to provide estimates of the remaining states. Finally, we provide a simulation example that illustrates the design procedure.
△ Less
Submitted 25 July, 2016;
originally announced July 2016.
