Showing 1–34 of 34 results for author: Kumar, P R

Optimal demand response policies for inertial thermal loads under stochastic renewable sources

Authors: Gaurav Sharma, P R Kumar

Abstract: In this paper, we consider the problem of preferentially utilizing intermittent renewable power, such as wind, optimally to support thermal inertial loads in a microgrid environment. Thermal inertial loads can be programmed to preferentially consume from renewable sources. The flexibility in power consumption of inertial loads therefore can be used to absorb the fluctuations in intermittently avai… ▽ More In this paper, we consider the problem of preferentially utilizing intermittent renewable power, such as wind, optimally to support thermal inertial loads in a microgrid environment. Thermal inertial loads can be programmed to preferentially consume from renewable sources. The flexibility in power consumption of inertial loads therefore can be used to absorb the fluctuations in intermittently available renewable power sources, and promote reduction of fossil fuel based costly non-renewable generation. Under a model which promotes renewable consumption by penalizing the non-renewable, but does not account for variations in the end-user requirements, the optimal solution leads to all the users' temperatures behave in a lockstep fashion, that is the power is allocated in such a fashion that all the temperatures are brought to a common value and they are kept the same after that point, resulting in synchronization among all the loads. In the first part, we showed that under a model which additionally penalizes the comfort range violation, the optimal solution is in-fact of desynchronization nature, where the temperatures are intentionally kept apart to avoid power surges resulting from simultaneous comfort violation from many loads. △ Less

Submitted 5 December, 2024; originally announced December 2024.

The Transmission Value of Energy Storage and Fundamental Limitations

Authors: Qian Zhang, P. R. Kumar, Le Xie

Abstract: This study addresses the transmission value of energy storage in electric grids. The inherent connection between storage and transmission infrastructure is captured from a "cumulative energy" perspective, which enables the reformulating of the conventional optimization problem by employing line power flow as the decision variable. The study also establishes the theoretical limitations of both stor… ▽ More This study addresses the transmission value of energy storage in electric grids. The inherent connection between storage and transmission infrastructure is captured from a "cumulative energy" perspective, which enables the reformulating of the conventional optimization problem by employing line power flow as the decision variable. The study also establishes the theoretical limitations of both storage and transmission lines that can be replaced by each other, providing explicit closed-form expressions for the minimum capacity needed. As a key departure from conventional practice in which transmission lines are designed according to the peak power delivery needs, with sufficient storage capacity, the transmission line capacity can be designed based on the average power delivery needs. The models of this paper only rely on a few basic assumptions, paving the way for understanding future storage as a transmission asset market design. Numerical experiments based on 2-bus, modified RTS 24-bus, RTS-GMLC, and Texas synthetic power systems illustrate the results. △ Less

Submitted 12 July, 2024; originally announced July 2024.

Seeing the Unseen: The REVEAL protocol to expose the wireless Man-in-the-Middle

Authors: Santosh Ganji, P R Kumar

Abstract: A Man-in-the-Middle (MiM) can collect over-the-air packets whether from a mobile or a base station, process them, possibly modify them, and forward them to the intended receiver. This paper exhibits the REVEAL protocol that can detect a MiM, whether it has half duplex capability, full duplex capability, or double full duplex capability. Protocol is based on synchronizing clocks between the mobile… ▽ More A Man-in-the-Middle (MiM) can collect over-the-air packets whether from a mobile or a base station, process them, possibly modify them, and forward them to the intended receiver. This paper exhibits the REVEAL protocol that can detect a MiM, whether it has half duplex capability, full duplex capability, or double full duplex capability. Protocol is based on synchronizing clocks between the mobile and the base station, with the MiM being detected if it interferes in the synchronization process. Once synchronized, the REVEAL protocol creates a sequence of challenge packets where the transmission times of the packets, their durations, and their frequencies, are chosen to create conflicts at the MiM, and make it impossible for the MiM to function. We implement the REVEAL protocol for detecting a MiM in 4G technology. We instantiate a MiM between the 4G/5G base station and a mobile, and exhibit the successful detection mechanisms. With the shared source code, our work can be reproduced using open software defined cellular networks with off-the-shelf devices △ Less

Submitted 14 June, 2024; v1 submitted 17 August, 2023; originally announced August 2023.

Finite Time Regret Bounds for Minimum Variance Control of Autoregressive Systems with Exogenous Inputs

Authors: Rahul Singh, Akshay Mete, Avik Kar, P. R. Kumar

Abstract: Minimum variance controllers have been employed in a wide-range of industrial applications. A key challenge experienced by many adaptive controllers is their poor empirical performance in the initial stages of learning. In this paper, we address the problem of initializing them so that they provide acceptable transients, and also provide an accompanying finite-time regret analysis, for adaptive mi… ▽ More Minimum variance controllers have been employed in a wide-range of industrial applications. A key challenge experienced by many adaptive controllers is their poor empirical performance in the initial stages of learning. In this paper, we address the problem of initializing them so that they provide acceptable transients, and also provide an accompanying finite-time regret analysis, for adaptive minimum variance control of an auto-regressive system with exogenous inputs (ARX). Following [3], we consider a modified version of the Certainty Equivalence (CE) adaptive controller, which we call PIECE, that utilizes probing inputs for exploration. We show that it has a $C \log T$ bound on the regret after $T$ time-steps for bounded noise, and $C\log^2 T$ in the case of sub-Gaussian noise. The simulation results demonstrate the advantage of PIECE over the algorithm proposed in [3] as well as the standard Certainty Equivalence controller especially in the initial learning phase. To the best of our knowledge, this is the first work that provides finite-time regret bounds for an adaptive minimum variance controller. △ Less

Submitted 26 May, 2023; originally announced May 2023.

TERRA: Beam Management for Outdoor mm-Wave Networks

Authors: Santosh Ganji, Jaewon Kim, Romil Sonigra, P. R. Kumar

Abstract: mm-Wave communication systems use narrow directional beams due to the spectrum's characteristic nature: high path and penetration losses. The mobile and the base station primarily employ beams in line of sight (LoS) direction and when needed in non-line of sight direction. Beam management protocol adapts the base station and mobile side beam direction during user mobility and to sustain the link d… ▽ More mm-Wave communication systems use narrow directional beams due to the spectrum's characteristic nature: high path and penetration losses. The mobile and the base station primarily employ beams in line of sight (LoS) direction and when needed in non-line of sight direction. Beam management protocol adapts the base station and mobile side beam direction during user mobility and to sustain the link during blockages. To avoid outage in transient pedestrian blockage of the LoS path, the mobile uses reflected or NLoS path available in indoor environments. Reflected paths can sustain time synchronization and maintain connectivity during temporary blockages. In outdoor environments, such reflections may not be available and prior work relied on dense base station deployment or co-ordinated multi-point access to address outage problem. Instead of dense and hence cost-intensive network deployments, we found experimentally that the mobile can capitalize on ground reflection. We developed TERRA protocol to effectively handle mobile side beam direction during transient blockage events. TERRA avoids outage during pedestrian blockages 84.5 $\%$ of the time in outdoor environments on concrete and gravel surfaces. TERRA also enables the mobile to perform a soft handover to a reserve neighbor base station in the event of a permanent blockage, without requiring any side information, unlike the existing works. Evaluations show that TERRA maintains received signal strength close to the optimal solution while keeping track of the neighbor base station. △ Less

Submitted 10 January, 2023; originally announced January 2023.

Energy System Digitization in the Era of AI: A Three-Layered Approach towards Carbon Neutrality

Authors: Le Xie, Tong Huang, Xiangtian Zheng, Yan Liu, Mengdi Wang, Vijay Vittal, P. R. Kumar, Srinivas Shakkottai, Yi Cui

Abstract: The transition towards carbon-neutral electricity is one of the biggest game changers in addressing climate change since it addresses the dual challenges of removing carbon emissions from the two largest sectors of emitters: electricity and transportation. The transition to a carbon-neutral electric grid poses significant challenges to conventional paradigms of modern grid planning and operation.… ▽ More The transition towards carbon-neutral electricity is one of the biggest game changers in addressing climate change since it addresses the dual challenges of removing carbon emissions from the two largest sectors of emitters: electricity and transportation. The transition to a carbon-neutral electric grid poses significant challenges to conventional paradigms of modern grid planning and operation. Much of the challenge arises from the scale of the decision making and the uncertainty associated with the energy supply and demand. Artificial Intelligence (AI) could potentially have a transformative impact on accelerating the speed and scale of carbon-neutral transition, as many decision making processes in the power grid can be cast as classic, though challenging, machine learning tasks. We point out that to amplify AI's impact on carbon-neutral transition of the electric energy systems, the AI algorithms originally developed for other applications should be tailored in three layers of technology, markets, and policy. △ Less

Submitted 2 November, 2022; originally announced November 2022.

Comments: To be published in Patterns (Cell Press)

Terra: Blockage Resilience in Outdoor mmWave Networks

Authors: Santosh Ganji, Jaewon Kim, P. R. Kumar

Abstract: We address the problem of pedestrian blockage in outdoor mm-Wave networks, which can disrupt Line of Sight (LoS) communication and result in an outage. This necessitates either reacquisition as a new user that can take up to 1.28 sec in 5G New Radio, disrupting high-performance applications, or handover to a different base station (BS), which however requires very costly dense deployments to ensur… ▽ More We address the problem of pedestrian blockage in outdoor mm-Wave networks, which can disrupt Line of Sight (LoS) communication and result in an outage. This necessitates either reacquisition as a new user that can take up to 1.28 sec in 5G New Radio, disrupting high-performance applications, or handover to a different base station (BS), which however requires very costly dense deployments to ensure multiple base stations are always visible to a mobile outdoors. We have found that there typically exists a strong ground reflection from concrete and gravel surfaces, within 4-6 dB of received signal strength (RSS) of LoS paths. The mobile can switch to such a Non-Line Sight (NLoS) beam to sustain the link for use as a control channel during a blockage event. This allows the mobile to maintain time-synchronization with the base station, allowing it to revert to the LoS path when the temporary blockage disappears. We present a protocol, Terra, to quickly discover, cache, and employ ground reflections. It can be used in most outdoor built environments since pedestrian blockages typically last only a few hundred milliseconds. △ Less

Submitted 25 September, 2022; originally announced September 2022.

On an Information and Control Architecture for Future Electric Energy Systems

Authors: Le Xie, Tong Huang, P. R. Kumar, Anupam A. Thatte, Sanjoy K. Mitter

Abstract: This paper presents considerations towards an information and control architecture for future electric energy systems driven by massive changes resulting from the societal goals of decarbonization and electrification. This paper describes the new requirements and challenges of an extended information and control architecture that need to be addressed for continued reliable delivery of electricity.… ▽ More This paper presents considerations towards an information and control architecture for future electric energy systems driven by massive changes resulting from the societal goals of decarbonization and electrification. This paper describes the new requirements and challenges of an extended information and control architecture that need to be addressed for continued reliable delivery of electricity. It identifies several new actionable information and control loops, along with their spatial and temporal scales of operation, which can together meet the needs of future grids and enable deep decarbonization of the electricity sector. The present architecture of electric power grids designed in a different era is thereby extensible to allow the incorporation of increased renewables and other emerging electric loads. △ Less

Submitted 4 November, 2022; v1 submitted 31 May, 2022; originally announced June 2022.

Comments: This paper is accepted, to appear in the Proceedings of the IEEE

Augmented RBMLE-UCB Approach for Adaptive Control of Linear Quadratic Systems

Authors: Akshay Mete, Rahul Singh, P. R. Kumar

Abstract: We consider the problem of controlling an unknown stochastic linear system with quadratic costs - called the adaptive LQ control problem. We re-examine an approach called ''Reward Biased Maximum Likelihood Estimate'' (RBMLE) that was proposed more than forty years ago, and which predates the ''Upper Confidence Bound'' (UCB) method as well as the definition of ''regret'' for bandit problems. It sim… ▽ More We consider the problem of controlling an unknown stochastic linear system with quadratic costs - called the adaptive LQ control problem. We re-examine an approach called ''Reward Biased Maximum Likelihood Estimate'' (RBMLE) that was proposed more than forty years ago, and which predates the ''Upper Confidence Bound'' (UCB) method as well as the definition of ''regret'' for bandit problems. It simply added a term favoring parameters with larger rewards to the criterion for parameter estimation. We show how the RBMLE and UCB methods can be reconciled, and thereby propose an Augmented RBMLE-UCB algorithm that combines the penalty of the RBMLE method with the constraints of the UCB method, uniting the two approaches to optimism in the face of uncertainty. We establish that theoretically, this method retains $\Tilde{\mathcal{O}}(\sqrt{T})$ regret, the best-known so far. We further compare the empirical performance of the proposed Augmented RBMLE-UCB and the standard RBMLE (without the augmentation) with UCB, Thompson Sampling, Input Perturbation, Randomized Certainty Equivalence and StabL on many real-world examples including flight control of Boeing 747 and Unmanned Aerial Vehicle. We perform extensive simulation studies showing that the Augmented RBMLE consistently outperforms UCB, Thompson Sampling and StabL by a huge margin, while it is marginally better than Input Perturbation and moderately better than Randomized Certainty Equivalence. △ Less

Submitted 24 March, 2023; v1 submitted 25 January, 2022; originally announced January 2022.

Comments: 36th Conference on Neural Information Processing Systems (NeurIPS 2022). https://openreview.net/forum?id=7pNV4PCjbQy

Overcoming Pedestrian Blockage in mm-Wave Bands using Ground Reflections

Authors: Santosh Ganji, Romil Sonigra, P. R. Kumar

Abstract: mm-Wave communication employs directional beams to overcome high path loss. High data rate communication is typically along line-of-sight (LoS). In outdoor environments, such communication is susceptible to temporary blockage by pedestrians interposed between the transmitter and receiver. It results in outages in which the user is lost, and has to be reacquired as a new user, severely disrupting i… ▽ More mm-Wave communication employs directional beams to overcome high path loss. High data rate communication is typically along line-of-sight (LoS). In outdoor environments, such communication is susceptible to temporary blockage by pedestrians interposed between the transmitter and receiver. It results in outages in which the user is lost, and has to be reacquired as a new user, severely disrupting interactive and high throughput applications. It has been presumed that the solution is to have a densely deployed set of base stations that will allow the mobile to perform a handover to a different non-blocked base station every time a current base station is blocked. This is however a very costly solution for outdoor environments. Through extensive experiments we show that it is possible to exploit a strong ground reflection with a received signal strength (RSS) about 4dB less than the LoS path in outdoor built environments with concrete or gravel surfaces, for beams that are narrow in azimuth but wide in zenith. While such reflected paths cannot support the high data rates of LoS paths, they can support control channel communication, and, importantly, sustain time synchronization between the mobile and the base station. This allows a mobile to quickly recover to the LoS path upon the cessation of the temporary blockage, which typically lasts a few hundred milliseconds. We present a simple in-band protocol that quickly discovers ground reflected radiation and uses it to recover the LoS link when the temporary blockage disappears. △ Less

Submitted 16 November, 2021; v1 submitted 26 October, 2021; originally announced October 2021.

BeamSurfer: Minimalist Beam Management of Mobile mm-Wave Devices

Authors: Santosh Ganji, Tzu-Hsiang Lin, Francisco A. Espinal, P. R. Kumar

Abstract: Management of narrow directional beams is critical for mm-wave communication systems. Translational or rotational motion of the user can cause misalignment of transmit and receive beams with the base station losing track of the mobile. Reacquiring the user can take about one second in 5G NewRadio systems and significantly impair performance of applications, besides being energy intensive. It is th… ▽ More Management of narrow directional beams is critical for mm-wave communication systems. Translational or rotational motion of the user can cause misalignment of transmit and receive beams with the base station losing track of the mobile. Reacquiring the user can take about one second in 5G NewRadio systems and significantly impair performance of applications, besides being energy intensive. It is therefore important to manage beams to continually maintain high received signal strength and prevent outages. It is also important to be able to recover from sudden but transient blockage caused by a handor face interposed in the Line-of-Sight (LoS) path. This work presents a beam management protocol called BeamSurfer that is targeted to the use case of users walking indoors near a base station. It is designed to be minimalistic, employing only in-band information, and not requiring knowledge such as location or orientation of the mobile device or any additional sensors. Evaluations under pedestrian mobility show that96%of the time it employs beams that are within3dB of what an omniscient Oracle would have chosen as the pair of optimal transmit-receive LoS beams. It also recovers from transient LoS blockage by continuing to maintain control packet communication over a pre-selected reflected path that preserves time-synchronization with the base station, which allows it to rapidly recover to the LoS link as soon as it is unblocked △ Less

Submitted 4 October, 2021; originally announced October 2021.

Silent Tracker: In-band Beam Management for Soft Handover for mm-Wave Networks

Authors: Santosh Ganji, Tzu-Hsiang Lin, Jaewon Kim, P. R. Kumar

Abstract: In mm-wave networks, cell sizes are small due to high path and penetration losses. Mobiles need to frequently switch softly from one cell to another to preserve network connections and context. Each soft handover involves the mobile performing directional neighbor cell search, tracking cell beam, completing cell access request, and finally, context switching. The mobile must independently discover… ▽ More In mm-wave networks, cell sizes are small due to high path and penetration losses. Mobiles need to frequently switch softly from one cell to another to preserve network connections and context. Each soft handover involves the mobile performing directional neighbor cell search, tracking cell beam, completing cell access request, and finally, context switching. The mobile must independently discover cell beams, derive timing information, and maintain beam alignment throughout the process to avoid packet loss and hard handover. We propose Silent tracker which enables a mobile to reliably manage handover events by maintaining an aligned beam until the successful handover completion. It is entirely in-band beam mechanism that does not need any side information. Experimental evaluations show that Silent Tracker maintains the mobile's receive beam aligned to the potential target base station's transmit beam till the successful conclusion of handover in three mobility scenarios: human walk, device rotation, and 20 mph vehicular speed. △ Less

Submitted 17 July, 2021; originally announced July 2021.

An Efficient Network Solver for Dynamic Simulation of Power Systems Based on Hierarchical Inverse Computation and Modification

Authors: Lu Zhang, Bin Wang, Vivek Sarin, Weiping Shi, P. R. Kumar, Le Xie

Abstract: In power system dynamic simulation, up to 90% of the computational time is devoted to solve the network equations, i.e., a set of linear equations. Traditional approaches are based on sparse LU factorization, which is inherently sequential. In this paper, an inverse-based network solution is proposed by a hierarchical method for computing and store the approximate inverse of the conductance matrix… ▽ More In power system dynamic simulation, up to 90% of the computational time is devoted to solve the network equations, i.e., a set of linear equations. Traditional approaches are based on sparse LU factorization, which is inherently sequential. In this paper, an inverse-based network solution is proposed by a hierarchical method for computing and store the approximate inverse of the conductance matrix in electromagnetic transient (EMT) simulations. The proposed method can also efficiently update the inverse by modifying only local sub-matrices to reflect changes in the network, e.g., loss of a line. Experiments on a series of simplified 179-bus Western Interconnection demonstrate the advantages of the proposed methods. △ Less

Submitted 5 July, 2021; v1 submitted 22 May, 2021; originally announced May 2021.

Comments: 6 pages, 10 figures

UNBLOCK: Low Complexity Transient Blockage Recovery for Mobile mm-Wave Devices

Authors: Santosh Ganji, Tzu-Hsiang Lin, Francisco A. Espinal, P. R. Kumar

Abstract: Directional radio beams are used in the mm-Wave band to combat the high path loss. The mm-Wave band also suffers from high penetration losses from drywall, wood, glass, concrete, etc., and also the human body. Hence, as a mobile user moves, the Line of Sight (LoS) path between the mobile and the Base Station (BS) can be blocked by objects interposed in the path, causing loss of the link. A mobile… ▽ More Directional radio beams are used in the mm-Wave band to combat the high path loss. The mm-Wave band also suffers from high penetration losses from drywall, wood, glass, concrete, etc., and also the human body. Hence, as a mobile user moves, the Line of Sight (LoS) path between the mobile and the Base Station (BS) can be blocked by objects interposed in the path, causing loss of the link. A mobile with a lost link will need to be re-acquired as a new user by initial access, a process that can take up to a second, causing disruptions to applications. UNBLOCK is a protocol that allows a mobile to recover from transient blockages, such as those caused by a human hand or another human walking into the line of path or other temporary occlusions by objects, which typically disappear within the order of $100$ ms, without having to go through re-acquisition. UNBLOCK is based on extensive experimentation in office type environments which has shown that while a LoS path is blocked, there typically exists a Non-LoS path, i.e., a reflected path through scatterers, with a loss within about $10$ dB of the LoS path. UNBLOCK proactively keeps such a NLoS path in reserve, to be used when blockage happens, typically without any warning. UNBLOCK uses this NLoS path to maintain time-synchronization with the BS until the blockage disappears, as well as to search for a better NLoS path if available. When the transient blockage disappears, it reestablishes LoS communication at the epochs that have been scheduled by the BS for communication with the mobile. △ Less

Submitted 23 February, 2021; originally announced April 2021.

Detection of Cyber Attacks in Renewable-rich Microgrids Using Dynamic Watermarking

Authors: Tong Huang, Bin Wang, Jorge Ramos-Ruiz, Prasad Enjeti, P. R. Kumar, Le Xie

Abstract: This paper presents the first demonstration of using an active mechanism to defend renewable-rich microgrids against cyber attacks. Cyber vulnerability of the renewable-rich microgrids is identified. The defense mechanism based on dynamic watermarking is proposed for detecting cyber anomalies in microgrids. The proposed mechanism is easily implementable and it has theoretically provable performanc… ▽ More This paper presents the first demonstration of using an active mechanism to defend renewable-rich microgrids against cyber attacks. Cyber vulnerability of the renewable-rich microgrids is identified. The defense mechanism based on dynamic watermarking is proposed for detecting cyber anomalies in microgrids. The proposed mechanism is easily implementable and it has theoretically provable performance in term of detecting cyber attacks. The effectiveness of the proposed mechanism is tested and validated in a renewable-rich microgrid. △ Less

Submitted 25 November, 2020; originally announced November 2020.

Comments: This is a paper appeared at IEEE PES General Meeting 2020

Reward Biased Maximum Likelihood Estimation for Reinforcement Learning

Authors: Akshay Mete, Rahul Singh, Xi Liu, P. R. Kumar

Abstract: The Reward-Biased Maximum Likelihood Estimate (RBMLE) for adaptive control of Markov chains was proposed to overcome the central obstacle of what is variously called the fundamental "closed-identifiability problem" of adaptive control, the "dual control problem", or, contemporaneously, the "exploration vs. exploitation problem". It exploited the key observation that since the maximum likelihood pa… ▽ More The Reward-Biased Maximum Likelihood Estimate (RBMLE) for adaptive control of Markov chains was proposed to overcome the central obstacle of what is variously called the fundamental "closed-identifiability problem" of adaptive control, the "dual control problem", or, contemporaneously, the "exploration vs. exploitation problem". It exploited the key observation that since the maximum likelihood parameter estimator can asymptotically identify the closed-transition probabilities under a certainty equivalent approach, the limiting parameter estimates must necessarily have an optimal reward that is less than the optimal reward attainable for the true but unknown system. Hence it proposed a counteracting reverse bias in favor of parameters with larger optimal rewards, providing a solution to the fundamental problem alluded to above. It thereby proposed an optimistic approach of favoring parameters with larger optimal rewards, now known as "optimism in the face of uncertainty". The RBMLE approach has been proved to be long-term average reward optimal in a variety of contexts. However, modern attention is focused on the much finer notion of "regret", or finite-time performance. Recent analysis of RBMLE for multi-armed stochastic bandits and linear contextual bandits has shown that it not only has state-of-the-art regret, but it also exhibits empirical performance comparable to or better than the best current contenders, and leads to strikingly simple index policies. Motivated by this, we examine the finite-time performance of RBMLE for reinforcement learning tasks that involve the general problem of optimal control of unknown Markov Decision Processes. We show that it has a regret of $\mathcal{O}( \log T)$ over a time horizon of $T$ steps, similar to state-of-the-art algorithms. Simulation studies show that RBMLE outperforms other algorithms such as UCRL2 and Thompson Sampling. △ Less

Submitted 15 May, 2021; v1 submitted 16 November, 2020; originally announced November 2020.

Comments: 3rd Annual Learning for Dynamics & Control Conference (L4DC 2021)

Incentive Compatibility in Stochastic Dynamic Systems

Authors: Ke Ma, P. R. Kumar

Abstract: While the classic Vickrey-Clarke-Groves mechanism ensures incentive compatibility for a static one-shot game, it does not appear to be feasible to construct a dominant truth-telling mechanism for agents that are stochastic dynamic systems. The contribution of this paper is to show that for a set of LQG agents a mechanism consisting of a sequence of layered payments over time decouples the intertem… ▽ More While the classic Vickrey-Clarke-Groves mechanism ensures incentive compatibility for a static one-shot game, it does not appear to be feasible to construct a dominant truth-telling mechanism for agents that are stochastic dynamic systems. The contribution of this paper is to show that for a set of LQG agents a mechanism consisting of a sequence of layered payments over time decouples the intertemporal effect of current bids on future payoffs and ensures truth-telling of dynamic states by their agents, if system parameters are known and agents are rational. Additionally, it is shown that there is a "Scaled" VCG mechanism that simultaneously satisfies incentive compatibility, social efficiency, budget balance as well as individual rationality under a certain "market power balance" condition where no agent is too negligible or too dominant. A further desirable property is that the SVCG payments converge to the Lagrange payments, the payments corresponding to the true price in the absence of strategic considerations, as the number of agents in the market increases. For LQ but non-Gaussian agents the optimal social welfare over the class of linear control laws is achieved. △ Less

Submitted 29 April, 2020; v1 submitted 4 January, 2019; originally announced January 2019.

Comments: arXiv admin note: text overlap with arXiv:1803.06734

A Synchrophasor Data-driven Method for Forced Oscillation Localization under Resonance Conditions

Authors: Tong Huang, Nikolaos M. Freris, P. R. Kumar, Le Xie

Abstract: This paper proposes a data-driven algorithm of locating the source of forced oscillations and suggests the physical interpretation of the method. By leveraging the sparsity of the forced oscillation sources along with the low-rank nature of synchrophasor data, the problem of source localization under resonance conditions is cast as computing the sparse and low-rank components using Robust Principa… ▽ More This paper proposes a data-driven algorithm of locating the source of forced oscillations and suggests the physical interpretation of the method. By leveraging the sparsity of the forced oscillation sources along with the low-rank nature of synchrophasor data, the problem of source localization under resonance conditions is cast as computing the sparse and low-rank components using Robust Principal Component Analysis (RPCA), which can be efficiently solved by the exact Augmented Lagrange Multiplier method. Based on this problem formulation, an efficient and practically implementable algorithm is proposed to pinpoint the forced oscillation source during real-time operation. Furthermore, we provide theoretical insights into the efficacy of the proposed approach by use of physical model-based analysis, in specific by establishing the fact that the rank of the resonance component matrix is at most 2. The effectiveness of the proposed method is validated in the IEEE 68-bus power system and the WECC 179-bus benchmark system. △ Less

Submitted 26 August, 2019; v1 submitted 15 December, 2018; originally announced December 2018.

Comments: This manuscript has been submitted to IEEE Transactions on Power Systems

Towards Provably Safe Mixed Transportation Systems with Human-driven and Automated Vehicles

Authors: Xi Liu, Ke Ma, P. R. Kumar

Abstract: Currently, we are in an environment where the fraction of automated vehicles is negligibly small. We anticipate that this fraction will increase in coming decades before if ever, we have a fully automated transportation system. Motivated by this we address the problem of provable safety of mixed traffic consisting of both intelligent vehicles (IVs) as well as human-driven vehicles (HVs). An import… ▽ More Currently, we are in an environment where the fraction of automated vehicles is negligibly small. We anticipate that this fraction will increase in coming decades before if ever, we have a fully automated transportation system. Motivated by this we address the problem of provable safety of mixed traffic consisting of both intelligent vehicles (IVs) as well as human-driven vehicles (HVs). An important issue that arises is that such mixed systems may well have lesser throughput than all human traffic systems if the automated vehicles are expected to remain provably safe with respect to human traffic. This necessitates the consideration of strategies such as platooning of automated vehicles in order to increase the throughput. In this paper, we address the design of provably safe systems consisting of a mix of automated and human-driven vehicles including the use of platooning by automated vehicles. We design motion planing policies and coordination rules for participants in this novel mixed system. HVs are considered as nearsighted and modeled with relatively loose constraints, while IVs are considered as capable of following much tighter constraints. HVs are expected to follow reasonable and simple rules. IVs are designed to move under a model predictive control (MPC) based motion plans and coordination protocols. Our contribution of this paper is in showing how to integrate these two types of models safely into a mixed system. System safety is proved in single lane scenarios, as well as in multi-lane situations allowing lane changes. △ Less

Submitted 13 October, 2018; originally announced October 2018.

Safe Intersection Management for Mixed Transportation Systems with Human-Driven and Autonomous Vehicles

Authors: Xi Liu, Ping-Chun Hsieh, P. R. Kumar

Abstract: Most recent studies on establishing intersection safety focus on the situation where all vehicles are fully autonomous. However, currently most vehicles are human-driven and so we will need to transition through regimes featuring a varying proportion of human-driven vehicles ranging from 100\% to 0\% before realizing such a fully autonomous future -- if ever. We will, therefore, need to address th… ▽ More Most recent studies on establishing intersection safety focus on the situation where all vehicles are fully autonomous. However, currently most vehicles are human-driven and so we will need to transition through regimes featuring a varying proportion of human-driven vehicles ranging from 100\% to 0\% before realizing such a fully autonomous future -- if ever. We will, therefore, need to address the safety of hybrid systems featuring an arbitrary mixture of human-driven and autonomous vehicles. In fact, recent incidents involving autonomous vehicles have already highlighted the need to study the safety of autonomous vehicles co-existing with human-driven vehicles. Motivated by this we address the design of provably safe intersection management for mixed traffic consisting of a mix of human-driven vehicles (HVs) as well as autonomous vehicles (AVs). To analyze such mixed traffic, we model HVs as nearsighted and with relatively loose constraints, permitting worst-case behavior while AVs are considered as capable of following much tighter constraints. HVs are allowed freedom to change their speed at any time while AVs are only allowed to change their speed beginning of a time slot through a Model Predictive Controller (MPC). AVs are assumed to possess a shorter response time and stronger braking capability than HVs in collision avoidance. Moreover, AVs obtain the permissions of passing through the intersection through vehicle-to-infrastructure (V2I) communication, while HVs achieve the same objective by following traffic lights. Taking the above differences into consideration, we propose a provably safe intersection management for mixed traffic comprised of an MPC-based protocol for AVs along with a coordination protocol for traffic lights... △ Less

Submitted 14 October, 2018; v1 submitted 20 August, 2018; originally announced August 2018.

Comments: Extended paper published in Allerton 2018

The Strategic LQG System: A Dynamic Stochastic VCG Framework for Optimal Coordination

Authors: Ke Ma, P. R. Kumar

Abstract: The classic Vickrey-Clarke-Groves (VCG) mechanism ensures incentive compatibility, i.e., that truth-telling of all agents is a dominant strategy, for a static one-shot game. However, in a dynamic environment that unfolds over time, the agents' intertemporal payoffs depend on the expected future controls and payments, and a direct extension of the VCG mechanism is not sufficient to guarantee incent… ▽ More The classic Vickrey-Clarke-Groves (VCG) mechanism ensures incentive compatibility, i.e., that truth-telling of all agents is a dominant strategy, for a static one-shot game. However, in a dynamic environment that unfolds over time, the agents' intertemporal payoffs depend on the expected future controls and payments, and a direct extension of the VCG mechanism is not sufficient to guarantee incentive compatibility. In fact, it does not appear to be feasible to construct mechanisms that ensure the dominance of dynamic truth-telling for agents comprised of general stochastic dynamic systems. The contribution of this paper is to show that such a dynamic stochastic extension does exist for the special case of Linear-Quadratic-Gaussian (LQG) agents with a careful construction of a sequence of layered payments over time. For a set of LQG agents, we propose a modified layered version of the VCG mechanism for payments that decouples the intertemporal effect of current bids on future payoffs, and prove that truth-telling of dynamic states forms a dominant strategy if system parameters are known and agents are rational. An important example of a problem needing such optimal dynamic coordination of stochastic agents arises in power systems where an Independent System Operator (ISO) has to ensure balance of generation and consumption at all time instants, while ensuring social optimality. The challenge is to determine a bidding scheme between all agents and the ISO that maximizes social welfare, while taking into account the stochastic dynamic models of agents, since renewable energy resources such as solar/wind are stochastic and dynamic in nature, as are consumptions by loads which are influenced by factors such as local temperatures and thermal inertias of facilities. △ Less

Submitted 11 June, 2019; v1 submitted 18 March, 2018; originally announced March 2018.

On the Use of the Observability Gramian for Partially Observed Robotic Path Planning Problems

Authors: Mohammadhussein Rafieisakhaei, Suman Chakravorty, P. R. Kumar

Abstract: Optimizing measures of the observability Gramian as a surrogate for the estimation performance may provide irrelevant or misleading trajectories for planning under observation uncertainty. Optimizing measures of the observability Gramian as a surrogate for the estimation performance may provide irrelevant or misleading trajectories for planning under observation uncertainty. △ Less

Submitted 30 January, 2018; originally announced January 2018.

Comments: 6 pages, 9 figures. CDC 2017

Journal ref: 2017 IEEE 56th Annual Conference on Decision and Control (CDC)

An Online Detection Framework for Cyber Attacks on Automatic Generation Control

Authors: Tong Huang, Bharadwaj Satchidanandan, P. R. Kumar, Le Xie

Abstract: We propose an online framework to detect cyber attacks on Automatic Generation Control (AGC). A cyber at- tack detection algorithm is designed based on the approach of Dynamic Watermarking. The detection algorithm provides a theoretical guarantee of detection of cyber attacks launched by sophisticated attackers possessing extensive knowledge of the physical and statistical models of targeted power… ▽ More We propose an online framework to detect cyber attacks on Automatic Generation Control (AGC). A cyber at- tack detection algorithm is designed based on the approach of Dynamic Watermarking. The detection algorithm provides a theoretical guarantee of detection of cyber attacks launched by sophisticated attackers possessing extensive knowledge of the physical and statistical models of targeted power systems. The proposed framework is practically implementable, as it needs no hardware update on generation units. The efficacy of the proposed framework is validated in both four-area system and 140-bus system. △ Less

Submitted 20 April, 2018; v1 submitted 15 December, 2017; originally announced December 2017.

Comments: This article has been accepted for publication in a future issue of IEEE Transactions on Power Systems, but has not been fully edited. Content may change prior to final publication

Scenario-based Economic Dispatch with Uncertain Demand Response

Authors: Hao Ming, Le Xie, Marco Campi, Simone Garatti, P. R. Kumar

Abstract: This paper introduces a new computational framework to account for uncertainties in day-ahead electricity market clearing process in the presence of demand response providers. A central challenge when dealing with many demand response providers is the uncertainty of its realization. In this paper, a new economic dispatch framework that is based on the recent theoretical development of the scenario… ▽ More This paper introduces a new computational framework to account for uncertainties in day-ahead electricity market clearing process in the presence of demand response providers. A central challenge when dealing with many demand response providers is the uncertainty of its realization. In this paper, a new economic dispatch framework that is based on the recent theoretical development of the scenario approach is introduced. By removing samples from a finite uncertainty set, this approach improves dispatch performance while guaranteeing a quantifiable risk level with respect to the probability of violating the constraints. The theoretical bound on the level of risk is shown to be a function of the number of scenarios removed. This is appealing to the system operator for the following reasons: (1) the improvement of performance comes at the cost of a quantifiable level of violation probability in the constraints; (2) the violation upper bound does not depend on the probability distribution assumption of the uncertainty in demand response. Numerical simulations on (1) 3-bus and (2) IEEE 14-bus system (3) IEEE 118-bus system suggest that this approach could be a promising alternative in future electricity markets with multiple demand response providers. △ Less

Submitted 29 November, 2017; originally announced November 2017.

Throughput Optimal Decentralized Scheduling of Multi-Hop Networks with End-to-End Deadline Constraints: II Wireless Networks with Interference

Authors: Rahul Singh, P. R. Kumar, Eytan Modiano

Abstract: Consider a multihop wireless network serving multiple flows in which wireless link interference constraints are described by a link interference graph. For such a network, we design routing-scheduling policies that maximize the end-to-end timely throughput of the network. Timely throughput of a flow $f$ is defined as the average rate at which packets of flow $f$ reach their destination node $d_f$… ▽ More Consider a multihop wireless network serving multiple flows in which wireless link interference constraints are described by a link interference graph. For such a network, we design routing-scheduling policies that maximize the end-to-end timely throughput of the network. Timely throughput of a flow $f$ is defined as the average rate at which packets of flow $f$ reach their destination node $d_f$ within their deadline. Our policy has several surprising characteristics. Firstly, we show that the optimal routing-scheduling decision for an individual packet that is present at a wireless node $i\in V$ is solely a function of its location, and "age". Thus, a wireless node $i$ does not require the knowledge of the "global" network state in order to maximize the timely throughput. We notice that in comparison, under the backpressure routing policy, a node $i$ requires only the knowledge of its neighbours queue lengths in order to guarantee maximal stability, and hence is decentralized. The key difference arises due to the fact that in our set-up the packets loose their utility once their "age" has crossed their deadline, thus making the task of optimizing timely throughput much more challenging than that of ensuring network stability. Of course, due to this key difference, the decision process involved in maximizing the timely throughput is also much more complex than that involved in ensuring network-wide queue stabilization. In view of this, our results are somewhat surprising. △ Less

Submitted 11 September, 2017; v1 submitted 6 September, 2017; originally announced September 2017.

Near-Optimal Belief Space Planning via T-LQG

Authors: Mohammadhussein Rafieisakhaei, Suman Chakravorty, P. R. Kumar

Abstract: We consider the problem of planning under observation and motion uncertainty for nonlinear robotics systems. Determining the optimal solution to this problem, generally formulated as a Partially Observed Markov Decision Process (POMDP), is computationally intractable. We propose a Trajectory-optimized Linear Quadratic Gaussian (T-LQG) approach that leads to quantifiably near-optimal solutions for… ▽ More We consider the problem of planning under observation and motion uncertainty for nonlinear robotics systems. Determining the optimal solution to this problem, generally formulated as a Partially Observed Markov Decision Process (POMDP), is computationally intractable. We propose a Trajectory-optimized Linear Quadratic Gaussian (T-LQG) approach that leads to quantifiably near-optimal solutions for the POMDP problem. We provide a novel "separation principle" for the design of an optimal nominal open-loop trajectory followed by an optimal feedback control law, which provides a near-optimal feedback control policy for belief space planning problems involving a polynomial order of calculations of minimum order. △ Less

Submitted 10 July, 2017; v1 submitted 25 May, 2017; originally announced May 2017.

Comments: 3 pages, 3 figures, In Robotics: Science and Systems (RSS) 2017 Workshop of "POMDPs in Robotics: State of The Art, Challenges, and Opportunities"

A Near-Optimal Separation Principle for Nonlinear Stochastic Systems Arising in Robotic Path Planning and Control

Authors: Mohammadhussein Rafieisakhaei, Suman Chakravorty, P. R. Kumar

Abstract: We consider nonlinear stochastic systems that arise in path planning and control of mobile robots. As is typical of almost all nonlinear stochastic systems, the optimally solving problem is intractable. We provide a design approach which yields a tractable design that is quantifiably near-optimal. We exhibit a "separation" principle under a small noise assumption consisting of the optimal open-loo… ▽ More We consider nonlinear stochastic systems that arise in path planning and control of mobile robots. As is typical of almost all nonlinear stochastic systems, the optimally solving problem is intractable. We provide a design approach which yields a tractable design that is quantifiably near-optimal. We exhibit a "separation" principle under a small noise assumption consisting of the optimal open-loop design of nominal trajectory followed by an optimal feedback law to track this trajectory, which is different from the usual effort of separating estimation from control. As a corollary, we obtain a trajectory-optimized linear quadratic regulator design for stochastic nonlinear systems with Gaussian noise. △ Less

Submitted 23 May, 2017; originally announced May 2017.

Comments: 7 pages, 4 Figures, Submitted to 56th IEEE Conference on Decision and Control (CDC), 2017

Architecture and Algorithms for Privacy Preserving Thermal Inertial Load Management by A Load Serving Entity

Authors: Abhishek Halder, Xinbo Geng, P. R. Kumar, Le Xie

Abstract: Motivated by the growing importance of demand response in modern power system's operations, we propose an architecture and supporting algorithms for privacy preserving thermal inertial load management as a service provided by the load serving entity (LSE). We focus on an LSE managing a population of its customers' air conditioners, and propose a contractual model where the LSE guarantees quality o… ▽ More Motivated by the growing importance of demand response in modern power system's operations, we propose an architecture and supporting algorithms for privacy preserving thermal inertial load management as a service provided by the load serving entity (LSE). We focus on an LSE managing a population of its customers' air conditioners, and propose a contractual model where the LSE guarantees quality of service to each customer in terms of keeping their indoor temperature trajectories within respective bands around the desired individual comfort temperatures. We show how the LSE can price the contracts differentiated by the flexibility embodied by the width of the specified bands. We address architectural questions of (i) how the LSE can strategize its energy procurement based on price and ambient temperature forecasts, (ii) how an LSE can close the real time control loop at the aggregate level while providing individual comfort guarantees to loads, without ever measuring the states of an air conditioner for privacy reasons. Control algorithms to enable our proposed architecture are given, and their efficacy is demonstrated on real data. △ Less

Submitted 29 November, 2016; v1 submitted 30 June, 2016; originally announced June 2016.

Dynamic Watermarking: Active Defense of Networked Cyber-Physical Systems

Authors: Bharadwaj Satchidanandan, P. R. Kumar

Abstract: The coming decades may see the large scale deployment of networked cyber-physical systems to address global needs in areas such as energy, water, healthcare, and transportation. However, as recent events have shown, such systems are vulnerable to cyber attacks. Being safety critical, their disruption or misbehavior can cause economic losses or injuries and loss of life. It is therefore important t… ▽ More The coming decades may see the large scale deployment of networked cyber-physical systems to address global needs in areas such as energy, water, healthcare, and transportation. However, as recent events have shown, such systems are vulnerable to cyber attacks. Being safety critical, their disruption or misbehavior can cause economic losses or injuries and loss of life. It is therefore important to secure such networked cyber-physical systems against attacks. In the absence of credible security guarantees, there will be resistance to the proliferation of cyber-physical systems, which are much needed to meet global needs in critical infrastructures and services. This paper addresses the problem of secure control of networked cyber-physical systems. This problem is different from the problem of securing the communication network, since cyber-physical systems at their very essence need sensors and actuators that interface with the physical plant, and malicious agents may tamper with sensors or actuators, as recent attacks have shown. We consider physical plants that are being controlled by multiple actuators and sensors communicating over a network, where some sensors could be "malicious," meaning that they may not report the measurements that they observe. We address a general technique by which the actuators can detect the actions of malicious sensors in the system, and disable closed-loop control based on their information. This technique, called "watermarking," employs the technique of actuators injecting private excitation into the system which will reveal malicious tampering with signals. We show how such an active defense can be used to secure networked systems of sensors and actuators. △ Less

Submitted 27 June, 2016; originally announced June 2016.

Throughput Optimal Decentralized Scheduling of Multi-Hop Networks with End-to-End Deadline Constraints: Unreliable Links

Authors: Rahul Singh, P. R. Kumar

Abstract: We consider unreliable multi-hop networks serving multiple flows in which packets not delivered to their destination nodes by their deadlines are dropped. We address the design of policies for routing and scheduling packets that optimize any specified weighted average of the throughputs of the flows. We provide a new approach which directly yields an optimal distributed scheduling policy that atta… ▽ More We consider unreliable multi-hop networks serving multiple flows in which packets not delivered to their destination nodes by their deadlines are dropped. We address the design of policies for routing and scheduling packets that optimize any specified weighted average of the throughputs of the flows. We provide a new approach which directly yields an optimal distributed scheduling policy that attains any desired maximal timely-throughput vector under average-power constraints on the nodes. It pursues a novel intrinsically stochastic decomposition of the Lagrangian of the constrained network-wide MDP rather than of the fluid model. All decisions regarding a packet's transmission scheduling, transmit power level, and routing, are completely distributed, based solely on the age of the packet, not requiring any knowledge of network state or queue lengths at any of the nodes. Global coordination is achieved through a tractably computable "price" for transmission energy. This price is different from that used to derive the backpressure policy where price corresponds to queue lengths. A quantifiably near-optimal policy is provided if nodes have peak-power constraints. △ Less

Submitted 5 June, 2016; originally announced June 2016.

MSC Class: 93E03; 49N30; 49N15; 60K25 ACM Class: C.2.1

Journal ref: EEE Transactions on Automatic Control, vol. 64, no. 1, pp. 127-142, Jan. 2019

The ISO Problem: Decentralized Stochastic Control via Bidding Schemes

Authors: Rahul Singh, P. R. Kumar, Le Xie

Abstract: We consider a smart-grid connecting several agents, modeled as stochastic dynamical systems, who may be electricity consumers/producers. At each discrete time instant, which may represent a 15 minute interval, each agent may consume/generate some quantity of electrical energy. The Independent System Operator (ISO) is given the task of assigning consumptions/generations to the agents so as to maxim… ▽ More We consider a smart-grid connecting several agents, modeled as stochastic dynamical systems, who may be electricity consumers/producers. At each discrete time instant, which may represent a 15 minute interval, each agent may consume/generate some quantity of electrical energy. The Independent System Operator (ISO) is given the task of assigning consumptions/generations to the agents so as to maximize the sum of the utilities accrued to the agents, subject to the constraint that energy generation equals consumption at each time. This task of coordinating generation and demand has to be accomplished by the ISO without the agents revealing their system states, dynamics, or utility/cost functions. We show how and when a simple iterative procedure converges to the optimal solution. The ISO iteratively obtains electricity bids by the agents, and declares the tentative market clearing prices. In response to these prices, the agents submit new bids. On the demand side, the solution yields an optimal demand response for dynamic and stochastic loads. On the generation side, it provides the optimal utilization of stochastically varying renewables such as solar/wind, and generation with fossil fuel based generation with dynamic constraints such as ramping rates. Thereby we solve a decentralized stochastic control problem, without agents sharing any information about their system models, states or utility functions. △ Less

Submitted 4 October, 2015; originally announced October 2015.

Journal ref: 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2015, pp. 552-559

A Theory for the Operation of the Independent System Operator in a Smart Grid with Stochastic Renewables, Demand Response and Storage

Authors: Rahul Singh, Ke Ma, Anupam Thatte, P. R. Kumar, Le Xie

Abstract: In this paper, we address a key issue of designing architectures and algorithms which generate optimal demand response in a decentralized manner for a smart-grid consisting of several stochastic renewables and dynamic loads. By optimal demand response, we refer to the demand response which maximizes the utility of the agents connected to the smart-grid. By decentralized we refer to the desirable c… ▽ More In this paper, we address a key issue of designing architectures and algorithms which generate optimal demand response in a decentralized manner for a smart-grid consisting of several stochastic renewables and dynamic loads. By optimal demand response, we refer to the demand response which maximizes the utility of the agents connected to the smart-grid. By decentralized we refer to the desirable case where neither the independent system operator (ISO) needs to know the dynamics/utilities of the agents, nor do the agents need to have a knowledge of the dynamics/utilities of other agents connected to the grid. The communication between the ISO and agents is restricted to the ISO announcing a pricing policy and the agents responding with their energy generation/consumption bids in response to the pricing policy. We provide a complete solution for both the deterministic and stochastic cases. It features a price iteration scheme that results in optimality of social welfare. We also provide an optimal solution for the case where there is a common randomness affecting and observed by all agents. This solution can be computationally complex, and we pose approximations. For the more general partially observed randomness case, we exhibit a relaxation that significantly reduces complexity. We also provide an approximation strategy that leads to a model predictive control (MPC) approach. Simulation results comparing the resulting optimal demand response with the existing architectures employed by the ISO illustrate the benefit in social welfare utility realized by our scheme. To the best of the authors' knowledge, this is the first work of its kind to explicitly mark out the optimal response of dynamic demand. △ Less

Submitted 16 July, 2015; originally announced July 2015.

Journal ref: 54th IEEE Conference on Decision and Control (CDC), 2015, pp. 3778-3785

Optimal Energy-Efficient Regular Delivery of Packets in Cyber-Physical Systems

Authors: Xueying Guo, Rahul Singh, P. R. Kumar, Zhisheng Niu

Abstract: In cyber-physical systems such as in-vehicle wireless sensor networks, a large number of sensor nodes continually generate measurements that should be received by other nodes such as actuators in a regular fashion. Meanwhile, energy-efficiency is also important in wireless sensor networks. Motivated by these, we develop scheduling policies which are energy efficient and simultaneously maintain "re… ▽ More In cyber-physical systems such as in-vehicle wireless sensor networks, a large number of sensor nodes continually generate measurements that should be received by other nodes such as actuators in a regular fashion. Meanwhile, energy-efficiency is also important in wireless sensor networks. Motivated by these, we develop scheduling policies which are energy efficient and simultaneously maintain "regular" deliveries of packets. A tradeoff parameter is introduced to balance these two conflicting objectives. We employ a Markov Decision Process (MDP) model where the state of each client is the time-since-last-delivery of its packet, and reduce it into an equivalent finite-state MDP problem. Although this equivalent problem can be solved by standard dynamic programming techniques, it suffers from a high-computational complexity. Thus we further pose the problem as a restless multi-armed bandit problem and employ the low-complexity Whittle Index policy. It is shown that this problem is indexable and the Whittle indexes are derived. Also, we prove the Whittle Index policy is asymptotically optimal and validate its optimality via extensive simulations. △ Less

Submitted 26 February, 2015; originally announced February 2015.

Comments: 6 pages conference

Bounded epsilon-Reach Set Computation of a Class of Deterministic and Transversal Linear Hybrid Automata

Authors: Kyoung-Dae Kim, Sayan Mitra, P. R. Kumar

Abstract: We define a special class of hybrid automata, called Deterministic and Transversal Linear Hybrid Automata (DTLHA), whose continuous dynamics in each location are linear time-invariant (LTI) with a constant input, and for which every discrete transition up to a given bounded time is deterministic and, importantly, transversal. For such a DTLHA starting from an initial state, we show that it is poss… ▽ More We define a special class of hybrid automata, called Deterministic and Transversal Linear Hybrid Automata (DTLHA), whose continuous dynamics in each location are linear time-invariant (LTI) with a constant input, and for which every discrete transition up to a given bounded time is deterministic and, importantly, transversal. For such a DTLHA starting from an initial state, we show that it is possible to compute an approximation of the reach set of a DTLHA over a finite time interval that is arbitrarily close to the exact reach set, called a bounded epsilon-reach set, through sampling and polyhedral over-approximation of sampled states. We propose an algorithm and an attendant architecture for the overall bounded epsilon-reach set computation process. △ Less

Submitted 15 May, 2012; originally announced May 2012.