-
Stable state and signal estimation in a network context
Authors:
Robert R. Bitmead
Abstract:
Power grid, communications, computer and product reticulation networks are frequently layered or subdivided by design. The layering divides responsibilities and can be driven by operational, commercial, regulatory and privacy concerns. From a control context, a layer, or part of a layer, in a network isolates the authority to manage, i.e. control, a dynamic system with connections into unknown par…
▽ More
Power grid, communications, computer and product reticulation networks are frequently layered or subdivided by design. The layering divides responsibilities and can be driven by operational, commercial, regulatory and privacy concerns. From a control context, a layer, or part of a layer, in a network isolates the authority to manage, i.e. control, a dynamic system with connections into unknown parts of the network. The topology of these connections is fully prescribed but the interconnecting signals, currents in the case of power grids and bandwidths in communications, are largely unavailable, through lack of sensing and even prohibition. Accordingly, one is driven to simultaneous input and state estimation methods. We study a class of algorithms for this joint task, which has the unfortunate issue of inverting a subsystem, which if it has unstable transmission zeros leads to an unstable and unimplementable estimator. Two modifications to the algorithm to ameliorate this problem were recently proposed involving replacing the troublesome subsystem with its outer factor from its inner-outer factorization or using a high-variance white signal model for the unknown inputs. Here, we establish the connections between the original estimation problem for state and input signal and the estimates from the algorithm applied solely to the outer factor. It is demonstrated that the state of the outer factor and that of the original system asymptotically coincide and that the estimate of the input signal to the outer factor has asymptotically stationary second-order statistics which are in one-to-one correspondence with those of the input signal to the original system, when this signal is itself stationary. Thus, the simultaneous input and state estimation algorithm applied just to the outer factor yields an unbiased state estimate for control and the statistics of the interface signals.
△ Less
Submitted 1 September, 2023; v1 submitted 8 August, 2023;
originally announced August 2023.
-
State estimation for control: an approach for output-feedback stochastic MPC
Authors:
Mohammad S. Ramadan,
Robert R. Bitmead,
Ke Huang
Abstract:
The paper provides a new approach to the determination of a single state value for stochastic output feedback problems using paradigms from Model Predictive Control, particularly the distinction between open-loop and closed-loop control and between deterministic optimal control and stochastic optimal control. The State Selection Algorithm is presented and relies on given dynamics and constraints,…
▽ More
The paper provides a new approach to the determination of a single state value for stochastic output feedback problems using paradigms from Model Predictive Control, particularly the distinction between open-loop and closed-loop control and between deterministic optimal control and stochastic optimal control. The State Selection Algorithm is presented and relies on given dynamics and constraints, a nominal deterministic state-feedback controller, and a sampling based method to select the best state value, based on optimizing a prescribed finite-horizon performance function, over the available candidates provided by a particle filter. The cost function is minimized over the horizon with controls determined by the nominal controller and the selected states. So, the minimization is performed not over the selection of the control other than through the choice of state value to use. The algorithm applies generally to nonlinear stochastic systems and relies on Monte Carlo sampling and averaging. However, in linear quadratic polyhedrally constrained cases the technique reduces to a quadratic program for the state value. The algorithm is evaluated in a set of computational examples, which illustrate its efficacy and limitations. Numerical aspects and the opportunity for parallelization are discussed. The examples demonstrate the algorithm operating, in closed-loop with its attendant particle filter, over the long horizon.
△ Less
Submitted 1 March, 2023;
originally announced March 2023.
-
Control-Oriented Modeling of Pipe Flow through Intersecting Pipe Geometries
Authors:
Sven Brüggemann,
Robert R. Bitmead
Abstract:
We present control-oriented models for transient dynamics of isothermal one-dimensional gas flow through multiple pipes in series and intersecting pipe geometries. These composite models subsume algebraic constraints that would otherwise appear due to boundary conditions, so that our linear state-space models are well-suited for model-based control design for gas flow in pipe networks with non-tri…
▽ More
We present control-oriented models for transient dynamics of isothermal one-dimensional gas flow through multiple pipes in series and intersecting pipe geometries. These composite models subsume algebraic constraints that would otherwise appear due to boundary conditions, so that our linear state-space models are well-suited for model-based control design for gas flow in pipe networks with non-trivial geometries.
△ Less
Submitted 20 May, 2022; v1 submitted 13 December, 2021;
originally announced December 2021.
-
Simultaneous input & state estimation, singular filtering and stability
Authors:
Mohammad Ali Abooshahab,
Mohammed M. J. Alyaseen,
Robert R. Bitmead,
Morten Hovd
Abstract:
Input estimation is a signal processing technique associated with deconvolution of measured signals after filtering through a known dynamic system. Kitanidis and others extended this to the simultaneous estimation of the input signal and the state of the intervening system. This is normally posed as a special least-squares estimation problem with unbiasedness. The approach has application in signa…
▽ More
Input estimation is a signal processing technique associated with deconvolution of measured signals after filtering through a known dynamic system. Kitanidis and others extended this to the simultaneous estimation of the input signal and the state of the intervening system. This is normally posed as a special least-squares estimation problem with unbiasedness. The approach has application in signal analysis and in control. Despite the connection to optimal estimation, the standard algorithms are not necessarily stable, leading to a number of recent papers which present sufficient conditions for stability. In this paper we complete these stability results in two ways in the time-invariant case: for the square case, where the number of measurements equals the number of unknown inputs, we establish exactly the location of the algorithm poles; for the non-square case, we show that the best sufficient conditions are also necessary. We then draw on our previous results interpreting these algorithms, when stable, as singular Kalman filters to advocate a direct, guaranteed stable implementation via Kalman filtering. This has the advantage of clarity and flexibility in addition to stability. En route, we decipher the existing algorithms in terms of system inversion and successive singular filtering. The stability results are extended to the time-varying case directly to recover the earlier sufficient conditions for stability via the Riccati difference equation.
△ Less
Submitted 20 August, 2020;
originally announced August 2020.
-
LQG Control Performance with Low Bitrate Periodic Coding
Authors:
Behrooz Amini,
Robert R. Bitmead
Abstract:
Specific low-bitrate coding strategies are examined through their effect on LQ control performance. By limiting the subject to these methods, we are able to identify principles underlying coding for control; a subject of significant recent interest but few tangible results. In particular, we consider coding the quantized output signal deploying period-two codes of differing delay-versus-accuracy t…
▽ More
Specific low-bitrate coding strategies are examined through their effect on LQ control performance. By limiting the subject to these methods, we are able to identify principles underlying coding for control; a subject of significant recent interest but few tangible results. In particular, we consider coding the quantized output signal deploying period-two codes of differing delay-versus-accuracy tradeoff. The quantification of coding performance is via the LQ control cost. The feedback control system comprises the coder-decoder in the path between the output and the state estimator, which is followed by linear state-variable feedback, as is optimal in the memoryless case. The quantizer is treated as the functional composition of an infinitely-long linear staircase function and a saturation. This permits the analysis to subdivide into estimator computations, seemingly independent of the control performance criterion, and an escape time evaluation, which ties the control back into the choice of quantizer saturation bound. An example is studied which illustrates the role of the control objective in determining the efficacy of coding using these schemes. The results mesh well with those observed in signal coding. However, the introduction of a realization-based escape time is a novelty departing significantly from mean square computations.
△ Less
Submitted 7 April, 2020;
originally announced April 2020.
-
Exponential convergence of recursive least squares with forgetting factor for multiple-output systems
Authors:
Sven Brüggemann,
Robert R. Bitmead
Abstract:
We extend results of the recursive-least-squares-with-forgetting-factor identifier for single-input-single-output systems to the multiple-output case by, under the assumption of persistence of excitation, deriving the corresponding minimized objective function and by showing exponential convergence of the estimation error.
We extend results of the recursive-least-squares-with-forgetting-factor identifier for single-input-single-output systems to the multiple-output case by, under the assumption of persistence of excitation, deriving the corresponding minimized objective function and by showing exponential convergence of the estimation error.
△ Less
Submitted 16 March, 2020;
originally announced March 2020.
-
Stochastic Model Predictive Control: Output-Feedback, Duality and Guaranteed Performance
Authors:
Martin A Sehr,
Robert R Bitmead
Abstract:
A new formulation of Stochastic Model Predictive Output Feedback Control is presented and analyzed as a translation of Stochastic Optimal Output Feedback Control into a receding horizon setting. This requires lifting the design into a framework involving propagation of the conditional state density, the information state, via the Bayesian Filter and solution of the Stochastic Dynamic Programming E…
▽ More
A new formulation of Stochastic Model Predictive Output Feedback Control is presented and analyzed as a translation of Stochastic Optimal Output Feedback Control into a receding horizon setting. This requires lifting the design into a framework involving propagation of the conditional state density, the information state, via the Bayesian Filter and solution of the Stochastic Dynamic Programming Equation for an optimal feedback policy, both stages of which are computationally challenging in the general, nonlinear setup. The upside is that the clearance of three bottleneck aspects of Model Predictive Control is connate to the optimality: output feedback is incorporated naturally; dual regulation and probing of the control signal is inherent; closed-loop performance relative to infinite-horizon optimal control is guaranteed. While the methods are numerically formidable, our aim is to develop an approach to Stochastic Model Predictive Control with guarantees and, from there, to seek a less onerous approximation. To this end, we discuss in particular the class of Partially Observable Markov Decision Processes, to which our results extend seamlessly, and demonstrate applicability with an example in healthcare decision making, where duality and associated optimality in the control signal are required for satisfactory closed-loop behavior.
△ Less
Submitted 2 June, 2017;
originally announced June 2017.
-
Performance of Model Predictive Control of POMDPs
Authors:
Martin A. Sehr,
Robert R. Bitmead
Abstract:
We revisit closed-loop performance guarantees for Model Predictive Control in the deterministic and stochastic cases, which extend to novel performance results applicable to receding horizon control of Partially Observable Markov Decision Processes. While performance guarantees similar to those achievable in deterministic Model Predictive Control can be obtained even in the stochastic case, the pr…
▽ More
We revisit closed-loop performance guarantees for Model Predictive Control in the deterministic and stochastic cases, which extend to novel performance results applicable to receding horizon control of Partially Observable Markov Decision Processes. While performance guarantees similar to those achievable in deterministic Model Predictive Control can be obtained even in the stochastic case, the presumed stochastic optimal control law is intractable to obtain in practice. However, this intractability relaxes for a particular instance of stochastic systems, namely Partially Observable Markov Decision Processes, provided reasonable problem dimensions are taken. This motivates extending available performance guarantees to this particular class of systems, which may also be used to approximate general nonlinear dynamics via gridding of state, observation, and control spaces. We demonstrate applicability of the novel closed-loop performance results on a particular example in healthcare decision making, which relies explicitly on the duality of the control decisions associated with Stochastic Optimal Control in weighing appropriate appointment times, diagnostic tests, and medical intervention for treatment of a disease modeled by a Markov Chain.
△ Less
Submitted 25 April, 2017;
originally announced April 2017.
-
Tractable Dual Optimal Stochastic Model Predictive Control: An Example in Healthcare
Authors:
Martin A. Sehr,
Robert R. Bitmead
Abstract:
Output-Feedback Stochastic Model Predictive Control based on Stochastic Optimal Control for nonlinear systems is computationally intractable because of the need to solve a Finite Horizon Stochastic Optimal Control Problem. However, solving this problem leads to an optimal probing nature of the resulting control law, called dual control, which trades off benefits of exploration and exploitation. In…
▽ More
Output-Feedback Stochastic Model Predictive Control based on Stochastic Optimal Control for nonlinear systems is computationally intractable because of the need to solve a Finite Horizon Stochastic Optimal Control Problem. However, solving this problem leads to an optimal probing nature of the resulting control law, called dual control, which trades off benefits of exploration and exploitation. In practice, intractability of Stochastic Model Predictive Control is typically overcome by replacement of the underlying Stochastic Optimal Control problem by more amenable approximate surrogate problems, which however come at a loss of the optimal probing nature of the control signals. While probing can be superimposed in some approaches, this is done sub-optimally. In this paper, we examine approximation of the system dynamics by a Partially Observable Markov Decision Process with its own Finite Horizon Stochastic Optimal Control Problem, which can be solved for an optimal control policy, implemented in receding horizon fashion. This procedure enables maintaining probing in the control actions. We further discuss a numerical example in healthcare decision making, highlighting the duality in stochastic optimal receding horizon control.
△ Less
Submitted 25 April, 2017;
originally announced April 2017.
-
Particle Model Predictive Control: Tractable Stochastic Nonlinear Output-Feedback MPC
Authors:
Martin A. Sehr,
Robert R. Bitmead
Abstract:
We combine conditional state density construction with an extension of the Scenario Approach for stochastic Model Predictive Control to nonlinear systems to yield a novel particle-based formulation of stochastic nonlinear output-feedback Model Predictive Control. Conditional densities given noisy measurement data are propagated via the Particle Filter as an approximate implementation of the Bayesi…
▽ More
We combine conditional state density construction with an extension of the Scenario Approach for stochastic Model Predictive Control to nonlinear systems to yield a novel particle-based formulation of stochastic nonlinear output-feedback Model Predictive Control. Conditional densities given noisy measurement data are propagated via the Particle Filter as an approximate implementation of the Bayesian Filter. This enables a particle-based representation of the conditional state density, or information state, which naturally merges with scenario generation from the current system state. This approach attempts to address the computational tractability questions of general nonlinear stochastic optimal control. The Particle Filter and the Scenario Approach are shown to be fully compatible and -- based on the time- and measurement-update stages of the Particle Filter -- incorporated into the optimization over future control sequences. A numerical example is presented and examined for the dependence of solution and computational burden on the sampling configurations of the densities, scenario generation and the optimization horizon.
△ Less
Submitted 25 April, 2017; v1 submitted 1 December, 2016;
originally announced December 2016.
-
Sequential Detection with Mutual Information Stopping Cost
Authors:
Vikram Krishnamurthy,
Robert Bitmead,
Michel Gevers,
Erik Miehling
Abstract:
This paper formulates and solves a sequential detection problem that involves the mutual information (stochastic observability) of a Gaussian process observed in noise with missing measurements. The main result is that the optimal decision is characterized by a monotone policy on the partially ordered set of positive definite covariance matrices. This monotone structure implies that numerically ef…
▽ More
This paper formulates and solves a sequential detection problem that involves the mutual information (stochastic observability) of a Gaussian process observed in noise with missing measurements. The main result is that the optimal decision is characterized by a monotone policy on the partially ordered set of positive definite covariance matrices. This monotone structure implies that numerically efficient algorithms can be designed to estimate and implement monotone parametrized decision policies.The sequential detection problem is motivated by applications in radar scheduling where the aim is to maintain the mutual information of all targets within a specified bound. We illustrate the problem formulation and performance of monotone parametrized policies via numerical examples in fly-by and persistent-surveillance applications involving a GMTI (Ground Moving Target Indicator) radar.
△ Less
Submitted 19 October, 2011;
originally announced October 2011.
