-
Finite Sample Analysis of Subspace Identification Methods
Authors:
Jiabao He,
Ingvar Ziemann,
Cristian R. Rojas,
S. Joe Qin,
Håkan Hjalmarsson
Abstract:
As one of the mainstream approaches in system identification, subspace identification methods (SIMs) are known for their simple parameterization for MIMO systems and robust numerical properties. However, a comprehensive statistical analysis of SIMs remains an open problem. Amid renewed focus on identifying state-space models in the non-asymptotic regime, this work presents a finite sample analysis…
▽ More
As one of the mainstream approaches in system identification, subspace identification methods (SIMs) are known for their simple parameterization for MIMO systems and robust numerical properties. However, a comprehensive statistical analysis of SIMs remains an open problem. Amid renewed focus on identifying state-space models in the non-asymptotic regime, this work presents a finite sample analysis for a large class of open-loop SIMs. It establishes high-probability upper bounds for system matrices obtained via SIMs, and reveals that convergence rates for estimating Markov parameters and system matrices are $\mathcal{O}(1/\sqrt{N})$ up to logarithmic terms, in line with classical asymptotic results. Following the key steps of SIMs, we arrive at the above results by a three-step procedure. In Step 1, we begin with a parsimonious SIM (PARSIM) that uses least-squares regression to estimate multiple high-order ARX models in parallel. Leveraging a recent analysis of an individual ARX model, we obtain a union error bound for a bank of ARX models. Step 2 involves model reduction via weighted singular value decomposition (SVD), where we consider different data-dependent weighting matrices and use robustness results for SVD to obtain error bounds on extended controllability and observability matrices, respectively. The final Step 3 focuses on deriving error bounds for system matrices, where two different realization algorithms, the MOESP type and the Larimore type, are considered. Although our study initially focuses on PARSIM, the methodologies apply broadly across many variants of SIMs.
△ Less
Submitted 17 January, 2025;
originally announced January 2025.
-
Probabilistic Reduced-Dimensional Vector Autoregressive Modeling with Oblique Projections
Authors:
Yanfang Mo,
S. Joe Qin
Abstract:
In this paper, we propose a probabilistic reduced-dimensional vector autoregressive (PredVAR) model to extract low-dimensional dynamics from high-dimensional noisy data. The model utilizes an oblique projection to partition the measurement space into a subspace that accommodates the reduced-dimensional dynamics and a complementary static subspace. An optimal oblique decomposition is derived for th…
▽ More
In this paper, we propose a probabilistic reduced-dimensional vector autoregressive (PredVAR) model to extract low-dimensional dynamics from high-dimensional noisy data. The model utilizes an oblique projection to partition the measurement space into a subspace that accommodates the reduced-dimensional dynamics and a complementary static subspace. An optimal oblique decomposition is derived for the best predictability regarding prediction error covariance. Building on this, we develop an iterative PredVAR algorithm using maximum likelihood and the expectation-maximization (EM) framework. This algorithm alternately updates the estimates of the latent dynamics and optimal oblique projection, yielding dynamic latent variables with rank-ordered predictability and an explicit latent VAR model that is consistent with the outer projection model. The superior performance and efficiency of the proposed approach are demonstrated using data sets from a synthesized Lorenz system and an industrial process from Eastman Chemical.
△ Less
Submitted 14 January, 2024;
originally announced January 2024.
-
Latent Dynamic Networked System Identification with High-Dimensional Networked Data
Authors:
Jiaxin Yu,
Yanfang Mo,
S. Joe Qin
Abstract:
Networked dynamic systems are ubiquitous in various domains, such as industrial processes, social networks, and biological systems. These systems produce high-dimensional data that reflect the complex interactions among the network nodes with rich sensor measurements. In this paper, we propose a novel algorithm for latent dynamic networked system identification that leverages the network structure…
▽ More
Networked dynamic systems are ubiquitous in various domains, such as industrial processes, social networks, and biological systems. These systems produce high-dimensional data that reflect the complex interactions among the network nodes with rich sensor measurements. In this paper, we propose a novel algorithm for latent dynamic networked system identification that leverages the network structure and performs dimension reduction for each node via dynamic latent variables (DLVs). The algorithm assumes that the DLVs of each node have an auto-regressive model with exogenous input and interactions from other nodes. The DLVs of each node are extracted to capture the most predictable latent variables in the high dimensional data, while the residual factors are not predictable. The advantage of the proposed framework is demonstrated on an industrial process network for system identification and dynamic data analytics.
△ Less
Submitted 29 September, 2023;
originally announced September 2023.
-
Probabilistic Reduced-Dimensional Vector Autoregressive Modeling for Dynamics Prediction and Reconstruction with Oblique Projections
Authors:
Yanfang Mo,
Jiaxin Yu,
S. Joe Qin
Abstract:
In this paper, we propose a probabilistic reduced-dimensional vector autoregressive (PredVAR) model with oblique projections. This model partitions the measurement space into a dynamic subspace and a static subspace that do not need to be orthogonal. The partition allows us to apply an oblique projection to extract dynamic latent variables (DLVs) from high-dimensional data with maximized predictab…
▽ More
In this paper, we propose a probabilistic reduced-dimensional vector autoregressive (PredVAR) model with oblique projections. This model partitions the measurement space into a dynamic subspace and a static subspace that do not need to be orthogonal. The partition allows us to apply an oblique projection to extract dynamic latent variables (DLVs) from high-dimensional data with maximized predictability. We develop an alternating iterative PredVAR algorithm that exploits the interaction between updating the latent VAR dynamics and estimating the oblique projection, using expectation maximization (EM) and a statistical constraint. In addition, the noise covariance matrices are estimated as a natural outcome of the EM method. A simulation case study of the nonlinear Lorenz oscillation system illustrates the advantages of the proposed approach over two alternatives.
△ Less
Submitted 3 September, 2023;
originally announced September 2023.
-
Optimal Online Algorithms for Peak-Demand Reduction Maximization with Energy Storage
Authors:
Yanfang Mo,
Qiulin Lin,
Minghua Chen,
Si-Zhao Joe Qin
Abstract:
The high proportions of demand charges in electric bills motivate large-power customers to leverage energy storage for reducing the peak procurement from the outer grid. Given limited energy storage, we expect to maximize the peak-demand reduction in an online fashion, challenged by the highly uncertain demands and renewable injections, the non-cumulative nature of peak consumption, and the coupli…
▽ More
The high proportions of demand charges in electric bills motivate large-power customers to leverage energy storage for reducing the peak procurement from the outer grid. Given limited energy storage, we expect to maximize the peak-demand reduction in an online fashion, challenged by the highly uncertain demands and renewable injections, the non-cumulative nature of peak consumption, and the coupling of online decisions. In this paper, we propose an optimal online algorithm that achieves the best competitive ratio, following the idea of maintaining a constant ratio between the online and the optimal offline peak-reduction performance. We further show that the optimal competitive ratio can be computed by solving a linear number of linear-fractional programs. Moreover, we extend the algorithm to adaptively maintain the best competitive ratio given the revealed inputs and actions at each decision-making round. The adaptive algorithm retains the optimal worst-case guarantee and attains improved average-case performance. We evaluate our proposed algorithms using real-world traces and show that they obtain up to 81% peak reduction of the optimal offline benchmark. Additionally, the adaptive algorithm achieves at least 20% more peak reduction against baseline alternatives.
△ Less
Submitted 10 May, 2021;
originally announced May 2021.
-
On the Convergence of the Dynamic Inner PCA Algorithm
Authors:
Sungho Shin,
Alex D. Smith,
S. Joe Qin,
Victor M. Zavala
Abstract:
Dynamic inner principal component analysis (DiPCA) is a powerful method for the analysis of time-dependent multivariate data. DiPCA extracts dynamic latent variables that capture the most dominant temporal trends by solving a large-scale, dense, and nonconvex nonlinear program (NLP). A scalable decomposition algorithm has been recently proposed in the literature to solve these challenging NLPs. Th…
▽ More
Dynamic inner principal component analysis (DiPCA) is a powerful method for the analysis of time-dependent multivariate data. DiPCA extracts dynamic latent variables that capture the most dominant temporal trends by solving a large-scale, dense, and nonconvex nonlinear program (NLP). A scalable decomposition algorithm has been recently proposed in the literature to solve these challenging NLPs. The decomposition algorithm performs well in practice but its convergence properties are not well understood. In this work, we show that this algorithm is a specialized variant of a coordinate maximization algorithm. This observation allows us to explain why the decomposition algorithm might work (or not) in practice and can guide improvements. We compare the performance of the decomposition strategies with that of the off-the-shelf solver Ipopt. The results show that decomposition is more scalable and, surprisingly, delivers higher quality solutions.
△ Less
Submitted 12 March, 2020;
originally announced March 2020.
