Auxiliary model based recursive generalized least squares identification algorithm for multivariate output-error autoregressive systems using the decomposition technique

This paper focuses on the parameter estimation problem of multivariate output-error autoregressive systems. Based on the decomposition technique and the auxiliary model identification idea, we derive a decomposition based auxiliary model recursive generalized least squares algorithm. The key is to divide the system into two fictitious subsystems, the one including a parameter vector and the other including a parameter matrix, and to estimate the two subsystems using the recursive least squares method, respectively. Compared with the auxiliary model based recursive generalized least squares algorithm, the proposed algorithm has less computational burden. Finally, an illustrative example is provided to verify the effectiveness of the proposed algorithms.     

Particle filtering based parameter estimation for systems with output-error type model structures

The output-error model structure is often used in practice and its identification is important for analysis of output-error type systems. This paper considers the parameter identification of linear and nonlinear output-error models. A particle filter which approximates the posterior probability density function with a weighted set of discrete random sampling points is utilized to estimate the unmeasurable true process outputs. To improve the convergence rate of the proposed algorithm, the scalar innovations are grouped into an innovation vector, thus more past information can be utilized. The convergence analysis shows that the parameter estimates can converge to their true values. Finally, both linear and nonlinear results are verified by numerical simulation and engineering.     

Maximum likelihood recursive least squares estimation for multivariate equation-error ARMA systems

This paper focuses on the parameter estimation problems of multivariate equation-error systems. A recursive generalized extended least squares algorithm is presented as a comparison. Based on the maximum likelihood principle and the coupling identification concept, the multivariate equation-error system is decomposed into several regressive identification models, each of which has only a parameter vector, and a coupled subsystem maximum likelihood recursive least squares identification algorithm is developed for estimating the parameter vectors of these submodels. The simulation example shows that the proposed algorithm is effective and has high estimation accuracy.     

Hierarchical gradient- and least squares-based iterative algorithms for input nonlinear output-error systems using the key term separation

This paper considers the parameter identification problems of the input nonlinear output-error (IN-OE) systems, that is the Hammerstein output-error systems. In order to overcome the excessive calculation amount of the over-parameterization method of the IN-OE systems. Through applying the hierarchial identification principle and decomposing the IN-OE system into three subsystems with a smaller number of parameters, we present the key term separation auxiliary model hierarchical gradient-based iterative algorithm and the key term separation auxiliary model hierarchical least squares-based iterative algorithm, which are called the key term separation auxiliary model three-stage gradient-based iterative algorithm and the key term separation auxiliary model three-stage least squares-based iterative algorithm. The comparison of the calculation amount and the simulation analysis indicate that the proposed algorithms are effective.     

Iterative identification for multiple-input systems with time-delays based on greedy pursuit and auxiliary model

This paper focuses on the identification of multiple-input single-output output-error systems with unknown time-delays. Since the time-delays are unknown, an identification model with a high dimensional and sparse parameter vector is derived based on overparameterization. Traditional identification methods cannot get sparse solutions and require a large number of observations unless the time-delays are predetermined. Inspired by the sparse optimization and the greedy algorithms, an auxiliary model based orthogonal matching pursuit iterative (AM-OMPI) algorithm is proposed by using the orthogonal matching pursuit, and then based on the gradient search, an auxiliary model based gradient pursuit iterative algorithm is proposed, which is computationally more efficient than the AM-OMPI algorithm. The proposed methods can simultaneously estimate the parameters and time-delays from a small number of sampled data. A simulation example is used to illustrate the effectiveness of the proposed algorithms.     

Maximum likelihood interval-varying recursive least squares identification for output-error autoregressive systems with scarce measurements

The identification problem of output-error autoregressive (OEAR) systems with scarce measurements is considered in this paper. In order to overcome the massive absence of outputs, an interval-varying recursive identification algorithm is proposed through changing the sampling interval and skipping the missing outputs. Based on the maximum likelihood principle, a maximum likelihood interval-varying recursive least squares algorithm is proposed. The effectiveness of the proposed algorithm is tested by a numerical simulation example, and an application example about the heading motion control of underwater vehicle.     

Kalman filtering based gradient estimation algorithms for observer canonical state-space systems with moving average noises

This paper focuses on the joint parameter and state estimation issue for observer canonical state-space systems with white noises in state equations and moving average noises in output equations. By means of the Kalman filtering and the gradient search, we derive a Kalman filtering based extended stochastic gradient algorithm. For purpose of achieving the higher parameter estimation accuracy, a Kalman filtering based multi-innovation extended stochastic gradient algorithm is proposed on the basis of the multi-innovation identification theory. Finally, the effectiveness of the proposed algorithms is validated through a numerical example.     

Combined state and parameter estimation for a bilinear state space system with moving average noise

This paper considers the identification problem of bilinear systems with measurement noise in the form of the moving average model. In particular, we present an interactive estimation algorithm for unmeasurable states and parameters based on the hierarchical identification principle. For unknown states, we formulate a novel bilinear state observer from input-output measurements using the Kalman filter. Then a bilinear state observer based multi-innovation extended stochastic gradient (BSO-MI-ESG) algorithm is proposed to estimate the unknown system parameters. A linear filter is utilized to improve the parameter estimation accuracy and a filtering based BSO-MI-ESG algorithm is presented using the data filtering technique. In the numerical example, we illustrate the effectiveness of the proposed identification methods.     

Robust identification of Wiener time-delay system with expectation-maximization algorithm

This paper considers the parameter estimation for Wiener time-delay systems with the output data contaminated with outliers. The time-delay and corrupted output data bring great challenges to the parameter estimation problem. The statistical model of the estimation problem is constructed based on the Laplace distribution and the identification problem is formulated in the scheme of the expectation-maximization (EM) algorithm. The negative effect of outliers imposed on the parameter estimation problem is sufficiently suppressed and the unknown time-delay and model parameters can be estimated simultaneously. The simulation example is given to demonstrate the effectiveness of the proposed algorithm.     

Distributed dynamic state estimation with parameter identification for large-scale systems

In this paper, we consider a distributed dynamic state estimation problem for time-varying systems. Based on the distributed maximum a posteriori (MAP) estimation algorithm proposed in our previous study, which studies the linear measurement models of each subsystem, and by weakening the constraint condition as that each time-varying subsystem is observable, this paper proves that the error covariances of state estimation and prediction obtained from the improved algorithm are respectively positive definite and have upper bounds, which verifies the feasibility of this algorithm. We also use new weighting functions and time-varying exponential smoothing method to ensure the robustness and improve the forecast accuracy of the distributed state estimation method. At last, an example is used to demonstrate the effectiveness of the proposed algorithm together with the parameter identification.     

Gradient-based iterative identification method for multivariate equation-error autoregressive moving average systems using the decomposition technique

This paper studies the parameter estimation problems of multivariate equation-error autoregressive moving average systems. Firstly, a gradient-based iterative algorithm is presented as a comparison. In order to improve the computational efficiency and the parameter estimation accuracy, a decomposition-based gradient iterative algorithm is presented by using the decomposition technique. The key is to transform an original system into two subsystems and to estimate the parameters of each subsystem, respectively. Compared with the gradient-based iterative algorithm, the decomposition-based algorithm requires less computational efforts, and the simulation results indicate that this algorithm is effective.     

Data filtering-based multi-innovation forgetting gradient algorithms for input nonlinear FIR-MA systems with piecewise-linear characteristics

The piecewise-linear characteristics often appear in the nonlinear systems that operate in different ways in different input regions. This paper studies the identification issue of a class of block-oriented systems with piecewise-linear characteristics. The asymmetric piecewise-linear nonlinearity is expressed as a linear parametric representation through introducing an appropriate switching function, then the identification model of the system is derived by using the key term separation technique. On this model basis, a multi-innovation forgetting gradient algorithm is presented to estimate the unknown parameters. To further enhance the identification accuracy, the filtering identification model of the system is derived by changing the structure of the system without changing the relationship between the input and output. Further, a data filtering-based multi-innovation forgetting gradient algorithm is proposed through the use of the data filtering technique. A simulation example is employed to illustrate that the proposed approaches are effective for parameter estimation and the data filtering-based multi-innovation forgetting gradient algorithm has better estimation performance.     

Frequency-shifting-based stable on-line algebraic parameter identification of linear systems

In this paper a new approach to algebraic parameter identification of the linear SISO systems is proposed. The standard approach to the algebraic parameter identification is based on the algebraic derivatives in Laplace domain as the main tool for algebraic manipulations like elimination of the initial conditions and generation of linearly independent equations. This approach leads to the unstable time-varying state-space realization of the filters for the on-line parameter estimation. In this paper, the finite difference and shift operators in combination with the frequency-shifting property of Laplace transform is applied instead of algebraic derivatives. Resulting state-space realization of the estimator filters is asymptotically stable and doesn’t require switch-of mechanism to prevent overflow of the estimator variables. The proposed method is especially suitable for applications in closed-loop on-line identification where the stable behavior of the estimators is a necessary requirement. The efficiency of the proposed algorithm is illustrated on three simulation examples.     

Hierarchical recursive least squares parameter estimation of non-uniformly sampled Hammerstein nonlinear systems based on Kalman filter

This paper focuses on parameter estimation problems for non-uniformly sampled Hammerstein nonlinear systems. By combining the lifting technique and state space transformation, we derive a nonlinear regression identification model with different input and output updating rates. Furthermore, the unmeasurable state vector is estimated by Kalman filter, and by using the hierarchical identification principle, we develop a hierarchical recursive least squares algorithm for estimating the unknown parameters of the identification model. Finally, illustrative examples are given to indicate that the proposed algorithm is effective.     

Model recovery for multi-input signal-output nonlinear systems based on the compressed sensing recovery theory

This paper considers the parameter and order estimation for multiple-input single-output nonlinear systems. Since the orders of the system are unknown, a high-dimensional identification model and a sparse parameter vector are established to include all the valid inputs and basic parameters. Applying the data filtering technique, the input-output data are filtered and the original identification model with autoregressive noise is changed into the identification model with white noise. Based on the compressed sensing recovery theory, a data filtering-based orthogonal matching pursuit algorithm is presented for estimating the system parameters and the orders. The presented method can obtain highly accurate estimates from a small number of measurements by finding the highest absolute inner product. The simulation results confirm that the proposed algorithm is effective for recovering the model of the multiple-input single-output Hammerstein finite impulse response systems.     

Modified multi-innovation stochastic gradient algorithm for Wiener–Hammerstein systems with backlash

In this paper, the identification of the Wiener–Hammerstein systems with unknown orders linear subsystems and backlash is investigated by using the modified multi-innovation stochastic gradient identification algorithm. In this scheme, in order to facilitate subsequent parameter identification, the orders of linear subsystems are firstly determined by using the determinant ratio approach. To address the multi-innovation length problem in the conventional multi-innovation least squares algorithm, the innovation updating is decomposed into sub-innovations updating through the usage of multi-step updating technique. In the identification procedure, by reframing two auxiliary models, the unknown internal variables are replaced by using the outputs of the corresponding auxiliary model. Furthermore, the convergence analysis of the proposed algorithm has shown that the parameter estimation error can converge to zero. Simulation examples are provided to validate the efficiency of the proposed algorithm.     

A filtering based recursive least squares estimation algorithm for pseudo-linear auto-regressive systems

This paper uses the filtering technique, transforms a pseudo-linear auto-regressive system into an identification model and presents a new recursive least squares parameter estimation algorithm pseudo-linear auto-regressive systems. The proposed algorithm has a high computational efficiency because the dimensions of its covariance matrices become small compared with the recursive generalized least squares algorithm.     

Predefined-time parameter estimation via modified dynamic Regressor extension and mixing

This paper presents a novel switching predefined-time parameter identification algorithm with a relaxed excitation condition based on the dynamic regressor extension and mixing (DREM) method. DREM often requires the persistent excitation (PE) of the extended square regressor's determinant to ensure exponential parameter convergence. Unlike the classical DREM method, a new parameter identification algorithm configured with a two-layer filter technique is proposed under a relaxed initial excitation (IE) condition, rather than strict PE. A key point in choosing IE instead of PE is the introduction of a smooth switching function that dominates the pure integral action and filter behavior of the extended square regressor. The proposed algorithm relies on the predefined-time stability theorem and the settling-time of the identification algorithm is set a priori as a system parameter. The contributions of this paper are a novel switching predefined-time parameter estimation algorithm that 1) relaxes the stringent PE condition, 2) achieves predefined-time convergence, and 3) guarantees the monotonicity of each element of the parameter error inherited from the classical DREM method. Comparative simulation results are presented to illustrate the effectiveness of the proposed algorithm.     

Parameter estimation for an input nonlinear state space system with time delay

This paper researches parameter estimation problems for an input nonlinear system with state time-delay. Combining the linear transformation and the property of the shift operator, the system is transformed into a bilinear parameter identification model. A gradient based and a least squares based iterative parameter estimation algorithms are presented for identifying the state time-delay system. The simulation results confirm that the proposed two algorithms are effective and the least squares based iterative algorithm has faster convergence rates than the gradient based iterative algorithm.