Recursive parameter estimation algorithm for multivariate output-error systems

In this paper, we consider the parameter estimation issues of a class of multivariate output-error systems. A decomposition based recursive least squares identification method is proposed using the hierarchical identification principle and the auxiliary model idea, and its convergence is analyzed through the stochastic process theory. Compared with the existing results on parameter estimation of multivariate output-error systems, a distinct feature for the proposed algorithm is that such a system is decomposed into several sub-systems with smaller dimensions so that parameters to be identified can be estimated interactively. The analysis shows that the estimation errors converge to zero in mean square under certain conditions. Finally, in order to show the effectiveness of the proposed approach, some numerical simulations are provided.     

Auxiliary model based recursive generalized least squares identification algorithm for multivariate output-error autoregressive systems using the decomposition technique

This paper focuses on the parameter estimation problem of multivariate output-error autoregressive systems. Based on the decomposition technique and the auxiliary model identification idea, we derive a decomposition based auxiliary model recursive generalized least squares algorithm. The key is to divide the system into two fictitious subsystems, the one including a parameter vector and the other including a parameter matrix, and to estimate the two subsystems using the recursive least squares method, respectively. Compared with the auxiliary model based recursive generalized least squares algorithm, the proposed algorithm has less computational burden. Finally, an illustrative example is provided to verify the effectiveness of the proposed algorithms.     

Iterative identification for multiple-input systems with time-delays based on greedy pursuit and auxiliary model

This paper focuses on the identification of multiple-input single-output output-error systems with unknown time-delays. Since the time-delays are unknown, an identification model with a high dimensional and sparse parameter vector is derived based on overparameterization. Traditional identification methods cannot get sparse solutions and require a large number of observations unless the time-delays are predetermined. Inspired by the sparse optimization and the greedy algorithms, an auxiliary model based orthogonal matching pursuit iterative (AM-OMPI) algorithm is proposed by using the orthogonal matching pursuit, and then based on the gradient search, an auxiliary model based gradient pursuit iterative algorithm is proposed, which is computationally more efficient than the AM-OMPI algorithm. The proposed methods can simultaneously estimate the parameters and time-delays from a small number of sampled data. A simulation example is used to illustrate the effectiveness of the proposed algorithms.     

A hierarchical least squares identification algorithm for Hammerstein nonlinear systems using the key term separation

Mathematical models are basic for designing controller and system identification is the theory and methods for establishing the mathematical models of practical systems. This paper considers the parameter identification for Hammerstein controlled autoregressive systems. Using the key term separation technique to express the system output as a linear combination of the system parameters, the system is decomposed into several subsystems with fewer variables, and then a hierarchical least squares (HLS) algorithm is developed for estimating all parameters involving in the subsystems. The HLS algorithm requires less computation than the recursive least squares algorithm. The computational efficiency comparison and simulation results both confirm the effectiveness of the proposed algorithms.     

Auxiliary model based least squares parameter estimation algorithm for feedback nonlinear systems using the hierarchical identification principle

This paper presents a decomposition based least squares estimation algorithm for a feedback nonlinear system with an output error model for the open-loop part by using the auxiliary model identification idea and the hierarchical identification principle and by decomposing a system into two subsystems. Compared with the auxiliary model based recursive least squares algorithm, the proposed algorithm has a smaller computational burden. The simulation results indicate that the proposed algorithm can estimate the parameters of feedback nonlinear systems effectively.     

Gradient-based iterative identification method for multivariate equation-error autoregressive moving average systems using the decomposition technique

This paper studies the parameter estimation problems of multivariate equation-error autoregressive moving average systems. Firstly, a gradient-based iterative algorithm is presented as a comparison. In order to improve the computational efficiency and the parameter estimation accuracy, a decomposition-based gradient iterative algorithm is presented by using the decomposition technique. The key is to transform an original system into two subsystems and to estimate the parameters of each subsystem, respectively. Compared with the gradient-based iterative algorithm, the decomposition-based algorithm requires less computational efforts, and the simulation results indicate that this algorithm is effective.     

Particle filtering based parameter estimation for systems with output-error type model structures

The output-error model structure is often used in practice and its identification is important for analysis of output-error type systems. This paper considers the parameter identification of linear and nonlinear output-error models. A particle filter which approximates the posterior probability density function with a weighted set of discrete random sampling points is utilized to estimate the unmeasurable true process outputs. To improve the convergence rate of the proposed algorithm, the scalar innovations are grouped into an innovation vector, thus more past information can be utilized. The convergence analysis shows that the parameter estimates can converge to their true values. Finally, both linear and nonlinear results are verified by numerical simulation and engineering.     

Maximum likelihood interval-varying recursive least squares identification for output-error autoregressive systems with scarce measurements

The identification problem of output-error autoregressive (OEAR) systems with scarce measurements is considered in this paper. In order to overcome the massive absence of outputs, an interval-varying recursive identification algorithm is proposed through changing the sampling interval and skipping the missing outputs. Based on the maximum likelihood principle, a maximum likelihood interval-varying recursive least squares algorithm is proposed. The effectiveness of the proposed algorithm is tested by a numerical simulation example, and an application example about the heading motion control of underwater vehicle.     

Gradient iterative algorithm for dual-rate nonlinear systems based on a novel particle filter

This paper proposes a novel particle filter based gradient iterative algorithm for the identification of dual-rate nonlinear systems. The novel particle filter is applied to estimate the missing outputs, and the measurable outputs are utilized to adjust the weights of particles during each interval of the slow sampled rate. Then the missing outputs and the unknown parameters can be estimated iteratively by the novel particle filter based gradient iterative algorithm. The simulation results indicate that the proposed method is more effective than the classical auxiliary model method.     

Hierarchical recursive least squares parameter estimation of non-uniformly sampled Hammerstein nonlinear systems based on Kalman filter

This paper focuses on parameter estimation problems for non-uniformly sampled Hammerstein nonlinear systems. By combining the lifting technique and state space transformation, we derive a nonlinear regression identification model with different input and output updating rates. Furthermore, the unmeasurable state vector is estimated by Kalman filter, and by using the hierarchical identification principle, we develop a hierarchical recursive least squares algorithm for estimating the unknown parameters of the identification model. Finally, illustrative examples are given to indicate that the proposed algorithm is effective.     

The relaxed gradient-based iterative algorithms for a class of generalized coupled Sylvester-conjugate matrix equations

In this paper, two relaxed gradient-based iterative algorithms for solving a class of generalized coupled Sylvester-conjugate matrix equations are proposed. The proposed algorithm is different from the gradient-based iterative algorithm and the modified gradient-based iterative algorithm that are recently available in the literature. With the real representation of a complex matrix as a tool, the sufficient and necessary condition for the convergence factor is determined to guarantee that the iterative solution given by the proposed algorithms converge to the exact solution for any initial matrices. Moreover, some sufficient convergence conditions for the suggested algorithms are presented. Finally, numerical example is provided to illustrate the effectiveness of the proposed algorithms and testify the conclusions suggested in this paper.     

Data filtering-based multi-innovation forgetting gradient algorithms for input nonlinear FIR-MA systems with piecewise-linear characteristics

The piecewise-linear characteristics often appear in the nonlinear systems that operate in different ways in different input regions. This paper studies the identification issue of a class of block-oriented systems with piecewise-linear characteristics. The asymmetric piecewise-linear nonlinearity is expressed as a linear parametric representation through introducing an appropriate switching function, then the identification model of the system is derived by using the key term separation technique. On this model basis, a multi-innovation forgetting gradient algorithm is presented to estimate the unknown parameters. To further enhance the identification accuracy, the filtering identification model of the system is derived by changing the structure of the system without changing the relationship between the input and output. Further, a data filtering-based multi-innovation forgetting gradient algorithm is proposed through the use of the data filtering technique. A simulation example is employed to illustrate that the proposed approaches are effective for parameter estimation and the data filtering-based multi-innovation forgetting gradient algorithm has better estimation performance.     

Modified multi-innovation stochastic gradient algorithm for Wiener–Hammerstein systems with backlash

In this paper, the identification of the Wiener–Hammerstein systems with unknown orders linear subsystems and backlash is investigated by using the modified multi-innovation stochastic gradient identification algorithm. In this scheme, in order to facilitate subsequent parameter identification, the orders of linear subsystems are firstly determined by using the determinant ratio approach. To address the multi-innovation length problem in the conventional multi-innovation least squares algorithm, the innovation updating is decomposed into sub-innovations updating through the usage of multi-step updating technique. In the identification procedure, by reframing two auxiliary models, the unknown internal variables are replaced by using the outputs of the corresponding auxiliary model. Furthermore, the convergence analysis of the proposed algorithm has shown that the parameter estimation error can converge to zero. Simulation examples are provided to validate the efficiency of the proposed algorithm.     

Performance analysis of the recursive parameter estimation algorithms for multivariable Box–Jenkins systems

Two auxiliary model based recursive identification algorithms, a generalized extended stochastic gradient algorithm and a recursive generalized extended least squares algorithm, are developed for multivariable Box–Jenkins systems. The basic idea is to use the auxiliary models to estimate the unknown noise-free outputs of the system and to replace the unmeasurable terms in the information vectors with their estimates. We prove that the estimation errors given by the proposed algorithms converge to zero under the persistent excitation condition. Finally, an example is provided to show the effectiveness of the proposed algorithms.     

Decomposition based least squares iterative estimation algorithm for two-input single-output output error systems

This paper develops a decomposition based least squares iterative identification algorithm for two-input single-output (TISO) systems. The basic idea is to decompose a TISO system into two subsystems and then to identify each subsystem, respectively. Compared with the least squares based iterative algorithm, the proposed algorithm has less computational load. The simulation results indicate that the proposed algorithm is effective.     

Filtering based parameter estimation for observer canonical state space systems with colored noise

This paper surveys the identification of observer canonical state space systems affected by colored noise. By means of the filtering technique, a filtering based recursive generalized extended least squares algorithm is proposed for enhancing the parameter identification accuracy. To ease the computational burden, the filtered regressive model is separated into two fictitious sub-models, and then a filtering based two-stage recursive generalized extended least squares algorithm is developed on the basis of the hierarchical identification. The stochastic martingale theory is applied to analyze the convergence of the proposed algorithms. An experimental example is provided to validate the proposed algorithms.     

Parameter estimation for an input nonlinear state space system with time delay

This paper researches parameter estimation problems for an input nonlinear system with state time-delay. Combining the linear transformation and the property of the shift operator, the system is transformed into a bilinear parameter identification model. A gradient based and a least squares based iterative parameter estimation algorithms are presented for identifying the state time-delay system. The simulation results confirm that the proposed two algorithms are effective and the least squares based iterative algorithm has faster convergence rates than the gradient based iterative algorithm.     

On the relaxed gradient-based iterative methods for the generalized coupled Sylvester-transpose matrix equations

In this paper, we propose the full-rank and reduced-rank relaxed gradient-based iterative algorithms for solving the generalized coupled Sylvester-transpose matrix equations. We provide analytically the necessary and sufficient condition for the convergence of the proposed iterative algorithm and give explicitly the optimal step size such that the convergence rate of the algorithm is maximized. Some numerical examples are examined to confirm the feasibility and efficiency of the proposed algorithms.     

Hierarchical recursive generalized extended least squares estimation algorithms for a class of nonlinear stochastic systems with colored noise

This paper considers the parameter identification problem of a bilinear state space system with colored noise based on its input-output representation. An input-output representation of a bilinear state-space system is derived for the parameter identification by eliminating the state variables in the model, and a recursive generalized extended least squares algorithm is presented for estimating the parameters of the obtained model. Furthermore, a three-stage recursive generalized extended least squares algorithm is proposed for reducing the computational cost. The validity of the proposed method is evaluated through a numerical example.     

The maximum likelihood least squares based iterative estimation algorithm for bilinear systems with autoregressive moving average noise

Maximum likelihood methods are significant for parameter estimation and system modeling. This paper gives the input-output representation of a bilinear system through eliminating the state variables in it, and derives a maximum likelihood least squares based iterative for identifying the parameters of bilinear systems with colored noises by using the maximum likelihood principle. A least squares based iterative (LSI) algorithm is presented for comparison. It is proved that the maximum of the likelihood function is equivalent to minimize the least squares cost function. The simulation results indicate that the proposed algorithm is effective for identifying bilinear systems and the maximum likelihood LSI algorithm is more accurate than the LSI algorithm.