Auxiliary model based recursive generalized least squares identification algorithm for multivariate output-error autoregressive systems using the decomposition technique

This paper focuses on the parameter estimation problem of multivariate output-error autoregressive systems. Based on the decomposition technique and the auxiliary model identification idea, we derive a decomposition based auxiliary model recursive generalized least squares algorithm. The key is to divide the system into two fictitious subsystems, the one including a parameter vector and the other including a parameter matrix, and to estimate the two subsystems using the recursive least squares method, respectively. Compared with the auxiliary model based recursive generalized least squares algorithm, the proposed algorithm has less computational burden. Finally, an illustrative example is provided to verify the effectiveness of the proposed algorithms.     

Maximum likelihood recursive least squares estimation for multivariate equation-error ARMA systems

This paper focuses on the parameter estimation problems of multivariate equation-error systems. A recursive generalized extended least squares algorithm is presented as a comparison. Based on the maximum likelihood principle and the coupling identification concept, the multivariate equation-error system is decomposed into several regressive identification models, each of which has only a parameter vector, and a coupled subsystem maximum likelihood recursive least squares identification algorithm is developed for estimating the parameter vectors of these submodels. The simulation example shows that the proposed algorithm is effective and has high estimation accuracy.     

The maximum likelihood least squares based iterative estimation algorithm for bilinear systems with autoregressive moving average noise

Maximum likelihood methods are significant for parameter estimation and system modeling. This paper gives the input-output representation of a bilinear system through eliminating the state variables in it, and derives a maximum likelihood least squares based iterative for identifying the parameters of bilinear systems with colored noises by using the maximum likelihood principle. A least squares based iterative (LSI) algorithm is presented for comparison. It is proved that the maximum of the likelihood function is equivalent to minimize the least squares cost function. The simulation results indicate that the proposed algorithm is effective for identifying bilinear systems and the maximum likelihood LSI algorithm is more accurate than the LSI algorithm.     

The data-filtering based bias compensation recursive least squares identification for multi-input single-output systems with colored noises

For the multi-input single-output (MISO) system corrupted by colored noise, we transform the original system model into a new MISO output error model with white noise through data filtering technology. Based on the newly obtained model and the bias compensation principle, a novel data filtering-based bias compensation recursive least squares (BCRLS) identification algorithm is developed for identifying the parameters of the MISO system with colored noise disturbance. Unlike the exiting BCRLS method for the MISO system (see, in Section 3), without computing the complicated noise correlation functions, still the proposed method can achieve the unbiased parameters estimation of the MISO system in the case of colored process noises. The proposed algorithm simplifies the implementation of and further expands the application scope of the existing BCRLS method. Three numerical examples clearly illustrate the validity of and the good performances of the proposed method, including its superiority over the BCRLS method and so on.     

A filtering based recursive least squares estimation algorithm for pseudo-linear auto-regressive systems

This paper uses the filtering technique, transforms a pseudo-linear auto-regressive system into an identification model and presents a new recursive least squares parameter estimation algorithm pseudo-linear auto-regressive systems. The proposed algorithm has a high computational efficiency because the dimensions of its covariance matrices become small compared with the recursive generalized least squares algorithm.     

Mixture quantized error entropy for recursive least squares adaptive filtering

Error entropy is a well-known learning criterion in information theoretic learning (ITL), and it has been successfully applied in robust signal processing and machine learning. To date, many robust learning algorithms have been devised based on the minimum error entropy (MEE) criterion, and the Gaussian kernel function is always utilized as the default kernel function in these algorithms, which is not always the best option. To further improve learning performance, two concepts using a mixture of two Gaussian functions as kernel functions, called mixture error entropy and mixture quantized error entropy, are proposed in this paper. We further propose two new recursive least-squares algorithms based on mixture minimum error entropy (MMEE) and mixture quantized minimum error entropy (MQMEE) optimization criteria. The convergence analysis, steady-state mean-square performance, and computational complexity of the two proposed algorithms are investigated. In addition, the reason why the mixture mechanism (mixture correntropy and mixture error entropy) can improve the performance of adaptive filtering algorithms is explained. Simulation results show that the proposed new recursive least-squares algorithms outperform other RLS-type algorithms, and the practicality of the proposed algorithms is verified by the electro-encephalography application.     

Hierarchical recursive generalized extended least squares estimation algorithms for a class of nonlinear stochastic systems with colored noise

This paper considers the parameter identification problem of a bilinear state space system with colored noise based on its input-output representation. An input-output representation of a bilinear state-space system is derived for the parameter identification by eliminating the state variables in the model, and a recursive generalized extended least squares algorithm is presented for estimating the parameters of the obtained model. Furthermore, a three-stage recursive generalized extended least squares algorithm is proposed for reducing the computational cost. The validity of the proposed method is evaluated through a numerical example.     

Hierarchical recursive least squares parameter estimation of non-uniformly sampled Hammerstein nonlinear systems based on Kalman filter

This paper focuses on parameter estimation problems for non-uniformly sampled Hammerstein nonlinear systems. By combining the lifting technique and state space transformation, we derive a nonlinear regression identification model with different input and output updating rates. Furthermore, the unmeasurable state vector is estimated by Kalman filter, and by using the hierarchical identification principle, we develop a hierarchical recursive least squares algorithm for estimating the unknown parameters of the identification model. Finally, illustrative examples are given to indicate that the proposed algorithm is effective.     

A hierarchical least squares identification algorithm for Hammerstein nonlinear systems using the key term separation

Mathematical models are basic for designing controller and system identification is the theory and methods for establishing the mathematical models of practical systems. This paper considers the parameter identification for Hammerstein controlled autoregressive systems. Using the key term separation technique to express the system output as a linear combination of the system parameters, the system is decomposed into several subsystems with fewer variables, and then a hierarchical least squares (HLS) algorithm is developed for estimating all parameters involving in the subsystems. The HLS algorithm requires less computation than the recursive least squares algorithm. The computational efficiency comparison and simulation results both confirm the effectiveness of the proposed algorithms.     

Auxiliary model based least squares parameter estimation algorithm for feedback nonlinear systems using the hierarchical identification principle

This paper presents a decomposition based least squares estimation algorithm for a feedback nonlinear system with an output error model for the open-loop part by using the auxiliary model identification idea and the hierarchical identification principle and by decomposing a system into two subsystems. Compared with the auxiliary model based recursive least squares algorithm, the proposed algorithm has a smaller computational burden. The simulation results indicate that the proposed algorithm can estimate the parameters of feedback nonlinear systems effectively.     

Hierarchical gradient- and least squares-based iterative algorithms for input nonlinear output-error systems using the key term separation

This paper considers the parameter identification problems of the input nonlinear output-error (IN-OE) systems, that is the Hammerstein output-error systems. In order to overcome the excessive calculation amount of the over-parameterization method of the IN-OE systems. Through applying the hierarchial identification principle and decomposing the IN-OE system into three subsystems with a smaller number of parameters, we present the key term separation auxiliary model hierarchical gradient-based iterative algorithm and the key term separation auxiliary model hierarchical least squares-based iterative algorithm, which are called the key term separation auxiliary model three-stage gradient-based iterative algorithm and the key term separation auxiliary model three-stage least squares-based iterative algorithm. The comparison of the calculation amount and the simulation analysis indicate that the proposed algorithms are effective.     

Data filtering based maximum likelihood extended gradient method for multivariable systems with autoregressive moving average noise

For multivariable systems with autoregressive moving average noises, we decompose the multivariable system into m subsystems (m denotes the number of outputs) and present a maximum likelihood generalized extended gradient algorithm and a data filtering based maximum likelihood extended gradient algorithm to estimate the parameter vectors of these subsystems. By combining the maximum likelihood principle and the data filtering technique, the proposed algorithms are effective and have computational advantages over existing estimation algorithms. Finally, a numerical simulation example is given to support the developed methods and to show their effectiveness.     

Particle filtering based parameter estimation for systems with output-error type model structures

The output-error model structure is often used in practice and its identification is important for analysis of output-error type systems. This paper considers the parameter identification of linear and nonlinear output-error models. A particle filter which approximates the posterior probability density function with a weighted set of discrete random sampling points is utilized to estimate the unmeasurable true process outputs. To improve the convergence rate of the proposed algorithm, the scalar innovations are grouped into an innovation vector, thus more past information can be utilized. The convergence analysis shows that the parameter estimates can converge to their true values. Finally, both linear and nonlinear results are verified by numerical simulation and engineering.     

Recursive parameter estimation algorithm for multivariate output-error systems

In this paper, we consider the parameter estimation issues of a class of multivariate output-error systems. A decomposition based recursive least squares identification method is proposed using the hierarchical identification principle and the auxiliary model idea, and its convergence is analyzed through the stochastic process theory. Compared with the existing results on parameter estimation of multivariate output-error systems, a distinct feature for the proposed algorithm is that such a system is decomposed into several sub-systems with smaller dimensions so that parameters to be identified can be estimated interactively. The analysis shows that the estimation errors converge to zero in mean square under certain conditions. Finally, in order to show the effectiveness of the proposed approach, some numerical simulations are provided.     

Decomposition based least squares iterative estimation algorithm for two-input single-output output error systems

This paper develops a decomposition based least squares iterative identification algorithm for two-input single-output (TISO) systems. The basic idea is to decompose a TISO system into two subsystems and then to identify each subsystem, respectively. Compared with the least squares based iterative algorithm, the proposed algorithm has less computational load. The simulation results indicate that the proposed algorithm is effective.     

Gradient-based iterative identification method for multivariate equation-error autoregressive moving average systems using the decomposition technique

This paper studies the parameter estimation problems of multivariate equation-error autoregressive moving average systems. Firstly, a gradient-based iterative algorithm is presented as a comparison. In order to improve the computational efficiency and the parameter estimation accuracy, a decomposition-based gradient iterative algorithm is presented by using the decomposition technique. The key is to transform an original system into two subsystems and to estimate the parameters of each subsystem, respectively. Compared with the gradient-based iterative algorithm, the decomposition-based algorithm requires less computational efforts, and the simulation results indicate that this algorithm is effective.     

Direction of arrival tracking for signals with known waveforms based on block least squares techniques

In this paper, we address the problem of tracking DOA of multiple moving targets with known signal source waveforms and unknown gains in the presence of Gaussian noise using a nonuniform linear array. Herein, we make use of the fact that the output of each sensor can be described as a linear regression model whose coefficients each contain a pair of DOA and gain information corresponding to one target. These coefficients are determined by solving a linear least squares (LS) problem and then updated recursively based on a block QR decomposition recursive least squares (QRD-RLS) technique or a block regularized LS technique. Since the coefficients from different sensors have the same amplitude but variable phase information for the same signal, along with simple algebraic manipulations the well-known generalized least squares (GLS) are used to obtain an asymptotically-optimal DOA estimate without requiring a search over a large region of the parameter space. Computer simulations show that the proposed DOA tracking techniques when applied to a sparse antenna array can provide a better tracking performance than some of the existing methods do.     

Maximum likelihood carrier frequency offset estimation algorithm with adjustable frequency acquisition region

This study presents a simple yet effective carrier frequency offset (CFO) estimation algorithm for orthogonal frequency division multiplexing (OFDM) systems. At the transmitter, the proposed algorithm uses null subcarriers to render the OFDM signal periodic in the time domain. At the receiver, these periodic time samples become CFO-bearing signals, which can be adopted to develop the maximum likelihood (ML) CFO estimation algorithm accordingly. In addition to providing reliable and efficient CFO estimation, the proposed algorithm has an adjustable acquisition region linearly proportional to the order of the null subcarrier insertion scheme.     

Dynamic event-based recursive filtering for networked systems under the encoding-decoding mechanism

This paper focuses on the dynamic event-based recursive filtering problem for a class of time-varying networked systems under the encoding-decoding mechanism. For the purpose of saving energy consumption, a dynamic event-triggered protocol is applied to determine whether the measurement of the sensor is transmitted or not. In the transmission process of the measurement, a dynamic-quantization-based encoding-decoding mechanism is introduced to encrypt the transmitted measurement. In specific, the measurement outputs are first encoded into codewords which are then transmitted from the encoder to the decoder. After received by the decoder, the codewords are first decoded and then sent to the filter. A bounded uncertainty is introduced to characterize the difference between the original measurement and the decoded measurement. This paper is devoted to developing a recursive filtering algorithm for the considered system such that a minimal upper bound on the filtering error covariance is derived through appropriately designing filter gain. Moreover, the mean-square exponential boundedness of the filtering error is analyzed. Finally, the efficiency and superiority of the proposed algorithm are verified through two simulation examples.     

Distributed dynamic state estimation with parameter identification for large-scale systems

In this paper, we consider a distributed dynamic state estimation problem for time-varying systems. Based on the distributed maximum a posteriori (MAP) estimation algorithm proposed in our previous study, which studies the linear measurement models of each subsystem, and by weakening the constraint condition as that each time-varying subsystem is observable, this paper proves that the error covariances of state estimation and prediction obtained from the improved algorithm are respectively positive definite and have upper bounds, which verifies the feasibility of this algorithm. We also use new weighting functions and time-varying exponential smoothing method to ensure the robustness and improve the forecast accuracy of the distributed state estimation method. At last, an example is used to demonstrate the effectiveness of the proposed algorithm together with the parameter identification.