Kalman filtering based gradient estimation algorithms for observer canonical state-space systems with moving average noises

This paper focuses on the joint parameter and state estimation issue for observer canonical state-space systems with white noises in state equations and moving average noises in output equations. By means of the Kalman filtering and the gradient search, we derive a Kalman filtering based extended stochastic gradient algorithm. For purpose of achieving the higher parameter estimation accuracy, a Kalman filtering based multi-innovation extended stochastic gradient algorithm is proposed on the basis of the multi-innovation identification theory. Finally, the effectiveness of the proposed algorithms is validated through a numerical example.     

Parameter estimation for a class of radial basis function-based nonlinear time-series models with moving average noises

This paper focuses on the parameter estimation for radial basis function-based state-dependent autoregressive models with moving average noises (RBF-ARMA models). An extended projection algorithm is derived based on the negative gradient search. In order to reduce the sensitivity of the algorithm to noise and reduce the fluctuations of the parameter estimation errors, a modified extended stochastic gradient algorithm is proposed. By introducing a moving data window, a modified moving data window-based extended stochastic gradient algorithm is further developed to improve the parameter estimation accuracy. The simulation results show that the proposed algorithms can effectively estimate the parameters of the RBF-ARMA models.     

Multi-innovation gradient estimation algorithms and convergence analysis for feedback nonlinear equation-error moving average systems

Considering the colored noises from the process environments, the parameter estimation problems for the feedback nonlinear equation-error systems interfered by moving average noises are addressed in this paper. Due to small computational burden, the gradient search principle is adopted to the feedback nonlinear systems and an overall extended stochastic gradient algorithm is derived for parameter estimation. Introducing the innovation length, the scalar innovation is expanded into the innovation vector and a multi-innovation extended stochastic gradient algorithm is further developed to reach the high estimation accuracy by utilizing more dynamical observed data. Furthermore, to assure the convergence of the proposed algorithms, their convergence properties are analyzed through the stochastic process theory. Finally, the experimental results indicate the effectiveness of the proposed algorithms.     

Modified multi-innovation stochastic gradient algorithm for Wiener–Hammerstein systems with backlash

In this paper, the identification of the Wiener–Hammerstein systems with unknown orders linear subsystems and backlash is investigated by using the modified multi-innovation stochastic gradient identification algorithm. In this scheme, in order to facilitate subsequent parameter identification, the orders of linear subsystems are firstly determined by using the determinant ratio approach. To address the multi-innovation length problem in the conventional multi-innovation least squares algorithm, the innovation updating is decomposed into sub-innovations updating through the usage of multi-step updating technique. In the identification procedure, by reframing two auxiliary models, the unknown internal variables are replaced by using the outputs of the corresponding auxiliary model. Furthermore, the convergence analysis of the proposed algorithm has shown that the parameter estimation error can converge to zero. Simulation examples are provided to validate the efficiency of the proposed algorithm.     

Combined state and parameter estimation for a bilinear state space system with moving average noise

This paper considers the identification problem of bilinear systems with measurement noise in the form of the moving average model. In particular, we present an interactive estimation algorithm for unmeasurable states and parameters based on the hierarchical identification principle. For unknown states, we formulate a novel bilinear state observer from input-output measurements using the Kalman filter. Then a bilinear state observer based multi-innovation extended stochastic gradient (BSO-MI-ESG) algorithm is proposed to estimate the unknown system parameters. A linear filter is utilized to improve the parameter estimation accuracy and a filtering based BSO-MI-ESG algorithm is presented using the data filtering technique. In the numerical example, we illustrate the effectiveness of the proposed identification methods.     

Fitting the exponential autoregressive model through recursive search

This paper focuses on the recursive parameter estimation methods for the exponential autoregressive (ExpAR) model. Applying the negative gradient search and introducing a forgetting factor, a stochastic gradient and a forgetting factor stochastic gradient algorithms are presented. In order to improve the parameter estimation accuracy and the convergence rate, the multi-innovation identification theory is employed to derive a forgetting factor multi-innovation stochastic gradient algorithm. A simulation example is provided to test the effectiveness of the proposed algorithms.     

Several gradient parameter estimation algorithms for dual-rate sampled systems

This paper presents three identification methods for dual-rate sampled systems. The first method combines the stochastic gradient algorithm with the polynomial transformation technique, which can estimate the parameters of the identification model. The second method is the finite impulse response model based stochastic gradient algorithm, which can indirectly estimate the parameters of the dual-rate systems by using all the inputs and the available outputs. The third method is the missing output estimation model based stochastic gradient algorithm with a forgetting factor, which can directly estimate the parameters of the dual-rate systems by using all the inputs and all the outputs (include the estimated outputs). An example is provided to verify the effectiveness of the proposed methods.     

Globally exponentially stable single beacon underwater navigation with unknown sound velocity estimation

Single beacon navigation methods with unknown effective sound velocity (ESV) have recently been proposed to solve the performance degeneration induced by ESV setting error. In these methods, a local linearization-based state estimator, which only exhibits local convergence, is adopted to estimate the navigation state. When the initial ESV setting error or vehicle initial position error is large, the local linearization-based state estimators have difficulty guaranteeing the filtering convergence. With this background, this paper proposes a linear time-varying single beacon navigation model with an unknown ESV that can realize global convergence under the condition of system observability. A Kalman filter is adopted to estimate the model state, and the corresponding stochastic model is inferred for the application of the Kalman filter. Numerical simulation confirms that the proposed linear time-varying single beacon navigation model can realize fast convergence in the case of a large initial error, and has superior steady-state performance compared with the existing methods.     

Cyclic gradient based iterative algorithm for a class of generalized coupled Sylvester-conjugate matrix equations

This paper focuses on the numerical solution of a class of generalized coupled Sylvester-conjugate matrix equations, which are general and contain many significance matrix equations as special cases, such as coupled discrete-time/continuous-time Markovian jump Lyapunov matrix equations, stochastic Lyapunov matrix equation, etc. By introducing the modular operator, a cyclic gradient based iterative (CGI) algorithm is provided. Different from some previous iterative algorithms, the most significant improvement of the proposed algorithm is that less information is used during each iteration update, which is conducive to saving memory and improving efficiency. The convergence of the proposed algorithm is discussed, and it is verified that the algorithm converges for any initial matrices under certain assumptions. Finally, the effectiveness and superiority of the proposed algorithm are verified with some numerical examples.     

The filtering based parameter identification for bilinear-in-parameter systems

This paper discusses the parameter estimation for a class of bilinear-in-parameter systems with colored noise. By utilizing the filtering technique, we derive the relationship between the filtered output and the measurement output and obtain two linear regressive sub-models. A filtering based multi-innovation stochastic gradient algorithm is derived for interactively identifying each sub-model. The proposed algorithm avoids the estimation of correlated noise and improves the parameter estimation accuracy by making full use of the measurement data. The numerical simulation results indicate that the proposed algorithm has higher estimation accuracy than the hierarchical multi-innovation stochastic gradient algorithm.     

An augmented Lagrangian algorithm for total bounded variation regularization based image deblurring

The augmented Lagrangian strategy has recently emerged as an important methodology for image processing problems. In this paper, based on this strategy, we propose a new projected gradient algorithm for image deblurring with total bounded variation regularization. The convergence property of our algorithm is discussed. Numerical experiments show that the proposed algorithm can yield better visual quality than the Rudin–Osher–Fatemi (ROF) method and the split Bregman iteration method.     

Event-based state and fault estimation for stochastic nonlinear system with Markov packet dropout

This paper investigates the event-based state and fault estimation problem for stochastic nonlinear system with Markov packet dropout. By introducing the fictitious noise, the fault is augmented to the system state. Then combining the unscented Kalman filter (UKF) with event-triggered and Markov packet dropout, the modified UKF is proposed to estimate the state and fault. Meanwhile, the stochastic stability of the proposed filter is also discussed. Finally, two simulation results illustrate the performance of the proposed method.     

Filtering based parameter estimation for observer canonical state space systems with colored noise

This paper surveys the identification of observer canonical state space systems affected by colored noise. By means of the filtering technique, a filtering based recursive generalized extended least squares algorithm is proposed for enhancing the parameter identification accuracy. To ease the computational burden, the filtered regressive model is separated into two fictitious sub-models, and then a filtering based two-stage recursive generalized extended least squares algorithm is developed on the basis of the hierarchical identification. The stochastic martingale theory is applied to analyze the convergence of the proposed algorithms. An experimental example is provided to validate the proposed algorithms.     

Parameter estimation for an input nonlinear state space system with time delay

This paper researches parameter estimation problems for an input nonlinear system with state time-delay. Combining the linear transformation and the property of the shift operator, the system is transformed into a bilinear parameter identification model. A gradient based and a least squares based iterative parameter estimation algorithms are presented for identifying the state time-delay system. The simulation results confirm that the proposed two algorithms are effective and the least squares based iterative algorithm has faster convergence rates than the gradient based iterative algorithm.     

一种新的变步长盲均衡算法研究

常数模算法（constant modulus algorithm,CMA）能够很好地克服无线信道引入的符号间干扰（ISI）,在信道均衡中广泛应用,但存在稳态误差大,相位旋转的问题;MMA算法解决了CMA算法的相位旋转问题,但仍然有较大的稳态误差。为了克服以上缺点,在研究各种算法的基础上,引入非线性函数来构造步长调整参数,计算机仿真结果表明,相比传统算法,变步长盲均衡算法有较快的收敛速度和更好的均衡效果。     

Recursive parameter estimation algorithm for multivariate output-error systems

In this paper, we consider the parameter estimation issues of a class of multivariate output-error systems. A decomposition based recursive least squares identification method is proposed using the hierarchical identification principle and the auxiliary model idea, and its convergence is analyzed through the stochastic process theory. Compared with the existing results on parameter estimation of multivariate output-error systems, a distinct feature for the proposed algorithm is that such a system is decomposed into several sub-systems with smaller dimensions so that parameters to be identified can be estimated interactively. The analysis shows that the estimation errors converge to zero in mean square under certain conditions. Finally, in order to show the effectiveness of the proposed approach, some numerical simulations are provided.     

Synchronization of linear dynamical networks under stochastic impulsive coupling protocols

This paper investigates a stochastic impulsive coupling protocol for synchronization of linear dynamical networks based on discrete-time sampled-data. The convergence of the networks under the proposed protocol is discussed, and some sufficient conditions are showed to guarantee almost sure exponential synchronization. Moreover, this coupling protocol with a pinning control scheme is developed to lead the state of all nodes to almost sure exponentially converge to a virtual synchronization target. It is shown that the almost sure exponential synchronization can be achieved by only interacting based on the stochastic feedback information at discrete-time instants. Some numerical examples are finally provided to present the effectiveness of the proposed stochastic coupling protocols.     

The relaxed gradient-based iterative algorithms for a class of generalized coupled Sylvester-conjugate matrix equations

In this paper, two relaxed gradient-based iterative algorithms for solving a class of generalized coupled Sylvester-conjugate matrix equations are proposed. The proposed algorithm is different from the gradient-based iterative algorithm and the modified gradient-based iterative algorithm that are recently available in the literature. With the real representation of a complex matrix as a tool, the sufficient and necessary condition for the convergence factor is determined to guarantee that the iterative solution given by the proposed algorithms converge to the exact solution for any initial matrices. Moreover, some sufficient convergence conditions for the suggested algorithms are presented. Finally, numerical example is provided to illustrate the effectiveness of the proposed algorithms and testify the conclusions suggested in this paper.     

Primal-dual stochastic distributed algorithm for constrained convex optimization

This paper investigates distributed convex optimization problems over an undirected and connected network, where each node’s variable lies in a private constrained convex set, and overall nodes aim at collectively minimizing the sum of all local objective functions. Motivated by a variety of applications in machine learning problems with large-scale training sets distributed to multiple autonomous nodes, each local objective function is further designed as the average of moderate number of local instantaneous functions. Each local objective function and constrained set cannot be shared with others. A primal-dual stochastic algorithm is presented to address the distributed convex optimization problems, where each node updates its state by resorting to unbiased stochastic averaging gradients and projects on its private constrained set. At each iteration, for each node the gradient of one local instantaneous function selected randomly is evaluated and the average of the most recent stochastic gradients is used to approximate the true local gradient. In the constrained case, we show that with strong-convexity of the local instantaneous function and Lipschitz continuity of its gradient, the algorithm converges to the global optimization solution almost surely. In the unconstrained case, an explicit linear convergence rate of the algorithm is provided. Numerical experiments are presented to demonstrate correctness of the theoretical results.     

M-estimation based sparse grid quadrature filter and stochastic stability analysis

In this study, a novel M-estimation based sparse grid quadrature filter (MSGQF) is proposed to improve the robust performance of the nonlinear system. We present a systematic formulation of the sparse grid quadrature filter (SGQF), and extend it to the discrete-time nonlinear system with abnormal measurement values. The M-estimation method is introduced in the SGQF, which uses the Huber’s cost function to update the measurement covariance. Convergence on the modified robust SGQF is established and proved. The sufficient conditions are shown to ensure stochastic stability of the MSGQF. A target tracking problem has been conducted to demonstrate the accuracy of the MSGQF. When measurement abnormal values appear, it outperforms the unscented Kalman filter (UKF), the cubature Kalman filter (CKF) and the SGQF. Theoretical analysis and simulation results prove that the MSGQF provides significant performance improvement in the robustness of the nonlinear system.