A generalized direct search acceptable-point technique for use with descent-type multivariate algorithms

Many non-linear programming algorithms employ a univariate subprocedure to determine the step length at each multivariate iteration. In recent years much work has been directed toward the development of algorithms which will exhibit favorable convergence properties on well-behaved functions without requiring that the univariate algorithm perform a sequence of one-dimensional minimizations.In this paper a direct search method (the golden section search) is modified to search for acceptable rather than minimizing step lengths and then used as the univariate subprocedure for a generalized conjugate gradient algorithm. The resulting multivariate minimization method is tested on standard unconstrained test functions and a constrained industrial problem. The new method is found to be relatively insensitive to tuning parameters (insofar as success or failure is concerned).A comparison of the golden section acceptable-point search (GSAP) with other popular acceptable-point methods indicates that GSAP is a superior strategy for use with the conjugate directions-type algorithms and is also suitable for use with the quasi-Newton methods. The comparison are based on equivalent function evaluations required to minimize multivariate test functions.     

Gradient-based neural networks for solving periodic Sylvester matrix equations

This paper considers neural network solutions of a category of matrix equation called periodic Sylvester matrix equation (PSME), which appear in the process of periodic system analysis and design. A linear gradient-based neural network (GNN) model aimed at solving the PSME is constructed, whose state is able to converge to the unknown matrix of the equation. In order to obtain a better convergence effect, the linear GNN model is extended to a nonlinear form through the intervention of appropriate activation functions, and its convergence is proved through theoretical derivation. Furthermore, the different convergence effects presented by the model with various activation functions are also explored and analyzed, for instance, the global exponential convergence and the global finite time convergence can be realized. Finally, the numerical examples are used to confirm the validity of the proposed GNN model for solving the PSME considered in this paper as well as the superiority in terms of the convergence effect presented by the model with different activation functions.     

Fitting the exponential autoregressive model through recursive search

This paper focuses on the recursive parameter estimation methods for the exponential autoregressive (ExpAR) model. Applying the negative gradient search and introducing a forgetting factor, a stochastic gradient and a forgetting factor stochastic gradient algorithms are presented. In order to improve the parameter estimation accuracy and the convergence rate, the multi-innovation identification theory is employed to derive a forgetting factor multi-innovation stochastic gradient algorithm. A simulation example is provided to test the effectiveness of the proposed algorithms.     

Multi-innovation gradient estimation algorithms and convergence analysis for feedback nonlinear equation-error moving average systems

Considering the colored noises from the process environments, the parameter estimation problems for the feedback nonlinear equation-error systems interfered by moving average noises are addressed in this paper. Due to small computational burden, the gradient search principle is adopted to the feedback nonlinear systems and an overall extended stochastic gradient algorithm is derived for parameter estimation. Introducing the innovation length, the scalar innovation is expanded into the innovation vector and a multi-innovation extended stochastic gradient algorithm is further developed to reach the high estimation accuracy by utilizing more dynamical observed data. Furthermore, to assure the convergence of the proposed algorithms, their convergence properties are analyzed through the stochastic process theory. Finally, the experimental results indicate the effectiveness of the proposed algorithms.     

并行分布式遗传算法的研究

传统遗传算法在面对一些搜索空间巨大的复杂问题时,其表现往往难以令人满意。作者针对传统遗传算法解决高维多峰值问题时可能会出现的困难进行了分析,然后根据困难出现的原因,基于PVM设计了并行分布式遗传算法,并对适应度评估、交叉、变异算子做了一些改进,旨在加强算法的全局搜索能力,提高算法的收敛速度。为了验证算法多项措施的有效性,对一多峰函数在高维条件下进行多方面的测试,实验结果表明这几项措施是有效的。     

An accelerated exact distributed first-order algorithm for optimization over directed networks

Distributed optimization over networked agents has emerged as an advanced paradigm to address large-scale control, optimization, and signal-processing problems. In the last few years, the distributed first-order gradient methods have witnessed significant progress and enrichment due to the simplicity of using only the first derivatives of local functions. An exact first-order algorithm is developed in this work for distributed optimization over general directed networks with only row-stochastic weighted matrices. It employs the rescaling gradient method to address unbalanced information diffusion among agents, where the weights on the received information can be arbitrarily assigned. Moreover, uncoordinated step-sizes are employed to magnify the autonomy of agents, and an error compensation term and a heavy-ball momentum are incorporated to accelerate convergency. A linear convergence rate is rigorously proven for strongly-convex objective functions with Lipschitz continuous gradients. Explicit upper bounds of step-size and momentum parameter are provided. Finally, simulations illustrate the performance of the proposed algorithm.     

Aitken based modified Kalman filtering stochastic gradient algorithm for dual-rate nonlinear models

This paper develops an Aitken based modified Kalman filtering stochastic gradient algorithm for dual-rate nonlinear models. The Aitken based method can increase the convergence rate and the modified Kalman filter can improve the estimation accuracy. Thus compared to the traditional auxiliary model based stochastic gradient algorithm, the proposed algorithm in this paper is more effective, and this is proved by the convergence analysis. Furthermore, two simulated examples are given to illustrate the effectiveness of the proposed algorithm.     

An interpolation based on Cauchy's integral representation

In this paper, we present a new method of interpolation on equally spaced collocation points in a fixed interval on the x-axis. The proposed scheme is an iterative one which combines Taylor expansion and Cauchy's integral representation. The convergence rate of the method is investigated theoretically and verified numerically. Additionally, some numerical results are also shown to discuss the usefulness of the present method.     

Primal-dual stochastic distributed algorithm for constrained convex optimization

This paper investigates distributed convex optimization problems over an undirected and connected network, where each node’s variable lies in a private constrained convex set, and overall nodes aim at collectively minimizing the sum of all local objective functions. Motivated by a variety of applications in machine learning problems with large-scale training sets distributed to multiple autonomous nodes, each local objective function is further designed as the average of moderate number of local instantaneous functions. Each local objective function and constrained set cannot be shared with others. A primal-dual stochastic algorithm is presented to address the distributed convex optimization problems, where each node updates its state by resorting to unbiased stochastic averaging gradients and projects on its private constrained set. At each iteration, for each node the gradient of one local instantaneous function selected randomly is evaluated and the average of the most recent stochastic gradients is used to approximate the true local gradient. In the constrained case, we show that with strong-convexity of the local instantaneous function and Lipschitz continuity of its gradient, the algorithm converges to the global optimization solution almost surely. In the unconstrained case, an explicit linear convergence rate of the algorithm is provided. Numerical experiments are presented to demonstrate correctness of the theoretical results.     

A composite pseudospectral method for solving multi-delay optimal control problems involving piecewise constant delay functions

An adaptive numerical method for solving multi-delay optimal control problems with piecewise constant delay functions is introduced. The proposed method is based on composite pseudospectral method using the well-known Legendre–Gauss–Lobatto points. In this approach, the main problem converts to a mathematical optimization problem whose solution is much more easier than the original one. The necessary conditions of optimality associated to nonlinear piecewise constant delay systems are derived. The method is easy to implement and provides very accurate results.     

刻度指数族参数的经验Bayes检验的收敛速度

在“加权线性损失”下讨论刻度指数族中参数的经验Ｂａｙｅｓ（ＥＢ）检验问题．利用基于 Ｂｅｓｓｅｌ函数的核估计方法构造了ＥＢ检验函数．在适当的条件下证明了获得的ＥＢ检验函数是渐近最优的具有收敛速度Ｏ（ｎ－１ｌｎ６ ｎ）．最后给出一个满足定理条件的例子．     

Numerical estimation of two convective boundary conditions of a two-dimensional rectangular fin

A simultaneous estimation of two convective boundary conditions problem of a two-dimensional rectangular fin is proposed by numerical approach. The aim is to estimate the evolution of the distributions of the unknown heat transfer coefficients from the transient temperature histories taken with several sensors inside a two-dimensional fin. The estimation algorithm of this inverse heat conduction problem is based on the iterative regularization method and on the conjugate gradient method. An optimal choice of the vector of the descent parameters is used in this study and shows an increase in the convergence rate. The effects of some parameters (sensor number, position, measurement errors) on the inverse solutions are discussed.     

Optimal switching with minimum dwell time constraint

The problem of optimal switching between different subsystems/configurations with minimum dwell time constraints is investigated in this study and a feedback solution, using the framework of approximate dynamic programming, is proposed. The method calls for a tuning stage in which parameters of a function approximator are tuned. Online control will then be conducted with a computational load as low as evaluating a few scalar-valued functions. Finite-horizon and infinite-horizon cost functions are investigated and convergence and stability are analyzed considering the presence of approximation errors. Moreover, the case of applying a full or partial constraint on the mode sequence, as well as having a mode-dependent minimum dwell time are investigated. Finally, performance of the scheme in handling different challenges is numerically evaluated through different examples.     

Factor gradient iterative algorithm for solving a class of discrete periodic Sylvester matrix equations

At present, gradient iteration methods have been used to solve various Sylvester matrix equations and proved effective. Based on this method, we generalize the factor gradient iterative method (FGI) for solving forward periodic Sylvester matrix equations (FPSME) and backward periodic Sylvester matrix equations (BPSME). To accelerate the convergence of the iterative method, we refer to Gauss-Seidel and Jacobi iterative construction ideas and use the latest matrix information in the FGI iterative method to obtain the modified factor gradient iterative (MFGI) method. Then, the convergence of the proposed methods and the selection of optimal factors are proved. The last numerical examples illustrate the effectiveness and applicability of the iterative methods.     

Solution of delay fractional optimal control problems using a hybrid of block-pulse functions and orthonormal Taylor polynomials

This paper introduces an efficient direct approach for solving delay fractional optimal control problems. The concepts of the fractional integral and the fractional derivative are considered in the Riemann–Liouville sense and the Caputo sense, respectively. The suggested framework is based on a hybrid of block-pulse functions and orthonormal Taylor polynomials. The convergence of the proposed hybrid functions with respect to the L²-norm is demonstrated. The operational matrix of fractional integration associated with the hybrid functions is constructed by using the Laplace transform method. The problem under consideration is transformed into a mathematical programming one. The method of Lagrange multipliers is then implemented for solving the resulting optimization problem. The performance and computational efficiency of the developed numerical scheme are assessed through various types of delay fractional optimal control problems. Our numerical findings are compared with either exact solutions or the existing results in the literature.     

Smoothed amplitude flow-based phase retrieval algorithm

Phase retrieval recovers signals from linear phaseless measurements via minimizing a quadratic or amplitude function, while its loss function is generally either non-convex or non-smooth. Existing methods are used to add a truncation procedure or reweighting to the gradient during the gradient descent process to address the non-smooth problem. However, these methods often cause inconsistency in the search direction and increase the sampling complexity. This paperproposes a smoothed amplitude flow-based phase retrieval (SAFPR) algorithm to solve these problems. By introducing the smoothing function into the phase retrieval problem, the loss function is smoothed, significantly reducing the sampling complexity. Moreover, we also develop a stochastic smooth amplitude flow-based phase retrieval (SSAF) algorithm with practical, scalable, and fast in large-scale applications. Experimental results show that whether SAFPR or SSAF, the number of measurements required to reconstruct the signal entirely is better than the existing most advanced phase retrieval algorithms. The proposed methods also perform well in terms of time cost and convergence rate.     

Novel model reference adaptive control laws for improved transient dynamics and guaranteed saturation constraints

In classical model reference adaptive control (MRAC), the adaptive rates must be tuned to meet multiple competing objectives. Large adaptive rates guarantee rapid convergence of the trajectory tracking error to zero. However, large adaptive rates may also induce saturation of the actuators and excessive overshoots of the closed-loop system’s trajectory tracking error. Conversely, low adaptive rates may produce unsatisfactory trajectory tracking performances. To overcome these limitations, in the classical MRAC framework, the adaptive rates must be tuned through an iterative process. Alternative approaches require to modify the plant’s reference model or the reference command input. This paper presents the first MRAC laws for nonlinear dynamical systems affected by matched and parametric uncertainties that constrain both the closed-loop system’s trajectory tracking error and the control input at all times within user-defined bounds, and enforce a user-defined rate of convergence on the trajectory tracking error. By applying the proposed MRAC laws, the adaptive rates can be set arbitrarily large and both the plant’s reference model and the reference command input can be chosen arbitrarily. The user-defined rate of convergence of the closed-loop plant’s trajectory is enforced by introducing a user-defined auxiliary reference model, which converges to the trajectory tracking error obtained by applying the classical MRAC laws before its transient dynamics has decayed, and steering the trajectory tracking error to the auxiliary reference model at a rate of convergence that is higher than the rate of convergence of the plant’s reference model. The ability of the proposed MRAC laws to prescribe the performance of the closed-loop system’s trajectory tracking error and control input is guaranteed by barrier Lyapunov functions. Numerical simulations illustrate both the applicability of our theoretical results and their effectiveness compared to other techniques such as prescribed performance control, which allows to constrain both the rate of convergence and the maximum overshoot on the trajectory tracking error of uncertain systems.     

A least squares finite element method applied to b-splines

In this article we construct a class of functions on a bounded irregular region Ω?R² which are of compact support, smooth and locally polynomials. The basic tool is the use of ordinary B-splines associated with the square containing Ω. This construction is then used for approximating the solution of Poisson's boundary value problem. The approximation is carried out through a least squares finite element method applied to the above class. Aside from some computational experiments, the objective is to emphasize the ease of generating the basis elements and the role of Kernel function in a convergence proof.     

Conjugate gradient-based iterative algorithm for solving generalized periodic coupled Sylvester matrix equations

This paper focuses on constructing a conjugate gradient-based (CGB) method to solve the generalized periodic coupled Sylvester matrix equations in complex space. The presented method is developed from a point of conjugate gradient methods. It is proved that the presented method can find the solution of the considered matrix equations within finite iteration steps in the absence of round-off errors by theoretical derivation. Some numerical examples are provided to verify the convergence performance of the presented method, which is superior to some existing numerical algorithms both in iteration steps and computation time.     

中国区域环境效率的收敛性、空间溢出及成因分析

利用中国1998~2013年省际面板数据,基于SBM模型在测度区域环境效率的基础上,将空间面板收敛模型和偏微分效应分解方法相结合研究了区域环境效率的收敛性、空间溢出效应及其成因。结果表明,全国区域环境效率存在绝对收敛,且东、中、西部三大区域差异明显,东部地区不存在收敛,中、西部地区存在收敛;区域环境效率收敛存在较强的空间联动性和依赖性,且空间溢出效应较为明显,而在其影响因素中,城镇化率、外贸依存度、技术水平对本地区和邻近地区环境效率的收敛有正向空间溢出效应,环境投资空间溢出效应为负,产业结构的空间溢出效应则不显著。