Backstepping-based adaptive dynamic programming for missile-target guidance systems with state and input constraints

In this paper, a novel backstepping-based adaptive dynamic programming (ADP) method is developed to solve the problem of intercepting a maneuver target in the presence of full-state and input constraints. To address state constraints, a barrier Lyapunov function is introduced to every backstepping procedure. An auxiliary design system is employed to compensate the input constraints. Then, an adaptive backstepping feedforward control strategy is designed, by which the tracking problem for strict-feedback systems can be reduced to an equivalence optimal regulation problem for affine nonlinear systems. Secondly, an adaptive optimal controller is developed by using ADP technique, in which a critic network is constructed to approximate the solution of the associated Hamilton–Jacobi–Bellman (HJB) equation. Therefore, the whole control scheme consists of an adaptive feedforward controller and an optimal feedback controller. By utilizing Lyapunov's direct method, all signals in the closed-loop system are guaranteed to be uniformly ultimately bounded (UUB). Finally, the effectiveness of the proposed strategy is demonstrated by using a simple nonlinear system and a nonlinear two-dimensional missile-target interception system.     

An improvement of single-network adaptive critic design for nonlinear systems with asymmetry constraints

Traditional approximate/adaptive dynamic programming (ADP) methods can handle a very special class of systems subject to symmetry constraints. In this study, I extend the exiting ADP to a broader class of nonlinear dynamic systems with asymmetry constraints. Firstly, I propose a novel nonquadratic cost function, based on which the developed optimal controller by solving Hamilton–Jacobi–Bellman equation can limit its value to arbitrarily prescribed bound. Then, to avoid “curse of dimensionality”, I approximately implement the addressed controller via single-network adaptive critic design. Fuzzy Hyperbolic Model is introduced to construct the single critic network by approximating optimal cost function, from which I further derive the optimal control law. The potential advantages are that the control structure is simple and the computational load is low. Lyapunov synthesis proves the ultimately uniformly bounded stability of closed-loop control system. Finally, numerical simulation results verify the efficiency and superiority of the proposed approach.     

Neural networks-based adaptive output feedback control for a class of uncertain nonlinear systems with input delay and disturbances

This paper focuses on the problem of adaptive output feedback control for a class of uncertain nonlinear systems with input delay and disturbances. Radial basis function neural networks (NNs) are employed to approximate the unknown functions and an NN observer is constructed to estimate the unmeasurable system states. Moreover, an auxiliary system is introduced to compensate for the effect of input delay. With the aid of the backstepping technique and Lyapunov stability theorem, an adaptive NN output feedback controller is designed which can guarantee the boundedness of all the signals in the closed-loop systems. Finally, a simulation example is given to illustrate the effectiveness of the proposed method.     

Event-triggered-based adaptive dynamic programming for distributed formation control of multi-UAV

This paper is concerned with the distributed formation control problem of multi-quadrotor unmanned aerial vehicle (UAV) in the framework of event triggering. First, for the position loop, an adaptive dynamic programming based on event triggering is developed to design the formation controller. The critic-only network structure is adopted to approximate the optimal cost function. The merit of the proposed algorithm lies in that the event triggering mechanism is incorporated the neural network (NN) to reduce calculations and actions of the multi-UAV system, which is significant for the practical application. What’s more, a new weight update law based on the gradient descent technology is proposed for the critic NN, which can ensure that the solution converges to the optimal value online. Then, a finite-time attitude tracking controller is adopted for the attitude loop to achieve rapid attitude tracking. Finally, the efficiency of the proposed method is illustrated by numerical simulations and experimental verification.     

Adaptive fault-tolerant attitude control for hypersonic reentry vehicle subject to complex uncertainties

In this paper, a novel fast attitude adaptive fault-tolerant control (FTC) scheme based on adaptive neural network and command filter is presented for the hypersonic reentry vehicles (HRV) with complex uncertainties which contain parameter uncertainties, un-modeled dynamics, actuator faults, and external disturbances. To improve the performance of closed-loop FTC, command filter and neural network are introduced to reconstruct system nonlinearities that are related to complex uncertainties. Compared with the FTC scheme with only neural network, the FTC scheme with command filter and neural network has fewer controller design parameters so that the computational complexity is decreased and the control efficiency is improved, which is of great significance for HRV. Then, the adaptive backstepping fault-tolerant controller based on command filter and neural network is designed, which can solve the complexity explosion problem in the standard backstepping control and the small uncertainty problem in the backstepping control only containing command filter. Moreover, to improve the approximation accuracy of the neural network-based universal approximator, an adaptive update law of neural network weights is designed by using the convex optimization technique. It is proved that the presented FTC scheme can ensure that the closed-loop control system is stable and the tracking errors are convergent. Finally, simulation results are carried out to verify the superiority and effectiveness of the presented FTC scheme.     

Disturbance observer-based robust missile autopilot design with full-state constraints via adaptive dynamic programming

This paper aims to develop a robust optimal control method for longitudinal dynamics of missile systems with full-state constraints suffering from mismatched disturbances by using adaptive dynamic programming (ADP) technique. First, the constrained states are mapped by smooth functions, thus, the considered systems become nonlinear systems without state constraints subject to unknown approximation error. In order to estimate the unknown disturbances, a nonlinear disturbance observer (NDO) is designed. Based on the output of disturbance observer, an integral sliding mode controller (ISMC) is derived to counteract the effects of disturbances and unknown approximation error, thus ensuring the stability of nonlinear systems. Subsequently, the ADP technique is utilized to learn an adaptive optimal controller for the nominal systems, in which a critic network is constructed with a novel weight update law. By utilizing the Lyapunov's method, the stability of the closed-loop system and the convergence of the estimation weight for critic network are guaranteed. Finally, the feasibility and effectiveness of the proposed controller are demonstrated by using longitudinal dynamics of a missile.     

Fraction dynamic-surface-based adaptive neural finite-time control for stochastic nonlinear systems subject to unknown control directions,time-varying input delay and state delay

In this paper, an adaptive neural finite-time tracking control is studied for a category of stochastic nonlinearly parameterized systems with multiple unknown control directions, time-varying input delay, and time-varying state delay. To this end, a novel criterion of semi-globally finite-time stability in probability (SGFSP) is proposed, in the sense of Lyapunov, for stochastic nonlinear systems with multiple unknown control directions. Secondly, a novel auxiliary system with finite-time convergence is presented to cope with the time-varying input delay, the appropriate Lyapunov Krasovskii functionals are utilized to compensate for the time-varying state delay, Nussbaum functions are exploited to identify multiple unknown control directions, and the neural networks (NNs) are applied to approximate the unknown functions of nonlinear parameters. Thirdly, the fraction dynamic surface control (FDSC) technique is embedded in the process of designing the controller, which not only the “explosion of complexity” problems are successfully avoided in traditional backstepping methods but also the command filter convergence can be obtained within a finite time to lead greatly improved for the response speed of command filter. Meanwhile, the error compensation mechanism is established to eliminate the errors of the command filter. Then, based on the proposed novel criterion, all closed-loop signals of the considered systems are SGPFS under the designed controller, and the tracking error can drive to a small neighborhood of the origin in a finite time. In the end, three simulation examples are applied to demonstrate the validity of the control method.     

Control of a time-varying hypersonic vehicle model subject to inlet un-start condition

A performance-guaranteed fault-tolerant controller is proposed for air-breathing hypersonic vehicles (AHVs) with synthetical consideration on time-varying uncertain parameters and inlet un-start condition. Different from most of the existing strategies for AHVs, the possible time-varying uncertain parameters in the control-oriented model are accommodated, which makes the resulting controller more adaptable to a real flight. Moreover, the velocity-dependent constraint on angle-of-attack, which is an important factor keeping the scramjet away from inlet un-start, is taken into account. With the utilization of barrier functions in adaptive design, velocity and altitude tracking errors are bounded by specific performance functions. Relevant analysis shows that the tracking performance can be further improved by proper adjustments on design parameters. The effectiveness of our proposed controller is evaluated through a simulation study.     

Data-driven adaptive optimal control of linear uncertain systems with unknown jumping dynamics

This paper focuses on the optimal control of a DC torque motor servo system which represents a class of continuous-time linear uncertain systems with unknown jumping internal dynamics. A data-driven adaptive optimal control strategy based on the integration of adaptive dynamic programming (ADP) and switching control is presented to minimize a predefined cost function. This takes the first step to develop switching ADP methods and extend the application of ADP to time-varying systems. Moreover, an analytical method to give the initial stabilizing controller for policy iteration ADP is proposed. It is shown that under the proposed adaptive optimal control law, the closed-loop switched system is asymptotically stable at the origin. The effectiveness of the strategy is validated via simulations on the DC motor system model.     

Disturbance-observer-based formation-containment control for UAVs via distributed adaptive event -triggered mechanisms

In this paper, the adaptive event-triggered formation-containment control for unmanned aerial vehicles (UAVs) is investigated in the presence of multiple leaders and external disturbances. By utilizing the leader-following model, the reference leader provides the desired flight trajectory for multiple formation leaders while the followers are driven into the convex hull spanned by the formation leaders. Initially, some effective disturbance observers are designed to obtain the estimations for eliminating the negative effects of external disturbances. Secondly, in order to alleviate the network burden, a dynamic triggering law is designed for the adaptive event-triggered mechanism (AETM) and the triggering frequency is heavily related to the triggering errors. Then, by exploiting Kronecker product technique and Lyapunov stability theory, two sufficient conditions on the stability of closed-loop system are established, which can help achieve the desired formation control target. Furthermore, the controller gains and observer ones can be determined by calculating the derived linear matrix inequalities (LMIs). Finally, a simulation example is given to illustrate the feasibility of the designed control protocol.     

Command filtered robust adaptive NN control for a class of uncertain strict-feedback nonlinear systems under input saturation

This paper develops a robust adaptive neural network (NN) tracking control scheme for a class of strict-feedback nonlinear systems with unknown nonlinearities and unknown external disturbances under input saturation. The radial basis function NNs with minimal learning parameter (MLP) are employed to online approximate the uncertain system dynamics. The adaptive laws are designed to online update the upper bound of the norm of ideal NN weight vectors, and the sum of the bounds of NN approximation errors and external disturbances, respectively. An auxiliary dynamic system is constructed to generate the augmented error signals which are used to modify the adaptive laws for preventing the destructive action due to the input saturation. Moreover, the command filtering backstepping control method is utilized to overcome the shortcoming of dynamic surface control method, the tracking-differentiator-based control method, etc. Our proposed scheme is qualified for simultaneously dealing with the input saturation effect, the heavy computational burden and the “explosion of complexity” problems. Theoretical analysis illuminates that our scheme ensures the boundedness of all signals in the closed-loop systems. Simulation results on two examples verify the effectiveness of our developed control scheme.     

Neural-network-based fault-tolerant control for nonlinear systems subjected to faults and saturations

This paper investigates a novel strategy which can address the fault-tolerant control (FTC) problem for nonlinear strict-feedback systems containing actuator saturation, unknown external disturbances, and faults related to actuators and components. In such method, the unknown dynamics including faults and disturbances are approximated by resorting to Neural-Networks (NNs) technique. Meanwhile, a back-stepping technique is employed to build a fault-tolerant controller. It should be stressed that the main advantage of this strategy is that the NN weights are updated online based on gradient descent (GD) algorithm by minimizing the cost function with respect to NNs approximation error rather than regarding weights as adaptive parameters, which are designed according to Lyapunov theory. In addition, the convergence proof of NN weights and the stability proof of the proposed FTC method are given. Finally, simulation is performed to demonstrate the effectiveness of the proposed strategy in dealing with unknown external disturbances, actuator saturation and the faults related to the components and actuators, simultaneously.     

Spacecraft output feedback attitude control based on extended state observer and adaptive dynamic programming

This paper investigates spacecraft output feedback attitude control problem based on extended state observer (ESO) and adaptive dynamic programming (ADP) approach. For the plant described by the unit quaternion, an ESO is first presented in view of the property of the attitude motion, and the norm constraint on the unit quaternion can be satisfied theoretically. The practical convergence proof of the developed ESO is illustrated by change of coordinates. Then, the controller is designed with an involvement of two parts: the basic part and the supplementary part. For the basic part, a proportional-derivative control law is designed. For the supplementary part, an ADP method called action-dependent heuristic dynamic programming (ADHDP) is adopted, which provides a supplementary control action according to the differences between the actual and the desired system signals. Simulation studies validate the effectiveness of the proposed scheme.     

Dynamic event-triggered optimal tracking control for constrained nonlinear stochastic systems

This paper investigates the optimal tracking control problem (OTCP) for nonlinear stochastic systems with input constraints under the dynamic event-triggered mechanism (DETM). Firstly, the OTCP is converted into the stabilizing optimization control problem by constructing a novel stochastic augmented system. The discounted performance index with nonquadratic utility function is formulated such that the input constraint can be encoded into the optimization problem. Then the adaptive dynamic programming (ADP) method of the critic-only architecture is employed to approximate the solutions of the OTCP. Unlike the conventional ADP methods based on time-driven mechanism or static event-triggered mechanism (SETM), the proposed adaptive control scheme integrates the DETM to further lighten the computing and communication loads. Furthermore, the uniform ultimately boundedness (UUB) of the critic weights and the tracking error are analysed with the Lyapunov theory. Finally, the simulation results are provided to validate the effectiveness of the proposed approach.     

Observer-based adaptive dynamic surface control of time-delayed non-strict systems with input nonlinearities

This paper proposes an adaptive dynamic surface controller for uncertain time-delay non-strict nonlinear systems with unknown control direction and unknown dead zone. To this end, the problem of uncertainty in nonlinear terms of the overall system is managed such that the estimation of these terms is obtained by applying a fuzzy logic, which is established based on an adaptive approach. A particular observer is then designed to approximate the immeasurable states. Furthermore, to overcome the delay issue in the system, the Lyapunov Krasovskii functional is used to achieve design conditions for dynamic surface control. Moreover, the breach of the output in the system is addressed by employing a Barrier Lyapunov Function. Then, with the aim of the designed controller, the stability of the closed-loop system is ensured such that all states are limited, and the errors are semi-globally uniformly ultimately bounded (SGUUB). Finally, as an illustration of the effectiveness of the proposed controller, a practical simulation is provided.     

Optimal adaptive reset control with guaranteed transient and steady state tracking error bounds

In this paper, a solution for improvement of transient performance in adaptive control of nonlinear systems is proposed. An optimal adaptive controller based on a reset mechanism and a prescribed performance bound is devised. The suggested controller has the structure of adaptive backstepping controller in which the estimated parameters are reset to an optimal value. The designed controller ensures both the transient bound and the asymptotical convergence of the states. It is shown that the tracking error satisfies the prescribed performance bound all the time, besides the speed of the convergence rate is increased by resetting the estimated parameters. The results have been proved through both the analytical and simulation studies. The proposed method is applied to an Augmented Quarter Car Model as a case study. Simulation results verify the established theoretical consequences that the prescribed performance bound based optimal adaptive reset controller can enhance the transient performance of the adaptive controller.     

Observer-Based adaptive neural inverse optimal consensus control of nonlinear multiagent systems

The objective of this article is to present an adaptive neural inverse optimal consensus tracking control for nonlinear multi-agent systems (MASs) with unmeasurable states. In the control process, firstly, to approximate the unknown state, a new observer is created which includes the outputs of other agents and their estimated information. The neural network is used to reckon the uncertain nonlinear dynamic systems. Based on a new inverse optimal method and the construction of tuning functions, an adaptive neural inverse optimal consensus tracking controller is proposed, which does not depend on the auxiliary system, thus greatly reducing the computational load. The developed scheme not only insures that all signals of the system are cooperatively semiglobally uniformly ultimately bounded (CSUUB), but also realizes optimal control of all signals. Eventually, two simulations provide the effectiveness of the proposed scheme.     

Decentralized adaptive neural control for interconnected stochastic nonlinear delay-time systems with asymmetric saturation actuators and output constraints

This paper investigates the problem of decentralized adaptive backstepping control for a class of large-scale stochastic nonlinear time-delay systems with asymmetric saturation actuators and output constraints. Firstly, the Gaussian error function is employed to represent a continuous differentiable asymmetric saturation nonlinearity, and barrier Lyapunov functions are designed to ensure that the output parameters are restricted. Secondly, the appropriate Lyapunov–Krasovskii functional and the property of hyperbolic tangent functions are used to deal with the unknown unmatched time-delay interactions, and the neural networks are employed to approximate the unknown nonlinearities. At last, based on Lyapunov stability theory, a decentralized adaptive neural control method is proposed, and the designed controller decreases the number of learning parameters. It is shown that the designed controller can ensure that all the closed-loop signals are 4-Moment (or 2 Moment) semi-globally uniformly ultimately bounded (SGUUB) and the tracking error converges to a small neighborhood of the origin. Two examples are provided to show the effectiveness of the proposed method.     

A yaw stability-guaranteed hierarchical coordination control strategy for four-wheel drive electric vehicles using an unscented Kalman filter

A novel hierarchical coordination control strategy (HCCS) is offered to guarantee the stability of four-wheel drive electric vehicles (4WD-EVs) combining the Unscented Kalman filter (UKF). First, a dynamics model of the 4WD-EVs is established, and the UKF-based estimator of sideslip angle and yaw rate is constructed concurrently. Second, the equivalent cornering stiffness coefficients are jointly estimated to consider the impact of vehicle uncertainty parameters on the estimator design. Afterwards, a HCCS with two-level controller is presented. The sideslip angle and yaw rate are controlled by an adaptive backstepping-based yaw moment controller, and the computational burden is relieved by an improved adaptive neural dynamic surface control technology in the upper-level controller. Simultaneously, the optimal torque distribution controller of hub motors is developed to optimize the adhesion utilization ratio of tire in the lower-level controller. Finally, the proposed HCCS has shown effective improvement in the trajectory tracking capability and yaw stability of the 4WD-EVs under various maneuver conditions compared with the traditional Luenberger observer-based and the federal-cubature Kalman filter-based hierarchical controller.     

Trajectory tracking control for rotary steerable systems using interval type-2 fuzzy logic and reinforcement learning

Rotary steerable system (RSS) is a directional drilling technique which has been applied in oil and gas exploration under complex environment for the requirements of fossil energy and geological prospecting. The nonlinearities and uncertainties which are caused by dynamical device, mechanical structure, extreme downhole environment and requirements of complex trajectory design in the actual drilling work increase the difficulties of accurate trajectory tracking. This paper proposes a model-based dual-loop feedback cooperative control method based on interval type-2 fuzzy logic control (IT2FLC) and actor-critic reinforcement learning (RL) algorithms with one-order digital low-pass filters (LPF) for three-dimensional trajectory tracking of RSS. In the proposed RSS trajectory tracking control architecture, an IT2FLC is utilized to deal with system nonlinearities and uncertainties, and an online iterative actor-critic RL controller structured by radial basis function neural networks (RBFNN) and adaptive dynamic programming (ADP) is exploited to eliminate the stick–slip oscillations relying on its approximate properties both in action function (actor) and value function (critic). The two control effects are fused to constitute cooperative controller to realize accurate trajectory tracking of RSS. The effectiveness of our controller is validated by simulations on designed function tests for angle building hole rate and complete downhole trajectory tracking, and by comparisons with other control methods.