Broad echo state network for multivariate time series prediction

In this paper, a broad echo state network with multiple reservoirs in parallel configuration (Broad-ESN) is proposed for a class of multivariate time series prediction. Firstly, through the unsupervised learning algorithm of restricted Boltzmann machine (RBM), the number of reservoirs of Broad-ESN can be determined, such that the dynamic characteristics of a class of multivariate time series can be fully reflected. Secondly, a parameter optimization method based on Davidon–Fletcher–Powell (DFP) quasi-Newton algorithm is proposed to optimize the reservoir parameters of Broad-ESN. Meanwhile, an output weights learning method based on output error is given to train the output weights of Broad-ESN. Thirdly, a sufficient condition for the echo state property of Broad-ESN is given. Finally, four examples are given to verify the effectiveness of Broad-ESN.     

变电所通信网络的优化

《电力系统光纤通信规程》中要求电力通信网的运行率为99.98%,而当前的通信网运行率只有97.55%,不符合上述要求。为了改变这种状况江苏扬州供电公司选择对原有的光传输网结构进行改造,利用新建的2个SDH相交环网将各变电所分别就近接入相应的局端通信站组成支环,经过组织实施,以较低的成本达到了预期目的。文章分析了改造方案、实施步骤和实际应用效果。     

Direct adaptive control for nonlinear systems using a TSK fuzzy echo state network based on fractional-order learning algorithm

This paper presents a new Takagi-Sugeno-Kang fuzzy Echo State Neural Network (TSKFESN) structure to design a direct adaptive control for uncertain SISO nonlinear systems. The proposed TSKFESN structure is based on the echo state neural network framework containing multiple sub-reservoirs. Each sub-reservoir is weighted with a TSK fuzzy rule. The adaptive law of the TSKFESN-based direct adaptive controller is derived by using a fractional-order sliding mode learning algorithm. Moreover, the Lyapunov stability criterion is employed to verify the convergence of the fractional-order adaptive law of the controller parameters. The evaluation of the proposed direct adaptive control scheme is verified using two case studies, the regulation problem of a torsional pendulum and the speed control of a direct current (DC) machine as a real-time application. The simulation and the experimental results show the effectiveness of the proposed control scheme.     

Detecting collusive spammers with heterogeneous graph attention network

Detecting collusive spammers who collaboratively post fake reviews is extremely important to guarantee the reliability of review information on e-commerce platforms. In this research, we formulate the collusive spammer detection as an anomaly detection problem and propose a novel detection approach based on heterogeneous graph attention network. First, we analyze the review dataset from different perspectives and use the statistical distribution to model each user's review behavior. By introducing the Bhattacharyya distance, we calculate the user-user and product-product correlation degrees to construct a multi-relation heterogeneous graph. Second, we combine the biased random walk strategy and multi-head self-attention mechanism to propose a model of heterogeneous graph attention network to learn the node embeddings from the multi-relation heterogeneous graph. Finally, we propose an improved community detection algorithm to acquire candidate spamming groups and employ an anomaly detection model based on the autoencoder to identify collusive spammers. Experiments show that the average improvements of precision@k and recall@k of the proposed approach over the best baseline method on the Amazon, Yelp_Miami, Yelp_New York, Yelp_San Francisco, and YelpChi datasets are [13%, 3%], [32%, 12%], [37%, 7%], [42%, 10%], and [18%, 1%], respectively.     

Multi-feature fused collaborative attention network for sequential recommendation with semantic-enriched contrastive learning

Recently, graph neural network (GNN) has been widely used in sequential recommendation because of its powerful ability to capture high-order collaborative relations, greatly promoting recommendation performance. However, some existing GNN-based methods fail to make full use of multiple relevant features of nodes and ignore the impact of semantic association between nodes on extracting user preferences. To this end, we propose a multi-feature fused collaborative attention network MASR, which sufficiently learns the temporal and positional features of nodes, and innovatively measures the importance of these two features for analyzing the nodes’ dynamic patterns. In addition, we incorporate semantic-enriched contrastive learning into collaborative filtering to enhance the semantic association between nodes and reduce the noise from the structural neighborhood, which has a positive effect on the sequential recommendation. Compared with the baseline models, the performance of MASR on MovieLens, CDs and Beauty datasets is improved by 2.0%, 2.1% and 1.7% respectively, proving its effectiveness in the sequential recommendation.     

Breakthrough recognition: Bias against novelty and competition for attention

Adding to the literature on the recognition and spread of ideas, and alongside the bias against novelty view documented in prior research, we introduce the perspective that articles compete for the attention of researchers who might build upon them. We investigate this effect by analyzing more than 5.3 million research publications from 1970 to 1999 in the life sciences. In support of our competition for attention perspective, we show that articles covering rarely addressed topics tend to receive more citations and have a higher chance of being breakthrough papers as compared to articles on more popular topics. We also explore conditions under which these effects might vary by using decade subsamples, home- versus foreign-field forward citations, as well as short-, medium- and long-term time windows. Finally, we also find evidence consistent with the previously documented bias against novelty and show that both mechanisms can work simultaneously.     

Data-driven fault monitoring for spacecraft control moment gyro with slice residual attention network

This paper studies the fault monitoring problem of a spacecraft control moment gyro (CMG) in complex environments based on the data-driven method. First, the wavelet denoising method and short-time Fourier transform (STFT) are utilized to preprocess the signal measured by an industrial personal computer (IPC) to obtain the frequency spectrum of each failure mode. Then, a slice residual attention network (SRAN) based on the ResNeXt model, attention mechanism, and random slice idea is proposed, which can fully capture the edge features of images while satisfying the learning efficiency. Furthermore, a set of comparative experiments are carried out to validate the ability of the proposed method, and the performance of SRAN is further verified under different datasets. Finally, based on the confusion matrix and t-SNE dimension reduction technique, the monitoring ability of SRAN for various faults is analyzed. Experimental results show that SRAN processes good fault monitoring capability and ideal robustness and can identify different fault degrees under the real-time fault monitoring scenario.     

VAD-assisted multitask transformer framework for emotion recognition and intensity prediction on suicide notes

Detecting suicidal tendencies and preventing suicides is an important social goal. The rise and continuance of emotion, the emotion category, and the intensity of the emotion are important clues about suicidal tendencies. The three determinants of emotion, viz. Valence, Arousal, and Dominance (VAD) can help determine a person’s exact emotion(s) and its intensity. This paper introduces an end-to-end VAD-assisted transformer-based multi-task network for detecting emotion (primary task) and its intensity (auxiliary task) in suicide notes. As part of this research, we expand the utility of the emotion-annotated benchmark dataset of suicide notes, CEASE-v2.0, by annotating all its sentences with emotion intensity labels. Empirical results show that our multi-task method performs better than the corresponding single-task systems, with the best attained overall Mean Recall (MR) of 65.25% on the emotion task. On a similar task, we improved MR by 8.78% over the existing state-of-the-art system. We evaluated our approach on three benchmark datasets for three different tasks. We observed that the introduced method consistently outperformed existing state-of-the-art approaches on the studied datasets, demonstrating its capacity to generalize to other downstream correlated tasks. We qualitatively examined our model’s output by comparing it to the labeling of a psychiatrist.     

Explicit time embedding based cascade attention network for information popularity prediction

Predicting information cascade popularity is a fundamental problem in social networks. Capturing temporal attributes and cascade role information (e.g., cascade graphs and cascade sequences) is necessary for understanding the information cascade. Current methods rarely focus on unifying this information for popularity predictions, which prevents them from effectively modeling the full properties of cascades to achieve satisfactory prediction performances. In this paper, we propose an explicit Time embedding based Cascade Attention Network (TCAN) as a novel popularity prediction architecture for large-scale information networks. TCAN integrates temporal attributes (i.e., periodicity, linearity, and non-linear scaling) into node features via a general time embedding approach (TE), and then employs a cascade graph attention encoder (CGAT) and a cascade sequence attention encoder (CSAT) to fully learn the representation of cascade graphs and cascade sequences. We use two real-world datasets (i.e., Weibo and APS) with tens of thousands of cascade samples to validate our methods. Experimental results show that TCAN obtains mean logarithm squared errors of 2.007 and 1.201 and running times of 1.76 h and 0.15 h on both datasets, respectively. Furthermore, TCAN outperforms other representative baselines by 10.4%, 3.8%, and 10.4% in terms of MSLE, MAE, and R-squared on average while maintaining good interpretability.     

FL-MGVN: Federated learning for anomaly detection using mixed gaussian variational self-encoding network

Anomalous data are such data that deviate from a large number of normal data points, which often have negative impacts on various systems. Current anomaly detection technology suffers from low detection accuracy, high false alarm rate and lack of labeled data. Anomaly detection is of great practical importance as an effective means to detect anomalies in the data and provide important support for the normal operation of various systems. In this paper, we propose an anomaly detection classification model that incorporates federated learning and mixed Gaussian variational self-encoding networks, namely MGVN. The proposed MGVN network model first constructs a variational self-encoder using a mixed Gaussian prior to extracting features from the input data, and then constructs a deep support vector network with the mixed Gaussian variational self-encoder to compress the feature space. The MGVN finds the minimum hypersphere to separate the normal and abnormal data and measures the abnormal fraction by calculating the Euclidean distance between the data features and the hypersphere center. Federated learning is finally incorporated with MGVN (FL-MGVN) to effectively address the problems that multiple participants collaboratively train a global model without sharing private data. The experiments are conducted on the benchmark datasets such as NSL-KDD, MNIST and Fashion-MNIST, which demonstrate that the proposed FL-MGVN has higher recognition performance and classification accuracy than other methods. The average AUC on MNIST and Fashion-MNIST reached 0.954 and 0.937, respectively.     

Echo state network with multiple delayed outputs for multiple delayed time series prediction

In this paper, considering the influence of multiple delayed output items on the prediction accuracy of echo state network, a novel echo state network with multiple delayed outputs (MDO-ESN) is proposed for time series prediction with multiple delayed outputs. Firstly, for a given learning task, through studying the autocorrelation of output signal, its delayed characteristics can be determined, and then the corresponding delayed item of output equation of the MDO-ESN can be adjusted adaptively. Secondly, in order to improve the adaptability of the MDO-ESN in different learning tasks, a sufficient condition is given to satisfy the stability of the MDO-ESN. Thirdly, a parameter optimization method is given to reduce the dependence of the prediction accuracy of the MDO-ESN on the reservoir parameters of the MDO-ESN. Finally, two numerical simulation examples and one actual simulation example are used for verifying the effectiveness of the MDO-ESN.     

Decentralized state estimation for the control of network systems

The paper proposes a decentralized state estimation method for the control of network systems, where a cooperative objective has to be achieved. The nodes of the network are partitioned into independent nodes, providing the control inputs, and dependent nodes, controlled by local interaction laws. The proposed state estimation algorithm allows the independent nodes to estimate the state of the dependent nodes in a completely decentralized way. To do that, it is necessary for each independent node of the network to estimate the control input components computed by the other independent nodes, without requiring communication among the independent nodes. The decentralized state estimator, including an input estimator, is developed and the convergence properties are studied. Simulation results show the effectiveness of the proposed approach.     

Optimal linear attack for multi-sensor network against state estimation

This paper is concerned with a security problem about malicious integrity attacks in state estimation system, in which multiple smart sensors locally measure information and transmit it to a remote fusion estimator though wireless channels. A joint constraint is considered for the attacker behaviour in each channel to keep stealthiness under a residual-based detector on the remote side. In order to degrade the estimator performance, the attacker will maximize the trace of the remote state estimation error covariance which is derived based on Kalman filter theory. It is proved that the optimal linear attack strategy design problem is convex and finally turned into a semi-definite programming problem. In addition, the tendency of attack behaviour on recursive and fixed Kalman filter system is analyzed. Several examples are given to illustrate the theoretical results.     

TCACNet: Temporal and channel attention convolutional network for motor imagery classification of EEG-based BCI

Brain–computer interface (BCI) is a promising intelligent healthcare technology to improve human living quality across the lifespan, which enables assistance of movement and communication, rehabilitation of exercise and nerves, monitoring sleep quality, fatigue and emotion. Most BCI systems are based on motor imagery electroencephalogram (MI-EEG) due to its advantages of sensory organs affection, operation at free will and etc. However, MI-EEG classification, a core problem in BCI systems, suffers from two critical challenges: the EEG signal’s temporal non-stationarity and the nonuniform information distribution over different electrode channels. To address these two challenges, this paper proposes TCACNet, a temporal and channel attention convolutional network for MI-EEG classification. TCACNet leverages a novel attention mechanism module and a well-designed network architecture to process the EEG signals. The former enables the TCACNet to pay more attention to signals of task-related time slices and electrode channels, supporting the latter to make accurate classification decisions. We compare the proposed TCACNet with other state-of-the-art deep learning baselines on two open source EEG datasets. Experimental results show that TCACNet achieves 11.4% and 7.9% classification accuracy improvement on two datasets respectively. Additionally, TCACNet achieves the same accuracy as other baselines with about 50% less training data. In terms of classification accuracy and data efficiency, the superiority of the TCACNet over advanced baselines demonstrates its practical value for BCI systems.     

Adaptive neural network control for nonstrict-feedback uncertain nonlinear systems with input delay and asymmetric time-varying state constraints

This paper is devoted to adaptive neural network control issue for a class of nonstrict-feedback uncertain systems with input delay and asymmetric time-varying state constraints. State-related external disturbances are involved into the system, and the upper bounds of disturbances are assumed as functions of state variables instead of constants. Additionally, during the approximations of unknown functions by neural networks, the online computation burdens are declined sharply, since the norms of neural network weight vectors are only estimated. In the process of dealing with input delay, an auxiliary function is applied such that the conditions for time delay are more general than the ones in existing literature. A novel adaptive neural network controller is designed by constructing the asymmetric barrier Lyapunov function, which guarantees that the output of system has a good tracking performance and the state variables never violate the asymmetric time-varying constraints. Finally, numerical simulations are presented to verify the proposed adaptive control scheme.     

Research of Chinese intangible cultural heritage knowledge graph construction and attribute value extraction with graph attention network

The development of digital technology promotes the construction of the Intangible cultural heritage (ICH) database but the data is still unorganized and not linked well, which makes the public hard to master the overall knowledge of the ICH. An ICH knowledge graph (KG) can help the public to understand the ICH and facilitate the protection of the ICH. However, a general framework of ICH KG construction is lacking now. In this study, we take the Chinese ICH (nation-level) as an example and propose a framework to build a Chinese ICH KG combining multiple data sources from Baike and the official website, which can extend the scale of the KG. Besides, the data of ICH grows daily, requiring us to design an efficient model to extract the knowledge from the data to update the KG in time. The built KG is based on the triple 〈entity, attribute, attribute value〉 and we introduce the attribute value extraction (AVE) task. However, the public Chinese ICH annotated AVE corpus is lacking. To solve that, we construct a Chinese ICH AVE corpus based on the Distant Supervision (DS) automatically rather than employing traditional manual annotation. Currently, AVE is usually seen as the sequence tagging task. In this paper, we take the ICH AVE as a node classification task and propose an AVE model BGC, combining the BiLSTM and graph attention network, which can fuse and utilize the word-level and character-level information by means of the ICH lexicon generated from the KG. We conduct extensive experiments and compare the proposed model with other state-of-the-art models. Experimental results show that the proposed model is of superiority.