Temporal group-aware graph diffusion networks for dynamic link prediction

Dynamic link prediction is a critical task in network research that seeks to predict future network links based on the relative behavior of prior network changes. However, most existing methods overlook mutual interactions between neighbors and long-distance interactions and lack the interpretability of the model’s predictions. To tackle the above issues, in this paper, we propose a temporal group-aware graph diffusion network(TGGDN). First, we construct a group affinity matrix to describe mutual interactions between neighbors, i.e., group interactions. Then, we merge the group affinity matrix into the graph diffusion to form a group-aware graph diffusion, which simultaneously captures group interactions and long-distance interactions in dynamic networks. Additionally, we present a transformer block that models the temporal information of dynamic networks using self-attention, allowing the TGGDN to pay greater attention to task-related snapshots while also providing interpretability to better understand the network evolutionary patterns. We compare the proposed TGGDN with state-of-the-art methods on five different sizes of real-world datasets ranging from 1k to 20k nodes. Experimental results show that TGGDN achieves an average improvement of 8.3% and 3.8% in terms of ACC and AUC on all datasets, respectively, demonstrating the superiority of TGGDN in the dynamic link prediction task.     

Improved sales time series predictions using deep neural networks with spatiotemporal dynamic pattern acquisition mechanism

The ability to predict product sales is invaluable for improving many of the routine decisions essential for the running of an enterprise. One significant challenge of sales prediction is that it is hard to dynamically capture changing dependent patterns along the sales time line, because sales are often influenced by complicated and changeable market environment. To address this issue, we model sales prediction as a task of multivariate time series (MTS) prediction, and propose a Spatiotemporal Dynamic Pattern Acquisition Mechanism (SDPA), which comprises four components, described below: (1) In the processing of input data: A Spatiotemporal Dynamic Kernel (SDK) component is designed for MTS to effectively capture different dependent correlation patterns during different time periods. (2) In terms of model design: A Simultaneous Regression (SR) component is proposed to dynamically detect stable correlations by using co-integration based dynamic programming over different time periods. (3) A novel Hierarchical Attention (HA) component is designed to incorporate SDK to detect spatiotemporal attention patterns from the captured dynamic correlations. (4) In the design of loss function, A Change Sensitive and Alignment component (DC) is proposed to provide more future information based on future trend correlations for better model training. The four components are incorporated into a unified framework by considering Homovariance Uncertainty (HU). This is referred to as SDPANet and contributes to model training and sales prediction. Extensive experiments were conducted on two real-world datasets: Galanz and Cainiao, and experimental results show that the proposed method achieves statistically significant improvements compared to the most state-of-the-art baselines, with average 41.5% reduction on RMAE, average 39.5% reduction on RRSE and average 46% improvement on CORR. Experiments are also conducted on two new datasets, which are Traffic and Exchange-Rate from other fields, to further verify the effectiveness of the proposed model. Case studies show that the model is capable of capturing dynamic changing patterns and of predicting future sales trends with greater accuracy.     

Spatio-temporal graph-based CNNs for anomaly detection in weakly-labeled videos

Abnormal event detection in videos plays an essential role for public security. However, most weakly supervised learning methods ignore the relationship between the complicated spatial correlations and the dynamical trends of temporal pattern in video data. In this paper, we provide a new perspective, i.e., spatial similarity and temporal consistency are adopted to construct Spatio-Temporal Graph-based CNNs (STGCNs). For the feature extraction, we use Inflated 3D (I3D) convolutional networks to extract features which can better capture appearance and motion dynamics in videos. For the spatio graph and temporal graph, each video segment is regarded as a vertex in the graph, and attention mechanism is introduced to allocate attention for each segment. For the spatial-temporal fusion graph, we propose a self-adapting weighting to fuse them. Finally, we build ranking loss and classification loss to improve the robustness of STGCNs. We evaluate the performance of STGCNs on UCF-Crime datasets (total 128 h) and ShanghaiTech datasets (total 317,398 frames) with the AUC score 84.2% and 92.3%, respectively. The experimental results also show the effectiveness and robustness with other evaluation metrics.     

Temporal burstiness and collaborative camouflage aware fraud detection

With the prosperity and development of the digital economy, many fraudsters have emerged on e-commerce platforms to fabricate fraudulent reviews to mislead consumers’ shopping decisions for profit. Moreover, in order to evade fraud detection, fraudsters continue to evolve and present the phenomenon of adversarial camouflage and collaborative attack. In this paper, we propose a novel temporal burstiness and collaborative camouflage aware method (TBCCA) for fraudster detection. Specifically, we capture the hidden temporal burstiness features behind camouflage strategy based on the time series prediction model, and identify highly suspicious target products by assigning suspicious scores as node priors. Meanwhile, a propagation graph integrating review collusion is constructed, and an iterative fraud confidence propagation algorithm is designed for inferring the label of nodes in the graph based on Loop Belief Propagation (LBP). Comprehensive experiments are conducted to compare TBCCA with state-of-the-art fraudster detection approaches, and experimental results show that TBCCA can effectively identify fraudsters in real review networks with achieving 6%–10% performance improvement than other baselines.     

Phishing scams detection via temporal graph attention network in Ethereum

Recently, phishing scams have become one of the most serious types of crime involved in Ethereum, the second-largest blockchain-based cryptocurrency platform. The existing phishing scams detection techniques for Ethereum mostly use traditional machine learning or network representation learning to mine the key information from the transaction network and identify phishing addresses. However, these methods typically crop the temporal transaction graph into snapshot sequences or construct temporal random wanderings to model the dynamic evolution of the topology of the transaction graph. In this paper, we propose PDTGA, a method that applies graph representation learning based on temporal graphs attention to improve the effectiveness of phishing scams detection in Ethereum. Specifically, we learn the functional representation of time directly and model the time signal through the interactions between the time encoding function and node features, edge features, and the topology of the graph. We collected a real-world Ethereum phishing scam dataset, containing over 250,000 transaction records between more than 100,000 account addresses, and divided them into three datasets of different sizes. Through data analysis, we first summarized the periodic pattern of Ethereum phishing scam activities. Then we constructed 14 kinds of account node features and 3 kinds of transaction edge features. Experimental evaluations based on the above three datasets demonstrate that PDTGA with 94.78% AUC score and 88.76% recall score outperforms the state-of-the-art methods.     

Explicit time embedding based cascade attention network for information popularity prediction

Predicting information cascade popularity is a fundamental problem in social networks. Capturing temporal attributes and cascade role information (e.g., cascade graphs and cascade sequences) is necessary for understanding the information cascade. Current methods rarely focus on unifying this information for popularity predictions, which prevents them from effectively modeling the full properties of cascades to achieve satisfactory prediction performances. In this paper, we propose an explicit Time embedding based Cascade Attention Network (TCAN) as a novel popularity prediction architecture for large-scale information networks. TCAN integrates temporal attributes (i.e., periodicity, linearity, and non-linear scaling) into node features via a general time embedding approach (TE), and then employs a cascade graph attention encoder (CGAT) and a cascade sequence attention encoder (CSAT) to fully learn the representation of cascade graphs and cascade sequences. We use two real-world datasets (i.e., Weibo and APS) with tens of thousands of cascade samples to validate our methods. Experimental results show that TCAN obtains mean logarithm squared errors of 2.007 and 1.201 and running times of 1.76 h and 0.15 h on both datasets, respectively. Furthermore, TCAN outperforms other representative baselines by 10.4%, 3.8%, and 10.4% in terms of MSLE, MAE, and R-squared on average while maintaining good interpretability.     

TCACNet: Temporal and channel attention convolutional network for motor imagery classification of EEG-based BCI

Brain–computer interface (BCI) is a promising intelligent healthcare technology to improve human living quality across the lifespan, which enables assistance of movement and communication, rehabilitation of exercise and nerves, monitoring sleep quality, fatigue and emotion. Most BCI systems are based on motor imagery electroencephalogram (MI-EEG) due to its advantages of sensory organs affection, operation at free will and etc. However, MI-EEG classification, a core problem in BCI systems, suffers from two critical challenges: the EEG signal’s temporal non-stationarity and the nonuniform information distribution over different electrode channels. To address these two challenges, this paper proposes TCACNet, a temporal and channel attention convolutional network for MI-EEG classification. TCACNet leverages a novel attention mechanism module and a well-designed network architecture to process the EEG signals. The former enables the TCACNet to pay more attention to signals of task-related time slices and electrode channels, supporting the latter to make accurate classification decisions. We compare the proposed TCACNet with other state-of-the-art deep learning baselines on two open source EEG datasets. Experimental results show that TCACNet achieves 11.4% and 7.9% classification accuracy improvement on two datasets respectively. Additionally, TCACNet achieves the same accuracy as other baselines with about 50% less training data. In terms of classification accuracy and data efficiency, the superiority of the TCACNet over advanced baselines demonstrates its practical value for BCI systems.     

Personalized Recommendation via Multi-dimensional Meta-paths Temporal Graph Probabilistic Spreading

Since meta-paths have the innate ability to capture rich structure and semantic information, meta-path-based recommendations have gained tremendous attention in recent years. However, how to composite these multi-dimensional meta-paths? How to characterize their dynamic characteristics? How to automatically learn their priority and importance to capture users' diverse and personalized preferences at the user-level granularity? These issues are pivotal yet challenging for improving both the performance and the interpretability of recommendations. To address these challenges, we propose a personalized recommendation method via Multi-Dimensional Meta-Paths Temporal Graph Probabilistic Spreading (MD-MP-TGPS). Specifically, we first construct temporal multi-dimensional graphs with full consideration of the interest drift of users, obsolescence and popularity of items, and dynamic update of interaction behavior data. Then we propose a dimension-free temporal graph probabilistic spreading framework via multi-dimensional meta-paths. Moreover, to automatically learn the priority and importance of these multi-dimensional meta-paths at the user-level granularity, we propose two boosting strategies for personalized recommendation. Finally, we conduct comprehensive experiments on two real-world datasets and the experimental results show that the proposed MD-MP-TGPS method outperforms the compared state-of-the-art methods in such performance indicators as precision, recall, F1-score, hamming distance, intra-list diversity and popularity in terms of accuracy, diversity, and novelty.     

Listening to the investors: A novel framework for online lending default prediction using deep learning neural networks

Online peer-to-peer (P2P) lending has developed dramatically over the last decade in China. But this rapid boom carries potential risks. Investors have incurred incalculable losses due to the recent increase in fraudulent and/or unreliable online P2P platforms. Hence, predicting and identifying potential default risk platforms is crucial at this juncture. To achieve this end, we propose a two-step method which employs a deep learning neural network to extract keywords from investor comments and then utilizes a bidirectional long short-term memory (BiLSTM) based model to predict the default risk of platforms. Experimental results on real-world datasets of about 1000 platforms show that in the keyword extraction phase, our model can better capture semantic features from highly colloquial comment-text and achieve significant improvement over other baselines. Additionally, in the default platform prediction stage, our model achieves an F1 value of 80.34% in identifying potential problem platforms, outperforming four baselines by 23.37%, 5.71%, 8.93%, and 4.98% of improvement and comprehensively verifying the effectiveness of our method. Our study provides an alternative solution for platform default risk prediction issues and validates the effectiveness of investor comments in revealing the risk situation of online lending platforms.     

MTGCN: A multi-task approach for node classification and link prediction in graph data

Both node classification and link prediction are popular topics of supervised learning on the graph data, but previous works seldom integrate them together to capture their complementary information. In this paper, we propose a Multi-Task and Multi-Graph Convolutional Network (MTGCN) to jointly conduct node classification and link prediction in a unified framework. Specifically, MTGCN consists of multiple multi-task learning so that each multi-task learning learns the complementary information between node classification and link prediction. In particular, each multi-task learning uses different inputs to output representations of the graph data. Moreover, the parameters of one multi-task learning initialize the parameters of the other multi-task learning, so that the useful information in the former multi-task learning can be propagated to the other multi-task learning. As a result, the information is augmented to guarantee the quality of representations by exploring the complex constructure inherent in the graph data. Experimental results on six datasets show that our MTGCN outperforms the comparison methods in terms of both node classification and link prediction.     

Where have you been: Dual spatiotemporal-aware user mobility modeling for missing check-in POI identification

The prevalence of Location-based Social Networks (LBSNs) services makes next personalized Point-of-Interest (POI) predictions become a trending research topic. However, due to device failure or intention camouflage, geolocation information missing prevents existing POI-oriented researches for advanced user preference analysis. To this end, we propose a novel model named Bi-STAN, which fuses bi-direction spatiotemporal transition patterns and personalized dynamic preferences, to identify where the user has visited at a past specific time, namely missing check-in POI identification. Furthermore, to relieve data sparsity issues, Bi-STAN explicitly exploits spatiotemporal characteristics by doing bilateral traceback to search related items with high predictive power from user mobility traces. Specifically, Bi-STAN introduces (1) a temporal-aware attention semantic category encoder to unveil the latent semantic category transition patterns by modeling temporal periodicity and attenuation; (2) a spatial-aware attention POI encoder to capture the latent POI transition pattern by modeling spatial regularity and proximity; (3) a multitask-oriented decoder to incorporate personalized and temporal variance preference into learned transition patterns for missing check-in POI and category identification. Based on the complementarity and compatibility of multi-task learning, we further develop Bi-STAN with a self-adaptive learning rate for model optimization. Experimental results on two real-world datasets show the effectiveness of our proposed method. Significantly, Bi-STAN can also be adaptively applied to next POI prediction task with outstanding performances.     

Knowledge graph representation learning with simplifying hierarchical feature propagation

Graph neural networks (GNN) have emerged as a new state-of-the-art for learning knowledge graph representations. Although they have shown impressive performance in recent studies, how to efficiently and effectively aggregate neighboring features is not well designed. To tackle this challenge, we propose the simplifying heterogeneous graph neural network (SHGNet), a generic framework that discards the two standard operations in GNN, including the transformation matrix and nonlinear activation. SHGNet, in particular, adopts only the essential component of neighborhood aggregation in GNN and incorporates relation features into feature propagation. Furthermore, to capture complex structures, SHGNet utilizes a hierarchical aggregation architecture, including node aggregation and relation weighting. Thus, the proposed model can treat each relation differently and selectively aggregate informative features. SHGNet has been evaluated for link prediction tasks on three real-world benchmark datasets. The experimental results show that SHGNet significantly promotes efficiency while maintaining superior performance, outperforming all the existing models in 3 out of 4 metrics on NELL-995 and in 4 out of 4 metrics on FB15k-237 dataset.     

Causality-based CTR prediction using graph neural networks

As a prevalent problem in online advertising, CTR prediction has attracted plentiful attention from both academia and industry. Recent studies have been reported to establish CTR prediction models in the graph neural networks (GNNs) framework. However, most of GNNs-based models handle feature interactions in a complete graph, while ignoring causal relationships among features, which results in a huge drop in the performance on out-of-distribution data. This paper is dedicated to developing a causality-based CTR prediction model in the GNNs framework (Causal-GNN) integrating representations of feature graph, user graph and ad graph in the context of online advertising. In our model, a structured representation learning method (GraphFwFM) is designed to capture high-order representations on feature graph based on causal discovery among field features in gated graph neural networks (GGNNs), and GraphSAGE is employed to obtain graph representations of users and ads. Experiments conducted on three public datasets demonstrate the superiority of Causal-GNN in AUC and Logloss and the effectiveness of GraphFwFM in capturing high-order representations on causal feature graph.     

Unifying knowledge iterative dissemination and relational reconstruction network for image–text matching

Image–text matching is a crucial branch in multimedia retrieval which relies on learning inter-modal correspondences. Most existing methods focus on global or local correspondence and fail to explore fine-grained global–local alignment. Moreover, the issue of how to infer more accurate similarity scores remains unresolved. In this study, we propose a novel unifying knowledge iterative dissemination and relational reconstruction (KIDRR) network for image–text matching. Particularly, the knowledge graph iterative dissemination module is designed to iteratively broadcast global semantic knowledge, enabling relevant nodes to be associated, resulting in fine-grained intra-modal correlations and features. Hence, vector-based similarity representations are learned from multiple perspectives to model multi-level alignments comprehensively. The relation graph reconstruction module is further developed to enhance cross-modal correspondences by constructing similarity relation graphs and adaptively reconstructing them. We conducted experiments on the datasets Flickr30K and MSCOCO, which have 31,783 and 123,287 images, respectively. Experiments show that KIDRR achieves improvements of nearly 2.2% and 1.6% relative to Recall@1 on Flicr30K and MSCOCO, respectively, compared to the current state-of-the-art baselines.     

Multi-kernel graph fusion for spectral clustering

Many methods of multi-kernel clustering have a bias to power base kernels by ignoring other kernels. To address this issue, in this paper, we propose a new method of multi-kernel graph fusion based on min–max optimization (namely MKGF-MM) for spectral clustering by making full use of all base kernels. Specifically, the proposed method investigates a novel min–max weight strategy to capture the complementary information among all base kernels. As a result, every base kernel contributes to the construction of the fusion graph from all base kernels so that the quality of the fusion graph is guaranteed. In addition, we design an iterative optimization method to solve the proposed objective function. Furthermore, we theoretically prove that our optimization method achieves convergence. Experimental results on real medical datasets and scientific datasets demonstrate that the proposed method outperforms all comparison methods and the proposed optimization method achieves fast convergence.     

Region-action LSTM for mouse interaction sequence based search satisfaction evaluation

Mouse interaction data contain a lot of interaction information between users and Search Engine Result Pages (SERPs), which can be useful for evaluating search satisfaction. Existing studies use aggregated features or anchor elements to capture the spatial information in mouse interaction data, which might lose valuable mouse cursor movement patterns for estimating search satisfaction. In this paper, we leverage regions together with actions to extract sequences from mouse interaction data. Using regions to capture the spatial information in mouse interaction data would reserve more details of the interaction processes between users and SERPs. To modeling mouse interaction sequences for search satisfaction evaluation, we propose a novel LSTM unit called Region-Action LSTM (RALSTM), which could capture the interactive relations between regions and actions without subjecting the network to higher training complexity. Simultaneously, we propose a data augmentation strategy Multi-Factor Perturbation (MFP) to increase the pattern variations on mouse interaction sequences. We evaluate the proposed approach on open datasets. The experimental results show that the proposed approach achieves significant performance improvement compared with the state-of-the-art search satisfaction evaluation approach.     

MGAT: Multimodal Graph Attention Network for Recommendation

Graph neural networks (GNNs) have shown great potential for personalized recommendation. At the core is to reorganize interaction data as a user-item bipartite graph and exploit high-order connectivity among user and item nodes to enrich their representations. While achieving great success, most existing works consider interaction graph based only on ID information, foregoing item contents from multiple modalities (e.g., visual, acoustic, and textual features of micro-video items). Distinguishing personal interests on different modalities at a granular level was not explored until recently proposed MMGCN (Wei et al., 2019). However, it simply employs GNNs on parallel interaction graphs and treats information propagated from all neighbors equally, failing to capture user preference adaptively. Hence, the obtained representations might preserve redundant, even noisy information, leading to non-robustness and suboptimal performance. In this work, we aim to investigate how to adopt GNNs on multimodal interaction graphs, to adaptively capture user preference on different modalities and offer in-depth analysis on why an item is suitable to a user. Towards this end, we propose a new Multimodal Graph Attention Network, short for MGAT, which disentangles personal interests at the granularity of modality. In particular, built upon multimodal interaction graphs, MGAT conducts information propagation within individual graphs, while leveraging the gated attention mechanism to identify varying importance scores of different modalities to user preference. As such, it is able to capture more complex interaction patterns hidden in user behaviors and provide a more accurate recommendation. Empirical results on two micro-video recommendation datasets, Tiktok and MovieLens, show that MGAT exhibits substantial improvements over the state-of-the-art baselines like NGCF (Wang, He, et al., 2019) and MMGCN (Wei et al., 2019). Further analysis on a case study illustrates how MGAT generates attentive information flow over multimodal interaction graphs.     

Multi-feature fused collaborative attention network for sequential recommendation with semantic-enriched contrastive learning

Recently, graph neural network (GNN) has been widely used in sequential recommendation because of its powerful ability to capture high-order collaborative relations, greatly promoting recommendation performance. However, some existing GNN-based methods fail to make full use of multiple relevant features of nodes and ignore the impact of semantic association between nodes on extracting user preferences. To this end, we propose a multi-feature fused collaborative attention network MASR, which sufficiently learns the temporal and positional features of nodes, and innovatively measures the importance of these two features for analyzing the nodes’ dynamic patterns. In addition, we incorporate semantic-enriched contrastive learning into collaborative filtering to enhance the semantic association between nodes and reduce the noise from the structural neighborhood, which has a positive effect on the sequential recommendation. Compared with the baseline models, the performance of MASR on MovieLens, CDs and Beauty datasets is improved by 2.0%, 2.1% and 1.7% respectively, proving its effectiveness in the sequential recommendation.     

Exploiting context-awareness and multi-criteria decision making to improve items recommendation using a tripartite graph-based model

Integrating useful input information is essential to provide efficient recommendations to users. In this work, we focus on improving items ratings prediction by merging both multiple contexts and multiple criteria based research directions which were addressed separately in most existent literature. Throughout this article, Criteria refer to the items attributes, while Context denotes the circumstances in which the user uses an item. Our goal is to capture more fine grained preferences to improve items recommendation quality using users’ multiple criteria ratings under specific contextual situations. Therefore, we examine the recommenders’ data from the graph theory based perspective by representing three types of entities (users, contextual situations and criteria) as well as their relationships as a tripartite graph. Upon the assumption that contextually similar users tend to have similar interests for similar item criteria, we perform a high-order co-clustering on the tripartite graph for simultaneously partitioning the graph entities representing users in similar contextual situations and their evaluated item criteria. To predict cluster-based multi-criteria ratings, we introduce an improved rating prediction method that considers the dependency between users and their contextual situations, and also takes into account the correlation between criteria in the prediction process. The predicted multi-criteria ratings are finally aggregated into a single representative output corresponding to an overall item rating. To guide our investigation, we create a research hypothesis to provide insights about the tripartite graph partitioning and design clear and justified preliminary experiments including quantitative and qualitative analyzes to validate it. Further thorough experiments on the two available context-aware multi-criteria datasets, TripAdvisor and Educational, demonstrate that our proposal exhibits substantial improvements over alternative recommendations approaches.     

Heterogeneous graph-based joint representation learning for users and POIs in location-based social network

Learning latent representations for users and points of interests (POIs) is an important task in location-based social networks (LBSN), which could largely benefit multiple location-based services, such as POI recommendation and social link prediction. Many contextual factors, like geographical influence, user social relationship and temporal information, are available in LBSN and would be useful for this task. However, incorporating all these contextual factors for user and POI representation learning in LBSN remains challenging, due to their heterogeneous nature. Although the encouraging performance of POI recommendation and social link prediction are delivered, most of the existing representation learning methods for LBSN incorporate only one or two of these contextual factors. In this paper, we propose a novel joint representation learning framework for users and POIs in LBSN, named UP2VEC. In UP2VEC, we present a heterogeneous LBSN graph to incorporate all these aforementioned factors. Specifically, the transition probabilities between nodes inside the heterogeneous graph are derived by jointly considering these contextual factors. The latent representations of users and POIs are then learnt by matching the topological structure of the heterogeneous graph. For evaluating the effectiveness of UP2VEC, a series of experiments are conducted with two real-world datasets (Foursquare and Gowalla) in terms of POI recommendation and social link prediction. Experimental results demonstrate that the proposed UP2VEC significantly outperforms the existing state-of-the-art alternatives. Further experiment shows the superiority of UP2VEC in handling cold-start problem for POI recommendation.