Multi-view enhanced zero-shot node classification

In recent years, Zero-shot Node Classification (ZNC), an emerging and more difficult task is starting to attract attention, where the classes of testing nodes are unobserved in the training stage. Existing studies for ZNC mainly utilize Graph Neural Networks (GNNs) to construct the feature subspace to align with the classes’ semantic subspace, thus enabling knowledge transfer from seen classes to unseen classes. However, the modeling of the node feature is single-view and unilateral, e.g., the bag-of-words vector, which is not enough to fully describe the characteristics of the node itself. To address this dilemma, we propose to develop the Multi-View Enhanced zero-shot node classification paradigm (MVE) to promote the machine’s generality to approach the human-like thinking mode. Specifically, multi-view features are obtained from different aspects such as pre-trained model embeddings, knowledge graphs, statistic methods, and then fused by a contrastive learning module into the compositional node representation. Meanwhile, a developed Graph Convolutional Network (GCN) is used to make the nodes fully absorb the information of neighbors while the over-smooth issue is alleviated by multi-view features and the proposed contrastive learning mechanism. Experimental results conducted on three public datasets show an average 25% improvement compared to baseline methods, proving the superiority of our multi-view learning framework. The code and data can be found at https://github.com/guaiqihen/MVE.     

Towards a big data framework for analyzing social media content

Modern companies generate value by digitalizing their services and products. Knowing what customers are saying about the firm through reviews in social media content constitutes a key factor to succeed in the big data era. However, social media data analysis is a complex discipline due to the subjectivity in text review and the additional features in raw data. Some frameworks proposed in the existing literature involve many steps that thereby increase their complexity. A two-stage framework to tackle this problem is proposed: the first stage is focused on data preparation and finding an optimal machine learning model for this data; the second stage relies on established layers of big data architectures focused on getting an outcome of data by taking most of the machine learning model of stage one. Thus, a first stage is proposed to analyze big and small datasets in a non-big data environment, whereas the second stage analyzes big datasets by applying the first stage machine learning model of. Then, a study case is presented for the first stage of the framework to analyze reviews of hotel-related businesses. Several machine learning algorithms were trained for two, three and five classes, with the best results being found for binary classification.     

An end-to-end deep generative approach with meta-learning optimization for zero-shot object classification

Zero-shot object classification aims to recognize the object of unseen classes whose supervised data are unavailable in the training stage. Recent zero-shot learning (ZSL) methods usually propose to generate new supervised data for unseen classes by designing various deep generative networks. In this paper, we propose an end-to-end deep generative ZSL approach that trains the data generation module and object classification module jointly, rather than separately as in the majority of existing generation-based ZSL methods. Due to the ZSL assumption that unseen data are unavailable in the training stage, the distribution of generated unseen data will shift to the distribution of seen data, and subsequently causes the projection domain shift problem. Therefore, we further design a novel meta-learning optimization model to improve the proposed generation-based ZSL approach, where the parameters initialization and the parameters update algorithm are meta-learned to assist model convergence. We evaluate the proposed approach on five standard ZSL datasets. The average accuracy increased by the proposed jointly training strategy is 2.7% and 23.0% for the standard ZSL task and generalized ZSL task respectively, and the meta-learning optimization further improves the accuracy by 5.0% and 2.1% on two ZSL tasks respectively. Experimental results demonstrate that the proposed approach has significant superiority in various ZSL tasks.     

Beyond effective use: Integrating wise reasoning in machine learning development

The introduction of machine learning (ML), as the engine of many artificial intelligence (AI)-enabled systems in organizations, comes with the claim that ML models provide automated decisions or help domain experts improve their decision-making. Such a claim gives rise to the need to keep domain experts in the loop. Hence, data scientists, as those who develop ML models and infuse them with human intelligence during ML development, interact with various ML stakeholders and reflect their views within ML models. This interaction comes with (often conflicting) demands from various ML stakeholders and potential tensions. Building on the theories of effective use and wise reasoning, this mixed method study proposes a model to better understand how data scientists can use wisdom for managing these tensions when they develop ML models. In Study 1, through interviewing 41 analytics and ML experts, we investigate the dimensions of wise reasoning in the context of ML development. In Study 2, we test the overall model using a sample of 249 data scientists. Our results confirm that to develop effective ML models, data scientists need to not only use ML systems effectively, but also practice wise reasoning in their interactions with domain experts. We discuss the implications of these findings for research and practice.     

Autonomic machine learning platform

Acquiring information properly through machine learning requires familiarity with the available algorithms and understanding how they work and how to address the given problem in the best possible way. However, even for machine-learning experts in specific industrial fields, in order to predict and acquire information properly in different industrial fields, it is necessary to attempt several instances of trial and error to succeed with the application of machine learning. For non-experts, it is much more difficult to make accurate predictions through machine learning.In this paper, we propose an autonomic machine learning platform which provides the decision factors to be made during the developing of machine learning applications. In the proposed autonomic machine learning platform, machine learning processes are automated based on the specification of autonomic levels. This autonomic machine learning platform can be used to derive a high-quality learning result by minimizing experts’ interventions and reducing the number of design selections that require expert knowledge and intuition. We also demonstrate that the proposed autonomic machine learning platform is suitable for smart cities which typically require considerable amounts of security sensitive information.     

Question categorization and classification using grammar based approach

Question-answering has become one of the most popular information retrieval applications. Despite that most question-answering systems try to improve the user experience and the technology used in finding relevant results, many difficulties are still faced because of the continuous increase in the amount of web content. Questions Classification (QC) plays an important role in question-answering systems, with one of the major tasks in the enhancement of the classification process being the identification of questions types. A broad range of QC approaches has been proposed with the aim of helping to find a solution for the classification problems; most of these are approaches based on bag-of-words or dictionaries. In this research, we present an analysis of the different type of questions based on their grammatical structure. We identify different patterns and use machine learning algorithms to classify them. A framework is proposed for question classification using a grammar-based approach (GQCC) which exploits the structure of the questions. Our findings indicate that using syntactic categories related to different domain-specific types of Common Nouns, Numeral Numbers and Proper Nouns enable the machine learning algorithms to better differentiate between different question types. The paper presents a wide range of experiments the results show that the GQCC using J48 classifier has outperformed other classification methods with 90.1% accuracy.     

Knowledge-sparse and knowledge-rich learning in information retrieval

This paper reviews some aspects of the relationship between the large and growing fields of machine learning (ML) and information retrieval (IR). Learning programs are described along several dimensions. One dimension refers to the degree of dependence of an ML + IR program on users, thesauri, or documents. This paper emphasizes the role of the thesaurus in ML + IR work. ML + IR programs are also classified in a dimension that extends from knowledge-sparse learning at one end to knowledge-rich learning at the other. Knowledge-sparse learning depends largely on user yes-no feedback or on word frequencies across documents to guide adjustments in the IR system. Knowledge-rich learning depends on more complex sources of feedback, such as the structure within a document or thesaurus, to direct changes in the knowledge bases on which an intelligent IR system depends. New advances in computer hardware make the knowledge-sparse learning programs that depend on word occurrences in documents more practical. Advances in artificial intelligence bode well for knowledge-rich learning.     

一种基于机器学习的研究前沿识别方法研究

研究前沿是科技创新过程中最具潜力和前瞻性的研究方向,尽早识别研究前沿对科学研究、企业研发资源优化配置、政府创新战略前瞻部署等至关重要。针对目前在研究前沿识别研究中存在的不足,提出一种基于机器学习的研究前沿识别方法。该方法首先通过构建机器学习模型来识别出潜在高被引论文,解决利用引文分析法来识别研究前沿的时滞性问题,并将潜在高被引论文纳入研究前沿识别的高被引论文核心文档集中;其次,以高被引论文核心文档集为数据源,利用聚类分析法识别出研究前沿主题,并对研究前沿主题进行对比和评价分析,进而识别出研究前沿;最后,以太阳能光伏电池研究领域为例进行了实证研究,验证了该方法的可行性和有效性,为研究前沿识别提供了新的研究方法。     

Bayesian network based extreme learning machine for subjectivity detection

Subjectivity detection is a task of natural language processing that aims to remove ‘factual’ or ‘neutral’ content, i.e., objective text that does not contain any opinion, from online product reviews. Such a pre-processing step is crucial to increase the accuracy of sentiment analysis systems, as these are usually optimized for the binary classification task of distinguishing between positive and negative content. In this paper, we extend the extreme learning machine (ELM) paradigm to a novel framework that exploits the features of both Bayesian networks and fuzzy recurrent neural networks to perform subjectivity detection. In particular, Bayesian networks are used to build a network of connections among the hidden neurons of the conventional ELM configuration in order to capture dependencies in high-dimensional data. Next, a fuzzy recurrent neural network inherits the overall structure generated by the Bayesian networks to model temporal features in the predictor. Experimental results confirmed the ability of the proposed framework to deal with standard subjectivity detection problems and also proved its capacity to address portability across languages in translation tasks.     

基于机器学习算法的研究热点趋势预测模型对比与分析——BP神经网络、支持向量机与LSTM模型

[目的/意义]细粒度分析学科领域热点主题发展脉络并对利用机器学习算法对未来发展趋势进行准确预测研究。[方法/过程]提出一种基于机器学习算法的研究热点趋势预测方法与分析框架，以基因工程领域为例利用主题概率模型识别WOS核心集中论文摘要数据研究热点主题并进行主题演化关联构建，然后选取BP神经网络、支持向量机及LSTM模型等3种典型机器学习算法进行预测分析，最后利用RE指标和精准度指标评价机器学习算法预测效果并对基因工程领域在医药卫生、农业食品等方面研究趋势进行分析。[结果/结论]实验表明基于LSTM模型对热点主题未来发展趋势预测准确度最高，支持向量机预测效果次之，BP神经网络预测效果较差且预测稳定性不足，同时结合专家咨询和文献调研表明本文方法可快速识别基因领域研究主题及发展趋势，可为我国学科领域大势研判和架构调整提供决策支持和参考。     

A Contrastive learning-based Task Adaptation model for few-shot intent recognition

Few-shot intent recognition aims to identify user’s intent from the utterance with limited training data. A considerable number of existing methods mainly rely on the generic knowledge acquired on the base classes to identify the novel classes. Such methods typically ignore the characteristics of each meta task itself, resulting in the inability to make full use of limited given samples when classifying unseen classes. To deal with such issues, we propose a Contrastive learning-based Task Adaptation model (CTA) for few-shot intent recognition. In detail, we leverage contrastive learning to help achieve task adaptation and make full use of the limited samples of novel classes. First, a self-attention layer is employed in the task adaptation module, which aims to establish interactions between samples of different categories so that new representations are task-specific rather than relying entirely on the base classes. Then, the contrastive-based loss functions and the semantics of the label name are respectively used for reducing the similarity between sample representations in different categories while increasing it in the same categories. Experimental results on a public dataset OOS verify the effectiveness of our proposal by beating the competitive baselines in terms of accuracy. Besides, we conduct the cross-domain experiments on three datasets, i.e., OOS, SNIPS as well as ATIS. We find that CTA gains obvious improvements in terms of accuracy in all cross-domain experiments, indicating that it has a better generalization ability than other competitive baselines in both cross-domain and single-domain settings.     

Zero-shot stance detection via multi-perspective contrastive learning with unlabeled data

Stance detection is to distinguish whether the text’s author supports, opposes, or maintains a neutral stance towards a given target. In most real-world scenarios, stance detection needs to work in a zero-shot manner, i.e., predicting stances for unseen targets without labeled data. One critical challenge of zero-shot stance detection is the absence of contextual information on the targets. Current works mostly concentrate on introducing external knowledge to supplement information about targets, but the noisy schema-linking process hinders their performance in practice. To combat this issue, we argue that previous studies have ignored the extensive target-related information inhabited in the unlabeled data during the training phase, and propose a simple yet efficient Multi-Perspective Contrastive Learning Framework for zero-shot stance detection. Our framework is capable of leveraging information not only from labeled data but also from extensive unlabeled data. To this end, we design target-oriented contrastive learning and label-oriented contrastive learning to capture more comprehensive target representation and more distinguishable stance features. We conduct extensive experiments on three widely adopted datasets (from 4870 to 33,090 instances), namely SemEval-2016, WT-WT, and VAST. Our framework achieves 53.6%, 77.1%, and 72.4% macro-average F1 scores on these three datasets, showing 2.71% and 0.25% improvements over state-of-the-art baselines on the SemEval-2016 and WT-WT datasets and comparable results on the more challenging VAST dataset.     

A distance-based weighting framework for boosting the performance of dynamic ensemble selection

Dynamic Ensemble Selection (DES) strategy is one of the most common and effective techniques in machine learning to deal with classification problems. DES systems aim to construct an ensemble consisting of the most appropriate classifiers selected from the candidate classifier pool according to the competence level of the individual classifier. Since several classifiers are selected, their combination becomes crucial. However, most of current DES approaches focus on the combination of the selected classifiers while ignoring the local information surrounding the query sample needed to be classified. In order to boost the performance of DES-based classification systems, we in this paper propose a dynamic weighting framework for the classifier fusion during obtaining the final output of an DES system. In particular, the proposed method first employs a DES approach to obtain a group of classifiers for a query sample. Then, the hypothesis vector of the selected ensemble is obtained based on the analysis of consensus. Finally, a distance-based weighting scheme is developed to adjust the hypothesis vector depending on the closeness of the query sample to each class. The proposed method is tested on 30 real-world datasets with six well-known DES approaches based on both homogeneous and heterogeneous ensemble. The obtained results, supported by proper statistical tests, show that our method outperforms, both in terms of accuracy and kappa measures, the original DES framework.     

A machine learning approach to primacy-peak-recency effect-based satisfaction prediction

Data retrieval systems supporting the discovery and reuse of open data are emerging as important tools in the open data ecosystem. However, user satisfaction with them is relatively low. This study proposes the primacy-peak-recency effect to investigate the cognitive mechanisms underlying data searchers’ overall satisfaction. To test the primacy-peak-recency effect, primacy-peak-recency cubes consisting of eye movement indicators at primacy, peak, and recency moments and their combinations are constructed as the theoretical model. A user experiment was conducted to collect eye movement data and satisfaction scores generated during 48 doctoral students’ interactions with data retrieval systems. An ensemble machine learning framework was then applied to analyze eye movement data to assess the theoretical model. The results indicate that the primacy-peak-recency cubes are salient predictors of data searchers’ satisfaction (the prediction accuracy=0.682 and regression R2=0.749). This finding suggests that data searchers’ complex cognitive processes at primacy, peak, and recency moments measured by uni-, bi-, and three-dimensional eye movement indicators are predictors of overall satisfaction, confirming the primacy-peak-recency effect. In addition, combinations of varying types of influential moments and multidimensional eye movement events are the best predictors of overall satisfaction. This suggests that influential moments and cognitive processes have additive effects on overall satisfaction. Combining theory-driven and data-driven approaches, this study sheds light on the potential of machine learning approaches for analyzing neuropsychological data for heuristics examination. With these insights, practical strategies to predict data searchers’ satisfaction and optimize the user-experience design of data retrieval systems are proposed.     

SecureCPS: Cognitive inspired framework for detection of cyber attacks in cyber–physical systems

In the era of autonomous systems, the security is indispensable module for flexible computing environment. Due to increased computer power and network speed, a new computing paradigm, such as cognitive inspired computing, will emerge. Such a paradigm provides human-centered services that are convenient and enjoyable at any time, anywhere, and on any device. On the foundation of smart city environment, human computer interaction, intelligent services, and universal device connectivity, Cyber Physical Computing for Cyber Physical systems has recently been investigated. However, in this proposal, a cognitive inspired framework for securing CPS is scrutinized. The cognitive ability is conceded to the search engines by updating the PageRank ranking methodology. The proposed framework, named SecureCPS is trained with real time collective dataset for marking the relevancy of web page with the support the facial expressions. The eye regions are marked using Focal Point Detector algorithm. The framework is validated with machine learning models and resulted in achieving 98.51% accuracy and its outperforms the existing frameworks.     

Papers with code or without code? Impact of GitHub repository usability on the diffusion of machine learning research

Open Science initiatives prompt machine learning (ML) researchers and experts to share source codes - "scientific artifacts" - alongside research papers via public repositories such as GitHub. Here we analyze the extent to which 1) the availability of GitHub repositories influences paper citation and 2) the popularity trend of ML frameworks (e.g., PyTorch and TensorFlow) affects article citation rates. To accomplish this, we connect ML research publications indexed by Papers with Code (PwC) to Microsoft Academic Graph (MAG) and collect repository-level metadata using the GitHub API. Applying nearest-neighbor matching and econometric considerations, we estimate that papers enjoy approximately 20% advantages in monthly citation rates after the creation of the first GitHub repositories, accounting for paper-level fixed effects and ages. We also find that the temporal popularity trends for frameworks used in the first associated repositories could influence the monthly citation rate for papers. The results highlight the importance of technological artifacts and infrastructure latent to the diffusion of research.     

Text classification from unlabeled documents with bootstrapping and feature projection techniques

Many machine learning algorithms have been applied to text classification tasks. In the machine learning paradigm, a general inductive process automatically builds a text classifier by learning, generally known as supervised learning. However, the supervised learning approaches have some problems. The most notable problem is that they require a large number of labeled training documents for accurate learning. While unlabeled documents are easily collected and plentiful, labeled documents are difficultly generated because a labeling task must be done by human developers. In this paper, we propose a new text classification method based on unsupervised or semi-supervised learning. The proposed method launches text classification tasks with only unlabeled documents and the title word of each category for learning, and then it automatically learns text classifier by using bootstrapping and feature projection techniques. The results of experiments showed that the proposed method achieved reasonably useful performance compared to a supervised method. If the proposed method is used in a text classification task, building text classification systems will become significantly faster and less expensive.     

Information retrieval and machine learning for probabilistic schema matching

Schema matching is the problem of finding correspondences (mapping rules, e.g. logical formulae) between heterogeneous schemas e.g. in the data exchange domain, or for distributed IR in federated digital libraries. This paper introduces a probabilistic framework, called sPLMap, for automatically learning schema mapping rules, based on given instances of both schemas. Different techniques, mostly from the IR and machine learning fields, are combined for finding suitable mapping candidates. Our approach gives a probabilistic interpretation of the prediction weights of the candidates, selects the rule set with highest matching probability, and outputs probabilistic rules which are capable to deal with the intrinsic uncertainty of the mapping process. Our approach with different variants has been evaluated on several test sets.     

Utilizing statistical physics and machine learning to discover collective behavior on temporal social networks

Computational social science has become a branch of social science that uses computationally intensive ways to investigate and model social phenomena. Exploitation on mathematics, physics, and computer sciences, and analytic approaches like Social Network Analysis (SNA), Machine Learning (ML), etc, develops and tests the theories of complex social phenomena. In the emerging environment of social media, the new characteristics of social collective behavior and its extensive phenomena have become the hot spot of common concern across many disciplines. In this paper, we propose a general quantitative framework to discover the social collective behavior in temporal social networks. The general framework incorporates the Time-Correlation Function (T.C.F.) in statistical physics and evolutionary approach in Machine Learning, and provides the quantitative evidence of the existence of social collective behavior. Results show collective behaviors are observed and there exists a tiny fraction of users whose behavior are constantly replicated by public, disregard of the behavior itself. Our method is assumption-independent and has the potential to be applied to various temporal systems.