Machine learning classification of entrepreneurs in British historical census data

This paper presents a binary classification of entrepreneurs in British historical data based on the recent availability of big data from the I-CeM dataset. The main task of the paper is to attribute an employment status to individuals that did not fully report entrepreneur status in earlier censuses (1851–1881). The paper assesses the accuracy of different classifiers and machine learning algorithms, including Deep Learning, for this classification problem. We first adopt a ground-truth dataset from the later censuses to train the computer with a Logistic Regression (which is standard in the literature for this kind of binary classification) to recognize entrepreneurs distinct from non-entrepreneurs (i.e. workers). Our initial accuracy for this base-line method is 0.74. We compare the Logistic Regression with ten optimized machine learning algorithms: Nearest Neighbors, Linear and Radial Support Vector Machine, Gaussian Process, Decision Tree, Random Forest, Neural Network, AdaBoost, Naive Bayes, and Quadratic Discriminant Analysis. The best results are boosting and ensemble methods. AdaBoost achieves an accuracy of 0.95. Deep-Learning, as a standalone category of algorithms, further improves accuracy to 0.96 without using the rich text-data that characterizes the OccString feature, a string of up to 500 characters with the full occupational statement of each individual collected in the earlier censuses. Finally, and now using this OccString feature, we implement both shallow (bag-of-words algorithm) learning and Deep Learning (Recurrent Neural Network with a Long Short-Term Memory layer) algorithms. These methods all achieve accuracies above 0.99 with Deep Learning Recurrent Neural Network as the best model with an accuracy of 0.9978. The results show that standard algorithms for classification can be outperformed by machine learning algorithms. This confirms the value of extending the techniques traditionally used in the literature for this type of classification problem.     

Text classification from unlabeled documents with bootstrapping and feature projection techniques

Many machine learning algorithms have been applied to text classification tasks. In the machine learning paradigm, a general inductive process automatically builds a text classifier by learning, generally known as supervised learning. However, the supervised learning approaches have some problems. The most notable problem is that they require a large number of labeled training documents for accurate learning. While unlabeled documents are easily collected and plentiful, labeled documents are difficultly generated because a labeling task must be done by human developers. In this paper, we propose a new text classification method based on unsupervised or semi-supervised learning. The proposed method launches text classification tasks with only unlabeled documents and the title word of each category for learning, and then it automatically learns text classifier by using bootstrapping and feature projection techniques. The results of experiments showed that the proposed method achieved reasonably useful performance compared to a supervised method. If the proposed method is used in a text classification task, building text classification systems will become significantly faster and less expensive.     

Advanced learning algorithms for cross-language patent retrieval and classification

We study several machine learning algorithms for cross-language patent retrieval and classification. In comparison with most of other studies involving machine learning for cross-language information retrieval, which basically used learning techniques for monolingual sub-tasks, our learning algorithms exploit the bilingual training documents and learn a semantic representation from them. We study Japanese–English cross-language patent retrieval using Kernel Canonical Correlation Analysis (KCCA), a method of correlating linear relationships between two variables in kernel defined feature spaces. The results are quite encouraging and are significantly better than those obtained by other state of the art methods. We also investigate learning algorithms for cross-language document classification. The learning algorithm are based on KCCA and Support Vector Machines (SVM). In particular, we study two ways of combining the KCCA and SVM and found that one particular combination called SVM_2k achieved better results than other learning algorithms for either bilingual or monolingual test documents.     

Adaptive vibration control of a flexible structure based on hybrid learning controlled active mass damping

Natural disasters such as earthquakes and strong winds will lead to vibrations in ultra-high or high-rise buildings and even the damages of the structures. The traditional approaches resist the destructive effects of natural disasters through enhancing the performance of the structure itself. However, due to the unpredictability of the disaster strength, the traditional approaches are no longer appropriate for earthquake mitigation in building structures. Therefore, designing an effective intelligent control method for suppressing vibrations of the flexible buildings is significant in practice. This paper focuses on a single-floor building-like structure with an active mass damper (AMD) and proposes a hybrid learning control strategy to suppress vibrations caused by unknown time-varying disturbances (earthquake, strong wind, etc.). As the flexible building structure is a distributed parameter system, a novel finite dimension dynamic model is firstly constructed by assumed mode method (AMM) to effectively analyze the complex dynamics of the flexible building stucture. Secondly, an adaptive hybrid learning control based on full-order state observer is designed through back-stepping method for dealing with system uncertainties, unknown disturbances and immeasurable states. Thirdly, semi-globally uniformly ultimate boundedness (SGUUB) of the closed-loop system is guaranteed via Lyapunov’s stability theory. Finally, the experimental investigation on Quanser Active Mass Damper demonstrates the effectiveness of the presented control approach in the field of vibration suppression. The research results will bring new ideas and methods to the field of disaster reduction for the engineering development.     

An interactive human centered data science approach towards crime pattern analysis

The traditional machine learning systems lack a pathway for a human to integrate their domain knowledge into the underlying machine learning algorithms. The utilization of such systems, for domains where decisions can have serious consequences (e.g. medical decision-making and crime analysis), requires the incorporation of human experts' domain knowledge. The challenge, however, is how to effectively incorporate domain expert knowledge with machine learning algorithms to develop effective models for better decision making.In crime analysis, the key challenge is to identify plausible linkages in unstructured crime reports for the hypothesis formulation. Crime analysts painstakingly perform time-consuming searches of many different structured and unstructured databases to collate these associations without any proper visualization. To tackle these challenges and aiming towards facilitating the crime analysis, in this paper, we examine unstructured crime reports through text mining to extract plausible associations. Specifically, we present associative questioning based searching model to elicit multi-level associations among crime entities. We coupled this model with partition clustering to develop an interactive, human-assisted knowledge discovery and data mining scheme.The proposed human-centered knowledge discovery and data mining scheme for crime text mining is able to extract plausible associations between crimes, identifying crime pattern, grouping similar crimes, eliciting co-offender network and suspect list based on spatial-temporal and behavioral similarity. These similarities are quantified through calculating Cosine, Jacquard, and Euclidean distances. Additionally, each suspect is also ranked by a similarity score in the plausible suspect list. These associations are then visualized through creating a two-dimensional re-configurable crime cluster space along with a bipartite knowledge graph.This proposed scheme also inspects the grand challenge of integrating effective human interaction with the machine learning algorithms through a visualization feedback loop. It allows the analyst to feed his/her domain knowledge including choosing of similarity functions for identifying associations, dynamic feature selection for interactive clustering of crimes and assigning weights to each component of the crime pattern to rank suspects for an unsolved crime.We demonstrate the proposed scheme through a case study using the Anonymized burglary dataset. The scheme is found to facilitate human reasoning and analytic discourse for intelligence analysis.     

Joint estimation of position and attitude for intelligent vehicles by fusing delayed visual measurements

Accurate position and attitude information is an important basis for normal driving of intelligent vehicles. In this paper, we investigate the estimation of position and attitude states for intelligent vehicles with low cost scheme. The low cost GNSS, camera, and proprioceptive sensors equipped by mass-produced vehicle are fused to estimate the states. The visual measurements adopted in this paper are based on the lateral distance and deflection angle to road features such as lane lines or curbs, which are generated more frequently than some other semantic features such as traffic lights, and leads to broader application scenario. Moreover, it is easier to implement compared with geometrical feature matching methods, since it only needs a simple prior map while latter needs large maps containing many high precision features. The visual measurements is often with large time delay due to negligible processing time. In order to fuse delayed measurements, a state-augmentation technique is adopted for the estimator design. The performance of the proposed method is evaluated by professional simulation software CarMaker, and shows that the incorporation of road features based visual measurement can effectively improve the position and attitude estimation accuracy by reducing the lateral position and yaw angle estimation error.     

An improved pipeline leak detection and localization method based on compressed sensing and event-triggered particle filter

This paper proposes an improved model based pipeline leak detection and localization method based on compressed sensing (CS) and event-triggered (ET) particle filter (ET-PF). First, the state space model of the pipeline system is established based on the characteristic line method. Then, the CS method is used to preprocess the sensor signals to recover the potentially lost leak information which is caused by the low sampling frequency of the industrial pipeline sensors, and an event based beetle antennae search (BAS) particle filter (BAS-PF) is proposed to improve the accuracy and efficiency of the pipeline state estimation. Finally, a pipeline leak detection and localization method is developed based on the proposed signal processing, and state estimation algorithms, as well as a pipeline partition strategy. Experiment results show that the proposed method can accurately detect and locate the leak of the pipeline system with a localization error of about 1.4%.     

Feature engineering vs. deep learning for paper section identification: Toward applications in Chinese medical literature

Section identification is an important task for library science, especially knowledge management. Identifying the sections of a paper would help filter noise in entity and relation extraction. In this research, we studied the paper section identification problem in the context of Chinese medical literature analysis, where the subjects, methods, and results are more valuable from a physician's perspective. Based on previous studies on English literature section identification, we experiment with the effective features to use with classic machine learning algorithms to tackle the problem. It is found that Conditional Random Fields, which consider sentence interdependency, is more effective in combining different feature sets, such as bag-of-words, part-of-speech, and headings, for Chinese literature section identification. Moreover, we find that classic machine learning algorithms are more effective than generic deep learning models for this problem. Based on these observations, we design a novel deep learning model, the Structural Bidirectional Long Short-Term Memory (SLSTM) model, which models word and sentence interdependency together with the contextual information. Experiments on a human-curated asthma literature dataset show that our approach outperforms the traditional machine learning methods and other deep learning methods and achieves close to 90% precision and recall in the task. The model shows good potential for use in other text mining tasks. The research has significant methodological and practical implications.     

Advancement of management information system for discovering fraud in master card based intelligent supervised machine learning and deep learning during SARS-CoV2

During coronavirus (SARS-CoV2) the number of fraudulent transactions is expanding at a rate of alarming (7,352,421 online transaction records). Additionally, the Master Card (MC) usage is increasing. To avoid massive losses, companies of finance must constantly improve their management information systems for discovering fraud in MC. In this paper, an approach of advancement management information system for discovering of MC fraud was developed using sequential modeling of data depend on intelligent forecasting methods such as deep Learning and intelligent supervised machine learning (ISML). The Long Short-Term Memory Network (LSTM), Logistic Regression (LR), and Random Forest (RF) were used. The dataset is separated into two parts: the training and testing data, with a ratio of 8:2. Also, the advancement of management information system has been evaluated using 10-fold cross validation depend on recall, f1-score, precision, Mean Absolute Error (MAE), Receiver Operating Curve (ROC), and Root Mean Square Error (RMSE). Finally various techniques of resampling used to forecast if a transaction of MC is genuine/fraudulent. Performance for without re-sampling, with under-sampling, and with over-sampling is measured for each Algorithm. Highest performance of without re-sampling was 0.829 for RF algorithm-F score. While for under-sampling, it was 0.871 for LSTM algorithm-RMSE. Further, for over-sampling, it was 0.921 for both RF algorithm-Precision and LSTM algorithm-F score. The results from running advancement of management information system revealed that using resampling technique with deep learning LSTM generated the best results than intelligent supervised machine learning.     

MKL-SING: A data-driven approach of sign recognition for managing and improving public services

The detection and identification of traffic signs is a fundamental function of an intelligent transportation system. The extraction or identification of a road sign poses the same problems as object identification in natural contexts: conditions of illumination are variable and uncontrollable, and various objects frequently surround road signs. These difficulties make the extraction of features difficult. The fusion of time and space features of traffic signs is important for improving the performance of sign recognition. Deep learning-based algorithms are time-consuming to train based on a large amount of data. They are difficult to deploy on resource-constrained portable devices and conduct sign detection in real time. The accuracy of sign detection should be further improved, which is related to the safety of traffic participants. To improve the accuracy of feature extraction and classification of traffic signs, we propose MKL-SING, a hybrid approach based on multi-kernel support vector machine (MKL-SVM) for public transportation SIGN recognition. It contains three main components: a principal component analysis for image dimension reduction, a fused feature extractor, and a multi-kernel SVM-based classifier. The fused feature extractor extracts and fuses the time and space features of traffic signs. The multi-kernel SVM then classifies the traffic signs based on the fused features. Different kernel functions in the multi-kernel SVM are fused based on a feature weighting procedure. Compared with single-core SVM, multi-kernel SVM can better process massive data because it can project each kernel function into high-dimensional feature space to get global solutions. Finally, the performance of SVM-TSR is validated based on three traffic sign datasets. Experiment results show that SVM-TSR performs better than state-of-the-art methods in terms of dynamic traffic sign identification and recognition.     

A novel intelligent classification model for breast cancer diagnosis

Breast cancer is one of the leading causes of death among women worldwide. Accurate and early detection of breast cancer can ensure long-term surviving for the patients. However, traditional classification algorithms usually aim only to maximize the classification accuracy, failing to take into consideration the misclassification costs between different categories. Furthermore, the costs associated with missing a cancer case (false negative) are clearly much higher than those of mislabeling a benign one (false positive). To overcome this drawback and further improving the classification accuracy of the breast cancer diagnosis, in this work, a novel breast cancer intelligent diagnosis approach has been proposed, which employed information gain directed simulated annealing genetic algorithm wrapper (IGSAGAW) for feature selection, in this process, we performs the ranking of features according to IG algorithm, and extracting the top m optimal feature utilized the cost sensitive support vector machine (CSSVM) learning algorithm. Our proposed feature selection approach which can not only help to reduce the complexity of SAGASW algorithm and effectively extracting the optimal feature subset to a certain extent, but it can also obtain the maximum classification accuracy and minimum misclassification cost. The efficacy of our proposed approach is tested on Wisconsin Original Breast Cancer (WBC) and Wisconsin Diagnostic Breast Cancer (WDBC) breast cancer data sets, and the results demonstrate that our proposed hybrid algorithm outperforms other comparison methods. The main objective of this study was to apply our research in real clinical diagnostic system and thereby assist clinical physicians in making correct and effective decisions in the future. Moreover our proposed method could also be applied to other illness diagnosis.     

频域分析在离心式压缩机故障诊断中的应用

简单阐述了压缩机旋转失速和喘振的关系,着重分析压缩机旋转失速和喘振的产生机理.由于离心武压缩机发生故障的主要特征是机器伴有异常的振动和噪声.振动信号是周期性信号,利用频域分析的方法对离心式压缩机故障进行诊断.取得了很好的效果.     

Towards a real-time processing framework based on improved distributed recurrent neural network variants with fastText for social big data analytics

Big data generated by social media stands for a valuable source of information, which offers an excellent opportunity to mine valuable insights. Particularly, User-generated contents such as reviews, recommendations, and users’ behavior data are useful for supporting several marketing activities of many companies. Knowing what users are saying about the products they bought or the services they used through reviews in social media represents a key factor for making decisions. Sentiment analysis is one of the fundamental tasks in Natural Language Processing. Although deep learning for sentiment analysis has achieved great success and allowed several firms to analyze and extract relevant information from their textual data, but as the volume of data grows, a model that runs in a traditional environment cannot be effective, which implies the importance of efficient distributed deep learning models for social Big Data analytics. Besides, it is known that social media analysis is a complex process, which involves a set of complex tasks. Therefore, it is important to address the challenges and issues of social big data analytics and enhance the performance of deep learning techniques in terms of classification accuracy to obtain better decisions.In this paper, we propose an approach for sentiment analysis, which is devoted to adopting fastText with Recurrent neural network variants to represent textual data efficiently. Then, it employs the new representations to perform the classification task. Its main objective is to enhance the performance of well-known Recurrent Neural Network (RNN) variants in terms of classification accuracy and handle large scale data. In addition, we propose a distributed intelligent system for real-time social big data analytics. It is designed to ingest, store, process, index, and visualize the huge amount of information in real-time. The proposed system adopts distributed machine learning with our proposed method for enhancing decision-making processes. Extensive experiments conducted on two benchmark data sets demonstrate that our proposal for sentiment analysis outperforms well-known distributed recurrent neural network variants (i.e., Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (BiLSTM), and Gated Recurrent Unit (GRU)). Specifically, we tested the efficiency of our approach using the three different deep learning models. The results show that our proposed approach is able to enhance the performance of the three models. The current work can provide several benefits for researchers and practitioners who want to collect, handle, analyze and visualize several sources of information in real-time. Also, it can contribute to a better understanding of public opinion and user behaviors using our proposed system with the improved variants of the most powerful distributed deep learning and machine learning algorithms. Furthermore, it is able to increase the classification accuracy of several existing works based on RNN models for sentiment analysis.     

Learning soft mask with DNN and DNN-SVM for multi-speaker DOA estimation using an acoustic vector sensor

Using an acoustic vector sensor (AVS), an efficient method has been presented recently for direction of arrival (DOA) estimation of multiple speech sources via the clustering of the inter-sensor data ratio (AVS-ISDR). Through extensive experiments on simulated and recorded data, we observed that the performance of the AVS-DOA method is largely dependent on the reliable extraction of the target speech dominated time–frequency points (TD-TFPs) which, however, may be degraded with the increase in the level of additive noise and room reverberation in the background. In this paper, inspired by the great success of deep learning in speech recognition, we design two new soft mask learners, namely deep neural network (DNN) and DNN cascaded with a support vector machine (DNN-SVM), for multi-source DOA estimation, where a novel feature, namely, the tandem local spectrogram block (TLSB) is used as the input to the system. Using our proposed soft mask learners, the TD-TFPs can be accurately extracted under different noisy and reverberant conditions. Additionally, the generated soft masks can be used to calculate the weighted centers of the ISDR-clusters for better DOA estimation as compared to the original center used in our previously proposed AVS-ISDR. Extensive experiments on simulated and recorded data have been presented to show the improved performance of our proposed methods over two baseline AVS-DOA methods in presence of noise and reverberation.     

Information gain and divergence-based feature selection for machine learning-based text categorization

Most previous works of feature selection emphasized only the reduction of high dimensionality of the feature space. But in cases where many features are highly redundant with each other, we must utilize other means, for example, more complex dependence models such as Bayesian network classifiers. In this paper, we introduce a new information gain and divergence-based feature selection method for statistical machine learning-based text categorization without relying on more complex dependence models. Our feature selection method strives to reduce redundancy between features while maintaining information gain in selecting appropriate features for text categorization. Empirical results are given on a number of dataset, showing that our feature selection method is more effective than Koller and Sahami’s method [Koller, D., & Sahami, M. (1996). Toward optimal feature selection. In Proceedings of ICML-96, 13th international conference on machine learning], which is one of greedy feature selection methods, and conventional information gain which is commonly used in feature selection for text categorization. Moreover, our feature selection method sometimes produces more improvements of conventional machine learning algorithms over support vector machines which are known to give the best classification accuracy.     

智能教学系统的自学习机制

本文主要介绍了智能教学系统中的机器自学习机制,研究如何提高智能教学系统的智能性和通用性等方面的问题。文章采用基于信息论的示例学习,改进了决策树学习算法,并建立了机器学习决策树。     

High-speed train localization algorithm via cooperative multi-classifier network using distributed heterogeneous signals

Long-distance high-speed train localization based on distributed optical fiber sensors (DOFS) has been a challenging issue due to the large-scale heterogeneous sensor nodes. It requires a competent localization algorithm to be capable of strong generalization and quick response. This paper proposes a cooperative multi-classifier network (CMCN) for locating HSTs based on heterogeneous DOFS signals by adaptive modeling of the local characteristics. The proposed CMCN is composed of adaptive feature extraction, lightweight base classifiers and spatial boostrap aggregating (SBA). First, the heterogeneous signals are adaptively transformed to an optimal intrinsic mode function for extracting the statistical features of base classifiers. The base classifiers are constructed based on dynamic soft-margin support vector machine to model local characteristics without computationally burdensome kernel functions by introducing a dynamic penalty factor. The factor is automatically initialized by evaluating the regional consistency before training. Furthermore, the SBA estimates the location of HSTs based on the local states of nodes. It can cooperate with base classifiers for enhanced accuracy by searching for the interval with maximum regional consistency. Finally, a trial is conducted in a high-speed railway in China in long-term running of 92 days. The results prove feasibility and accuracy of the proposed algorithm.     

Stop ordering machine learning algorithms by their explainability! A user-centered investigation of performance and explainability

Machine learning algorithms enable advanced decision making in contemporary intelligent systems. Research indicates that there is a tradeoff between their model performance and explainability. Machine learning models with higher performance are often based on more complex algorithms and therefore lack explainability and vice versa. However, there is little to no empirical evidence of this tradeoff from an end user perspective. We aim to provide empirical evidence by conducting two user experiments. Using two distinct datasets, we first measure the tradeoff for five common classes of machine learning algorithms. Second, we address the problem of end user perceptions of explainable artificial intelligence augmentations aimed at increasing the understanding of the decision logic of high-performing complex models. Our results diverge from the widespread assumption of a tradeoff curve and indicate that the tradeoff between model performance and explainability is much less gradual in the end user’s perception. This is a stark contrast to assumed inherent model interpretability. Further, we found the tradeoff to be situational for example due to data complexity. Results of our second experiment show that while explainable artificial intelligence augmentations can be used to increase explainability, the type of explanation plays an essential role in end user perception.     

Evaluating collaborative filtering recommendations inside large learning object repositories

Collaborative filtering (CF) algorithms are techniques used by recommender systems to predict the utility of items for users based on the similarity among their preferences and the preferences of other users. The enormous growth of learning objects on the internet and the availability of preferences of usage by the community of users in the existing learning object repositories (LORs) have opened the possibility of testing the efficiency of CF algorithms on recommending learning materials to the users of these communities. In this paper we evaluated recommendations of learning resources generated by different well known memory-based CF algorithms using two databases (with implicit and explicit ratings) gathered from the popular MERLOT repository. We have also contrasted the results of the generated recommendations with several existing endorsement mechanisms of the repository to explore possible relations among them. Finally, the recommendations generated by the different algorithms were compared in order to evaluate whether or not they were overlapping. The results found here can be used as a starting point for future studies that account for the specific context of learning object repositories and the different aspects of preference in learning resource selection.     

Mixture quantized error entropy for recursive least squares adaptive filtering

Error entropy is a well-known learning criterion in information theoretic learning (ITL), and it has been successfully applied in robust signal processing and machine learning. To date, many robust learning algorithms have been devised based on the minimum error entropy (MEE) criterion, and the Gaussian kernel function is always utilized as the default kernel function in these algorithms, which is not always the best option. To further improve learning performance, two concepts using a mixture of two Gaussian functions as kernel functions, called mixture error entropy and mixture quantized error entropy, are proposed in this paper. We further propose two new recursive least-squares algorithms based on mixture minimum error entropy (MMEE) and mixture quantized minimum error entropy (MQMEE) optimization criteria. The convergence analysis, steady-state mean-square performance, and computational complexity of the two proposed algorithms are investigated. In addition, the reason why the mixture mechanism (mixture correntropy and mixture error entropy) can improve the performance of adaptive filtering algorithms is explained. Simulation results show that the proposed new recursive least-squares algorithms outperform other RLS-type algorithms, and the practicality of the proposed algorithms is verified by the electro-encephalography application.