Communication skills training exploiting multimodal emotion recognition

The teaching of communication skills is a labour-intensive task because of the detailed feedback that should be given to learners during their prolonged practice. This study investigates to what extent our FILTWAM facial and vocal emotion recognition software can be used for improving a serious game (the Communication Advisor) that delivers a web-based training of communication skills. A test group of 25 participants played the game wherein they were requested to mimic specific facial and vocal emotions. Half of the assignments included direct feedback and the other half included no feedback. It was investigated whether feedback on the mimicked emotions would lead to better learning. The results suggest the facial performance growth was found to be positive, particularly significant in the feedback condition. The vocal performance growth was significant in both conditions. The results are a significant indication that the automated feedback from the software improves learners’ communication performances.     

Robust video foreground segmentation and face recognition

Face recognition provides a natural visual interface for human computer interaction (HCI) applications. The process of face recognition, however, is inhibited by variations in the appearance of face images caused by changes in lighting, expression, viewpoint, aging and introduction of occlusion. Although various algorithms have been presented for face recognition, face recognition is still a very challenging topic. A novel approach of real time face recognition for HCI is proposed in the paper. In view of the limits of the popular approaches to foreground segmentation, wavelet multi-scale transform based background subtraction is developed to extract foreground objects. The optimal selection of the threshold is automatically determined, which does not require any complex supervised training or manual experimental calibration. A robust real time face recognition algorithm is presented, which combines the projection matrixes without iteration and kernel Fisher discriminant analysis (KFDA) to overcome some difficulties existing in the real face recognition. Superior performance of the proposed algorithm is demonstrated by comparing with other algorithms through experiments. The proposed algorithm can also be applied to the video image sequences of natural HCI. Project supported by the National Natural Science Foundation of China (Grant No.60872117), and the Leading Academic Discipline Project of Shanghai Municipal Education Commission (Grant No.J50104)     

基于半监督判别分析的语音情感识别(英文)

将基于多个嵌入图组合形式的半监督判别分析(SDA)以及核SDA(KSDA)应用于全监督的语音情感识别.在语音信号样本情感成分的预处理阶段,从样本语段中提取出多种特征及其统计参数,包括基音、过零率、能量、持续长度、共振峰和MFCC(Mel频率倒谱系数).在将样本特征送入分类器之前的维数约简阶段,使用经过参数优化的SDA或KSDA进行降维.Berlin语音情感数据库上的实验表明,在使用多类SVM分类器时的全监督语音情感识别中,SDA优于其他一些先进的基于谱图学习的维数约简算法,如LDA,LPP,MFA等,而KSDA通过核化的数据映射,能够取得比上述所有算法更好的识别效果.     

基于支持向量机的语音情感识别

针对语音情感识别特征识别问题，本利用支持向量机进行了研究．分析表明语音信号的情感特征参数在输入空间中不完全是一个线性分类的问题，使用非线性的核函数对输入空间进行映射可以有效地提高识别效率．与已有的多模式语音情感识别方式相比，利用高斯(径向基)核函数的支持向量机的识别效果优于其他已有的方法．     

Extraction of novel features for emotion recognition

Hilbert-Huang transform method has been widely utilized from its inception because of the superiority in varieties of areas. The Hilbert spectrum thus obtained is able to reflect the distribution of the signal energy in a number of scales accurately. In this paper, a novel feature called ECC is proposed via feature extraction of the Hilbert energy spectrum which describes the distribution of the instantaneous energy. The experimental results conspicuously demonstrate that ECC outperforms the traditional short-term average energy. Combination of the ECC with mel frequency cepstral coefficients (MFCC) delineates the distribution of energy in the time domain and frequency domain, and the features of this group achieve a better recognition effect compared with the feature combination of the short-term average energy, pitch and MFCC. Afterwards, further improvements of ECC are developed. TECC is gained by combining ECC with the teager energy operator, and EFCC is obtained by introducing the instantaneous frequency to the energy. In the experiments, seven status of emotion are selected to be recognized and the highest recognition rate 83.57% is achieved within the classification accuracy of boredom reaching 100%. The numerical results indicate that the proposed features ECC, TECC and EFCC can improve the performance of speech emotion recognition substantially.     

浅析软件人机界面设计

本文介绍了人机界面在当今软件产品中的重要性和人机界面发展的几个阶段以及设计软件人机界面的几个常见原则,最后对人机界面的总体发展趋势做了简单描述。     

基于时域正则化共空间模式的单次脑电情绪识别(英文)

通过使用单次提取脑电信号的分类技术进行情绪词的脑电(EEG)识别研究.以中文情绪双字词为实验材料,通过其诱发的EEG信号,对正性词与中性词、负性词与中性词分别进行分类.使用时域正则化的共空间模式对单次提取脑电信号进行特征提取,并利用线性判别分析方法进行特征分类,分类准确率集中于55%~65%.置换检验验证了实验分类准确率的统计学显著性,表明了情绪词和中性词的成功识别,也有效地证实了基于脑电信号的语言情绪信息的可识别性.此外,在15名被试中,10名被试的负性词与中性词识别率显著,而仅有4名被试的正性词与中性词识别率显著,说明负性情绪更易被识别.     

The first steps for adapting an artificial intelligence emotion expression recognition software for emotional management in the educational context

To test the suitability of an automatic system for emotional management in the classroom following the control-value theory of achievement emotions (CVT) framework, the performance of an emotional expression recognition software of our creation is evaluated in an online synchronous context. Sixty students from the Faculty of Education at the University of Alicante participated in 16 educational activities recording close-ups of their faces and completing the AEQ emotional self-report, as well as detailed reports from the subsequent review of their videos. In addition, they completed the VCQ-36 test to measure their volitional competencies and relate their influence on their emotional response. The results indicate a high coherence between the emotional expressions detected by the automatic system and the detailed emotional self-reports, but insufficient precision to meet the CVT requirements. On the other hand, both the AEQ test results and the emotion expression recognition software suggest students' preference for participative activities as opposed to passive ones. Meanwhile, statistical analysis results indicate that volitional competencies seem to influence the emotional response of students in the educational context, although the AI system does not show sufficient sensitivity in this field. Implications and limitations of this study for future work are discussed.

Practitioner notes

What is already known about this topic

• Student motivation and involvement in the learning process are highly related to appropriate emotional regulation, which can be associated with particular educational activities, strategies and methodologies.
• Deep learning technology based on convolutional neural networks feeds automatic systems focused on facial expression recognition from image analysis.

What this paper adds

• There is high coherence between the emotional expressions detected by the AI system and the students' emotional self-reports, but the AI system provides just emotional valences, insufficient to meet the CVT framework.
• Both emotional self-reports and the emotion recognition software suggest students' preference for active educational activities as opposed to passive ones.
• Volitional competencies seem to influence the emotional response of students in the educational context.

Implications for practice and/or policy

• It is possible to use automatic systems to effectively monitor the emotional response of students in the learning process.
• Only if sensitivity improved, a real-time, easy-to-interpret emotional expression recognition software interface could be implemented to assist teachers with the emotional management of their classes within the CVT framework, maximizing their motivation and engagement.

     

Towards the successful integration of e-learning systems in higher education in Iraq: A student perspective

This study aims to analyse the factors that can explain the adoption and effective use of a new e-learning system in Iraq. To achieve this, it uses a selection of factors that are present in the technology acceptance model (TAM) and the unified theory of acceptance and use of technology (UTAUT) model, and it includes three additional factors. The study hypothesises new relationships between the selected factors. Questionnaires were distributed to 300 university students in Iraq. Partial least squares-structural equation modelling (PLS-SEM) was used to analyse the data received. The findings suggest that perceived usefulness (PU), perceived ease of use (PEOU), subjective norms (SNs), information quality (IQ), system quality (SQ), technical support (TS) and self-efficacy (SE) have significant effects on behavioural intention (BI). In turn, BI and TS have significant direct effects on the actual use (AU) of e-learning systems. The factors age, gender and experience significantly moderated some of the relationships in the model. The research has several implications for policy makers, universities and the management of e-learning systems.     

Teacher facilitation support in ubiquitous learning environments

ABSTRACT

Teacher support for students in ubiquitous learning environments is challenging owing to physical distance and a lack of reliable real-time multimedia-rich communication. This is further complicated because the learning process is dynamic and problems need quick resolution. Given these challenges, this study proposes an architecture for a teacher facilitation support system. The system was implemented in the form of an interactive teacher dashboard. The interface also generates possible solutions to learning challenges while leaving the ultimate decision up to the teacher. The system feasibility was tested by work-through scenarios designed and validated by two experts. The system usability was evaluated using the System Usability Scale by 40 potential users. The ?ndings revealed that the dashboard has good feasibility and usability for providing teachers relevant information about their students’ learning progress in ubiquitous learning environments. It also enabled opportunity for pedagogical intervention when needed.     

用于人脸识别的特征融合

针对人脸识别中人脸图像的特征提取问题,提出了一种将全局特征与局部特征相融合的人脸识别方法.全局特征的提取采用主成分分析算法.主动外观模型定位58个特征点,在其中17个特征点处进行Gabor小波变换则可提取局部特征.归一化的全局匹配度(局部匹配度)可由测试图像和训练图像的全局特征(局部特征)得到.对归一化的全局匹配度和局部匹配度进行融合后,融合匹配度最大的训练图像所属的类即为识别结果.实验利用2个人脸图像数据库(AR和SJTU-IP-PR)测试该方法的识别率,结果表明该方法要优于PCA和EBGM,并且在一定的表情、光照和姿态变化的条件下是有效、稳健的.     

新闻视频图像中的模式识别技术研究

本文概述了新闻视频图像研究中所涉及到的多种模式识别技术,分别对各模式识别技术的发展历史和研究状况作了介绍与评论,剖析了它们的优点和不足.讨论了新闻视频图像中模式识别技术的最新发展方向及其国内的发展情况.     

Isolation and distinctiveness in the design of e-learning systems influence user preferences

When faced with excessive detail in an online environment, typical users have difficulty processing all the elements of representation. This in turn creates cognitive overload, which narrows the user's focus to a few select items. In the context of e-learning, we translated this aspect as the learner's demand for a system that facilitates the retrieval of learning content – one in which the representation is easy to read and understand. We hypothesized that the representation of content in an e-learning system's design is an important antecedent for learner preferences. The aspects of isolation and distinctiveness were incorporated into the design of e-learning representation as an attempt to promote student cognition. Following its development, the model was empirically validated by conducting a survey of 300 university students. We found that isolation and distinctiveness in the design elements appeared to facilitate the ability of students to read and remember online learning content. This in turn was found to drive user preferences for using e-learning systems. The findings provide designers with managerial insights for enticing learners to continue using e-learning systems.     

Human motion recognition using ultra-wideband radar and cameras on mobile robot

Cameras can reliably detect human motions in a normal environment, but they are usually affected by sudden illumination changes and complex conditions, which are the major obstacles to the reliability and robustness of the system. To solve this problem, a novel integration method was proposed to combine bi-static ultra-wideband radar and cameras. In this recognition system, two cameras are used to localize the object’s region, regions while a radar is used to obtain its 3D motion models on a mobile robot. The recognition results can be matched in the 3D motion library in order to recognize its motions. To confirm the effectiveness of the proposed method, the experimental results of recognition using vision sensors and those of recognition using the integration method were compared in different environments. Higher correct-recognition rate is achieved in the experiment.     

Emotions recognition in an intelligent elearning environment

ABSTRACT

For the purpose of improving the quality in Elearning process and overcoming the limitations of the current online educational environments, we propose to take into consideration the emotional states of students during Elearning sessions. Our objective is to ensure the ability of emotional intelligence: Emotion Recognition, in an eLearning environment. Thus, we present an architecture of Emotionally Intelligent Elearning System (EIES). Within the development of a computational probabilistic model of emotions, we proposed a Bayesian Network (BN) model to deal with emotions in Elearning environments and handle the uncertain nature of emotion recognition process. In a second phase, we focus on the incorporation of the emotion recognition in the Elearning systems by developing a simulation of EIES based on the BN model, able to predict the students’ affects. Consequently, we reached positive and promising results related to the fact that simulated EIES based on the BN model of emotions predicts correctly the student’s emotion when an event occurs during an Elearning session.     

Using emotion regulation to cope with challenges: a study of Chinese students in the United Kingdom

There is increasing research on the challenges that Chinese students experience during their time studying abroad, but limited studies have explored how they self-regulate their emotions to address these challenges. This paper identifies key stressors experienced by Chinese postgraduate students during their study in academic institutions in the United Kingdom as well as the emotion regulation strategies that they employed. Understanding the emotional experiences will provide important insights into Chinese students’ learning behaviour and their choice of different emotion regulation strategies when interacting with peers and academic tutors. Emotion regulation strategies believed to improve individual student experiences and their interaction with tutors and peers are discussed.     

航管仿真系统中语音技术的应用

分析了在航管雷达模拟系统增加语音识别和合成功能的实现过程。根据航管雷达模拟系统的特点,采用中英文皆可识别且适合连续识别的语音引擎,以及符合人类语言特点的TTS语音库,与系统内部通讯机制相结合,实现语音识别和合成的过程。该系统运行稳定,在识别率和数据传输速度等方面效果比较好。     

基于支持向量机的人脸融合特征提取方法

针对人脸识别系统中的主成分分析和线性判别分析两种特征提取方法的优缺点,提出了一个融合特征提取方法,并构造了一个能够将图像数据空间的人脸映射到人脸特征空间中并实施识别的实验系统。最后分析了该系统的构成与特点,并给出了实验测试结果。     

The effect of content representation design principles on users’ intuitive beliefs and use of e-learning systems

A model is proposed to assess the effect of different content representation design principles on learners’ intuitive beliefs about using e-learning. We hypothesized that the impact of the representation of course contents is mediated by the design principles of alignment, quantity, clarity, simplicity, and affordance, which influence the learner’s intuitive beliefs about using e-learning systems. The model was empirically validated using data collected from a survey administered to university students. This study demonstrates that these design principles are essential predictors of learners’ intuitive beliefs, which in turn directly influence their decisions about using e-learning systems. The findings provide system designers with quasi-quantitative managerial insights into how to motivate users to continue using e-learning systems.     

Theorising effective uses of digital technology with activity theory

ABSTRACT

Effective uses of digital technologies are vital to full inclusion in a network society. Digital-abilities researchers have produced several major frameworks, but these have generally not incorporated socio-contextual perspectives. To explore this lacuna, and engage in a reflective act of theorisation, activity theory is used to conceptualise four sub-systems of digitally mediated action. Eschewing technological determinism, humans are positioned as active agents capable of identifying, taking up, modifying and even subverting established technology uses in pursuit of meaningful objectives. At the same time, attention is given to contextual conditions shaping diverse activity systems supported by assemblages of humans and machines. Having theorised effective digital-technology uses through the lens of activity theory, the author reflects on this conceptual apparatus itself. In so doing, activity theory is characterised as a fertile, if complex and contested, future-oriented tradition that challenges individual-social dichotomies and addresses both humans and machines as mediators of activity, development and learning.