全景透视多模态学习分析的数据整合方法

多模态学习分析被认为是学习分析研究的新生长点,其中,多模态数据如何整合是推进学习分析研究的难点。利用系统文献综述及元分析方法,有助于为研究和实践领域提供全景式的关于多模态数据整合的方法与策略指导。通过对国内外363篇相关文献的系统分析发现:(1)多模态学习分析中的数据类型主要包含数字空间数据、物理空间数据、生理体征数据、心理测量数据和环境场景数据等5类。在技术支持的教与学环境中,高频、精细、微观的多模态学习数据变得可得、易得、准确。(2)多模态学习分析中的学习指标主要有行为、注意、认知、元认知、情感、协作、交互、投入、学习绩效和技能等。随着技术的发展和人们对学习过程的深刻洞察,学习指标也会变得更加精细化。(3)数据与指标之间展现出"一对一""一对多"和"多对一"三种对应关系。把握数据与指标之间的复杂关系是数据整合的前提,测量学习指标时既要考虑最适合的数据,也要考虑其他模态数据的补充。(4)多模态学习分析中的数据整合方式主要有"多对一""多对多"和"三角互证"三种,旨在提高测量的准确性、信息的全面性和整合的科学性。总之,多模态数据整合具有数据的多模态、指标的多维度和方法的多样性等三维特性。将多模态数据时间线对齐是实现数据整合的关键环节,综合考虑三维特性提高分析结果的准确性是多模态数据整合未来研究的方向。

Data Fusion Method in Multimodal Learning Analytics:From a Panoramic Perspective

Multimodal data analysis helps us to understand the learning processes accurately.This paper systematically surveyed 312 articles in English and 51 articles in Chinese on multimodal data analysis and the findings show as follows.The analysis stages are collecting multimodal data in the learning process,converting multimodal data into learning indicators,and applying learning indicators to teaching and learning.High-frequency,fine-grained and micro-level multidimensional data in the learning processes are available,convenient and accurate,including digital data,physical data,physiological data,psychometric data and environment data.The learning indicators include behavior,cognition,emotion,collaboration and so on.The corresponding relationships between learning data and learning indicators are classified into one-to-one,one-to-many,and many-to-one.The complex relationship between learning data and learning indicators is the premise of data fusion.When measuring a learning indicator,two issues need to be considered:which type of data is the most effective one in measuring the indicator and whether there are any other types of data that contribute to more accurate measurements.Aligning the timeline of multimodal data is the key to data integration.In a word,the main characteristics of multimodal data analysis are characterized as multimodality of learning data,multi-dimension of learning indicators and diversity of analysis methods.Comprehensive consideration of the three-dimensional characteristics to improve the accuracy of analysis results is the direction of future research on multimodal data integration.