基于ELK构建图书馆多数据源日志管理分析平台

论文针对图书馆日志多数据源且格式多样化导致的日志处理效率低下的问题,提出了基于ELK的日志综合解决方案。该方案首先探讨了日志采集、数据清洗、日志解析的规则和技巧以及基于TF-IDF、SVM算法设计的日志异常检测模型;其次基于提出方案的整体架构,详细地分析了日志采集、存储、可视化、异常检测的整个流程;最后,将该方案应用于中国矿业大学OPAC系统,对读者用户的检索行为、不良用户的攻击行为进行可视化分析。实验结果表明,论文提出的日志管理解决方案,在集群模式下检索性能极其高效,有效地提高了图书馆日志的处理能力。

Construction of Library Multi Data Source Log Management and Analysis Platform Based on ELK

Aiming at the low efficiency of log processing caused by multiple data sources and diversified formats of library logs,this paper puts forward a comprehensive log solution based on ELK.The scheme first discusses the rules and skills of log collection,data cleaning and log analysis,and the log anomaly detection model based on TF-IDF and SVM algorithm.Secondly,based on the overall architecture of the proposed scheme,the whole process of log collection,processing,visualization and anomaly detection is analyzed in detail.Finally,the scheme is applied to OPAC system of China University of Mining and Technology to visually analyze the retrieval behavior of readers and the attack behavior of bad users.The experimental results show that the log management solution proposed in this paper has extremely efficient retrieval performance in the cluster mode,and effectively improves the processing capacity of library logs.