电子信息资源检索网络教学系统的研究与应用

网络的发展推动了教学方式的改革,我馆采用流媒体技术制作多媒体网络课件,为学生提供视频、音频、文本、图片一体化的真正的多媒体互动教学.本文探讨了网络教学的发展趋势和网络教学的特点与意义,阐述了建设网络教学系统的必要性与可行性,提出了组织电子信息资源检索网络教学的具体措施,对优化网上教学资源,更好地组织电子信息资源检索网上教学进行了探讨.     

网络课程内容表征设计研究

网络课程是现代远程教育的重要资源,是终身教育的重要方式,网络课程是通过互联网进行传播的,网络课程内容的呈现需要网络多媒体技术的支持,和传统的面对面的课程相比有其网络环境下的特性,根据网络课程内容的特点选择合适的表征方式非常重要,本文探讨了网络课程文本、图片、课件、视频媒体等的表征特点,以及制作方式,为网络课程的内容表征设计提供了一定的参考。     

舞蹈多媒体资源信息管理

舞蹈是一门时间与空间相结合的视觉艺术,所以,舞蹈的多媒体技术可以让大家更加直观地对舞蹈知识进行了解,舞蹈的多媒体信息包括视频、音频、文字、PPT等,我们要将这些资源进行有效地分类、管理,让学生和老师可以更好地进行互动学习,也让许多舞蹈爱好者可以更好地提高舞蹈水平。     

Real time video scene detection and classification

The VISION (video indexing for searching over networks) digital video library system has been developed in our laboratory as a testbed for evaluating automatic and comprehensive mechanisms for video archive creation and content-based search, filtering and retrieval of video over local and wide area networks. In order to provide access to video footage within seconds of broadcast, we have developed a new pipelined digital video processing architecture which is capable of digitizing, processing, indexing and compressing video in real time on an inexpensive general purpose computer. These videos were automatically partitioned into short scenes using video, audio and closed-caption information. The resulting scenes are indexed based on their captions and stored in a multimedia database. A client-server-based graphical user interface was developed to enable users to remotely search this archive and view selected video segments over networks of different bandwidths. Additionally, VISION classifies the incoming videos with respect to a taxonomy of categories and will selectively send users videos which match their individual profiles.     

卫星定位与视频监控联动的方法研究

通过获取视音频数据和卫星定位数据,实现了对事发现场信息的采集和管理。系统由视音频数据采集、定位数据数据采集、应用软件等部分组成。在分析视音频数据特征、定位数据特征以及项目现有的GPS监控系统资源的基础上,设计了一个在卫星监控定位系统添加实现视频监控的方案。     

Polarity Classification of Twitter Messages using Audio Processing

Polarity classification is one of the most fundamental problems in sentiment analysis. In this paper, we propose a novel method, Sound Cosine Similaritye Matching, for polarity classification of Twitter messages which incorporates features based on audio data rather than on grammar or other text properties, i.e., eliminates the dependency on external dictionaries. It is useful especially for correctly identifying misspelled or shortened words that are frequently encountered in text from online social media. Method performance is evaluated in two levels: i) capture rate of the misspelled and shortened words, ii) classification performance of the feature set. Our results show that classification accuracy is improved, compared to two other models in the literature, when the proposed features are used.     

Rare-aware attention network for image–text matching

Image and text matching bridges visual and textual modality differences and plays a considerable role in cross-modal retrieval. Much progress has been achieved through semantic representation and alignment. However, the distribution of multimedia data is severely unbalanced and contains many low-frequency occurrences, which are often ignored and cause performance degradation, i.e., the long-tail effect. In this work, we propose a novel rare-aware attention network (RAAN), which explores and exploits textual rare content for tackling the long-tail effect of image and text matching. Specifically, we first design a rare-aware mining module, which contains global prior information construction and rare fragment detector for modeling the characteristic of rare content. Then, the rare attention matching utilizes prior information as attention to guide the representation enhancement of rare content and introduces the rareness representation to strengthen the similarity calculation. Finally, we design prior information loss to optimize the model together with the triplet loss. We perform quantitative and qualitative experiments on two large-scale databases and achieve leading performance. In particular, we conduct 0-shot test for rare content and improve rSum by 21.0 and 41.5 on Flickr30K (155,000 image and text pairs) and MSCOCO (616,435 image and text pairs), demonstrating the effectiveness of the proposed method for the long-tail effect.     

Bayesian network based extreme learning machine for subjectivity detection

Subjectivity detection is a task of natural language processing that aims to remove ‘factual’ or ‘neutral’ content, i.e., objective text that does not contain any opinion, from online product reviews. Such a pre-processing step is crucial to increase the accuracy of sentiment analysis systems, as these are usually optimized for the binary classification task of distinguishing between positive and negative content. In this paper, we extend the extreme learning machine (ELM) paradigm to a novel framework that exploits the features of both Bayesian networks and fuzzy recurrent neural networks to perform subjectivity detection. In particular, Bayesian networks are used to build a network of connections among the hidden neurons of the conventional ELM configuration in order to capture dependencies in high-dimensional data. Next, a fuzzy recurrent neural network inherits the overall structure generated by the Bayesian networks to model temporal features in the predictor. Experimental results confirmed the ability of the proposed framework to deal with standard subjectivity detection problems and also proved its capacity to address portability across languages in translation tasks.     

音视频资源辅助中职语文教学模式初探

随着职业教育办学条件的改善和多媒体技术的发展,音视频资源逐渐进入中职语文课堂。在中职语文教学过程中运用音视频资源辅助教学是可行且必要的,教师要认真研究教材,搜集整理音视频资源,细化课堂教学环节,恰当选择且精准运用音视频资源,对教学效果积极反馈、反思、客观评判,建构新的音视频资源辅助中职语文教学的模式。     

How narratives move your mind: A corpus of shared-character stories for connecting emotional flow and interestingness

Creativity is considered a human characteristic; creative endeavors, including automatic story generation, have been a major challenge for artificial intelligences. To understand how humans create and evaluate stories, we (1) construct a story dataset and (2) analyze the relationship between emotions and story interestingness. Given that understanding how to move readers emotionally is a crucial creative technique, we focus on the role of emotions in evaluating reader satisfaction. Although conventional research has highlighted emotions read from a text, we hypothesize that readers’ emotions do not necessarily coincide with those of the characters. The story dataset created for this study describes situations surrounding two characters. Crowdsourced volunteers label stories with the emotions of the two characters and those of readers; we then empirically analyze the relationship between emotions and interestingness. The results show that a story’s score has a stronger relationship to the readers’ emotions than the characters’ emotions.     

数字人文视角下《谭延闿日记》人物关系挖掘及可视化研究

【目的/意义】基于数字人文理论与方法,挖掘《谭延闿日记》中蕴含的人物关系,形成能够呈现日记人物同 现关系的可视化图谱,将非结构化的日记文本以更加清晰直观的方式进行展示,并力图在此过程中发现和提炼有 用的知识。【方法/过程】以 1923—1926年的《谭延闿日记》内容为研究对象,抽取具有同现关系的人物实体要素,运 用 Gephi数据可视化软件构建日记人物同现关系网络图谱,并通过量化统计、社会网络分析等方法对网络拓扑特 征、人物中心性特征以及基于模块化和k-core的人物群体特征等问题进行分析与讨论。【结果/结论】以人物关系挖 掘为切入点,发现和提炼《谭延闿日记》中蕴含的知识,展现了数字人文视阈下细粒度开发名人日记资源的可行性。 【创新/局限】构建了《谭延闿日记》人物同现关系网络,从不同角度对其进行分析与可视化呈现,并结合相关历史研 究进行对比验证,以更加直观的方式展现对日记文本内容的挖掘过程与结果,为其他历史档案资源的开发提供参 考。但是所抽取数据为局部时间段数据,仅能展现局部时间段的特定人物关系,更多、更丰富人物关系的挖掘与呈 现还需更长时段的数据与更多相关资料的充实。     

What Is the Content of the World's Technologically Mediated Information and Communication Capacity: How Much Text,Image, Audio,and Video?

This article asks whether the global process of digitization has led to noteworthy changes in the shares of the amount of text, images, audio, and video in worldwide technologically stored and communicated information content. We empirically quantify the amount of information that is globally broadcast, telecommunicated, and stored (1986–2007) and assess the evolution of the respective content shares. Somewhat unexpectedly, it turns out that the transfer from analog to digital has not led to toward increasing shares of media-rich audio and video content, despite vastly increased bandwidth. First, there is a certain inertia in the evolution of content, which seems to stick to stable proportions independent of its technological medium (be it analog vinyl and VHS tapes, or digital CDs and hard disks). Second, the relative share of text and still images actually captures a larger portion of the total amount than before the digital age. Text merely represented 0.3% of the (optimally compressed) bits that flowed through global information channels in 1986 but grew to almost 30% in 2007. On another level, we are seeing an increasing transition of text and images from one-way information diffusion networks (like newspapers) to digital storage and two-way telecommunications networks, where they are more socially embedded. Both tendencies are good news for big-data analysts who extract intelligence from easily analyzable text and image data.     

Investigating the document structure as a source of evidence for multimedia fragment retrieval

Multimedia objects can be retrieved using their context that can be for instance the text surrounding them in documents. This text may be either near or far from the searched objects. Our goal in this paper is to study the impact, in term of effectiveness, of text position relatively to searched objects. The multimedia objects we consider are described in structured documents such as XML ones. The document structure is therefore exploited to provide this text position in documents. Although structural information has been shown to be an effective source of evidence in textual information retrieval, only a few works investigated its interest in multimedia retrieval. More precisely, the task we are interested in this paper is to retrieve multimedia fragments (i.e. XML elements having at least one multimedia object). Our general approach is built on two steps: we first retrieve XML elements containing multimedia objects, and we then explore the surrounding information to retrieve relevant multimedia fragments. In both cases, we study the impact of the surrounding information using the documents structure.     

多媒体语料库管理系统设计与实现

开发了一个集多媒体语料库构建、标注、检索与定位播放的管理系统。通过该平台用户不仅能够方便自主地构建多媒体语料库,还可方便快速实现对多媒体语料的多维度标注、语料库的检索以及音视频内容的自动定位与播放。     

PS-Mixer: A Polar-Vector and Strength-Vector Mixer Model for Multimodal Sentiment Analysis

Multimodal sentiment analysis aims to judge the sentiment of multimodal data uploaded by the Internet users on various social media platforms. On one hand, existing studies focus on the fusion mechanism of multimodal data such as text, audio and visual, but ignore the similarity of text and audio, text and visual, and the heterogeneity of audio and visual, resulting in deviation of sentiment analysis. On the other hand, multimodal data brings noise irrelevant to sentiment analysis, which affects the effectness of fusion. In this paper, we propose a Polar-Vector and Strength-Vector mixer model called PS-Mixer, which is based on MLP-Mixer, to achieve better communication between different modal data for multimodal sentiment analysis. Specifically, we design a Polar-Vector (PV) and a Strength-Vector (SV) for judging the polar and strength of sentiment separately. PV is obtained from the communication of text and visual features to decide the sentiment that is positive, negative, or neutral sentiment. SV is gained from the communication between the text and audio features to analyze the sentiment strength in the range of 0 to 3. Furthermore, we devise an MLP-Communication module (MLP-C) composed of several fully connected layers and activation functions to make the different modal features fully interact in both the horizontal and the vertical directions, which is a novel attempt to use MLP for multimodal information communication. Finally, we mix PV and SV to obtain a fusion vector to judge the sentiment state. The proposed PS-Mixer is tested on two publicly available datasets, CMU-MOSEI and CMU-MOSI, which achieves the state-of-the-art (SOTA) performance on CMU-MOSEI compared with baseline methods. The codes are available at: https://github.com/metaphysicser/PS-Mixer.     

Web search engine multimedia functionality

Web search engines are beginning to offer access to multimedia searching, including audio, video and image searching. In this paper we report findings from a study examining the state of multimedia search functionality on major general and specialized Web search engines. We investigated 102 Web search engines to examine: (1) how many Web search engines offer multimedia searching, (2) the type of multimedia search functionality and methods offered, such as “query by example”, and (3) the supports for personalization or customization which are accessible as advanced search. Findings include: (1) few major Web search engines offer multimedia searching and (2) multimedia Web search functionality is generally limited. Our findings show that despite the increasing level of interest in multimedia Web search, those few Web search engines offering multimedia Web search, provide limited multimedia search functionality. Keywords are still the only means of multimedia retrieval, while other methods such as “query by example” are offered by less than 1% of Web search engines examined.     

基于Blackfin双核处理器的音频系统设计与实现

音频系统在嵌入式领域中有着广泛的应用，在网络视频公话终端产品中，音频系统需要实现语音采集、编码、解码、输出和网络传输五种功能。为了充分利用处理器的双核特性，采取了音频系统各功能模块分别运行于两个核心上的策略；同时设计了双核间的软件通讯机制，并在现有的开发环境下，实现了音频系统的各功能模块及uClinux操作系统下的音频驱动程序．     

The rise and fall of network stars: Analyzing 2.5 million graphs to reveal how high-degree vertices emerge over time

Trends change rapidly in today’s world, prompting this key question: What is the mechanism behind the emergence of new trends? By representing real-world dynamic systems as complex networks, the emergence of new trends can be symbolized by vertices that “shine.” That is, at a specific time interval in a network’s life, certain vertices become increasingly connected to other vertices. This process creates new high-degree vertices, i.e., network stars. Thus, to study trends, we must look at how networks evolve over time and determine how the stars behave. In our research, we constructed the largest publicly available network evolution dataset to date, which contains 38,000 real-world networks and 2.5 million graphs. Then, we performed the first precise wide-scale analysis of the evolution of networks with various scales. Three primary observations resulted: (a) links are most prevalent among vertices that join a network at a similar time; (b) the rate that new vertices join a network is a central factor in molding a network’s topology; and (c) the emergence of network stars (high-degree vertices) is correlated with fast-growing networks. We applied our learnings to develop a flexible network-generation model based on large-scale, real-world data. This model gives a better understanding of how stars rise and fall within networks, and is applicable to dynamic systems both in nature and society.Multimedia Links▶ Video ▶ Interactive Data Visualization ▶ Data ▶ Code Tutorials     

基于组播和P2P技术的会议直播系统的研究

利用P2P技术在广域网中传输多媒体数据,可以充分利用各个节点资源,提高传输效率。但在同一局域网内,能够将数据同时发给多个接收者的组播技术更具优势。文章给出组播技术同P2P技术相结合的具体实现方法,实现一种混合型的音视频会议直播系统。     

论数字多媒体资源的被动取证

阐述数字取证的定义及重要作用,从数字图像、数字视频、数字音频3个方面,论述数字多媒体资源的被动取证技术,认为目前还没有形成一套完整的多媒体资源的数字取证理论和实验体系,大多数还处于探索阶段,有待进一步深入研究.