Dynamic two-stage image retrieval from large multimedia databases

Content-based image retrieval (CBIR) with global features is notoriously noisy, especially for image queries with low percentages of relevant images in a collection. Moreover, CBIR typically ranks the whole collection, which is inefficient for large databases. We experiment with a method for image retrieval from multimedia databases, which improves both the effectiveness and efficiency of traditional CBIR by exploring secondary media. We perform retrieval in a two-stage fashion: first rank by a secondary medium, and then perform CBIR only on the top-K items. Thus, effectiveness is improved by performing CBIR on a ‘better’ subset. Using a relatively ‘cheap’ first stage, efficiency is also improved via the fewer CBIR operations performed. Our main novelty is that K is dynamic, i.e. estimated per query to optimize a predefined effectiveness measure. We show that our dynamic two-stage method can be significantly more effective and robust than similar setups with static thresholds previously proposed. In additional experiments using local feature derivatives in the visual stage instead of global, such as the emerging visual codebook approach, we find that two-stage does not work very well. We attribute the weaker performance of the visual codebook to the enhanced visual diversity produced by the textual stage which diminishes codebook’s advantage over global features. Furthermore, we compare dynamic two-stage retrieval to traditional score-based fusion of results retrieved visually and textually. We find that fusion is also significantly more effective than single-medium baselines. Although, there is no clear winner between two-stage and fusion, the methods exhibit different robustness features; nevertheless, two-stage retrieval provides efficiency benefits over fusion.     

An analysis of image retrieval behavior for metadata type image database

The aim of this paper was to analyze users’ behavior during image retrieval exercises. Results revealed that users tend to follow a set search strategy: firstly they input one or two keyword search terms one after another and view the images generated by their initial search and after they navigate their way around the web by using the ‘back to home’ or ‘previous page’ buttons. These results are consistent with existing Web research. Many of the actions recorded revealed that subjects behavior differed depending on if the task set was presented as a closed or open task. In contrast no differences were found for the time subjects took to perform a single action or their use of the AND operator.     

Object identification and retrieval from efficient image matching. Snap2Tell with the STOIC dataset

Traditional content based image retrieval attempts to retrieve images using syntactic features for a query image. Annotated image banks and Google allow the use of text to retrieve images. In this paper, we studied the task of using the content of an image to retrieve information in general. We describe the significance of object identification in an information retrieval paradigm that uses image set as intermediate means in indexing and matching. We also describe a unique Singapore Tourist Object Identification Collection with associated queries and relevance judgments for evaluating the new task and the need for efficient image matching using simple image features. We present comprehensive experimental evaluation on the effects of feature dimensions, context, spatial weightings, coverage of image indexes, and query devices on task performance. Lastly we describe the current system developed to support mobile image-based tourist information retrieval.     

Seeing is believing: Towards interactive visual exploration of data privacy in federated learning

Federated learning (FL), as a popular distributed machine learning paradigm, has driven the integration of knowledge in ubiquitous data owners under one roof. Although designed for privacy-preservation by nature, the supposed well-sanitized parameters still convey sensitive information (e.g., reconstruction attack), while existing technical countermeasures provide weak explainability for privacy understanding and protection practices of general users. This work investigates these privacy concerns with an exploratory study and elaborates on data owners’ expectations in FL. Based on the analysis, we design the first interactive visualization system for FL privacy that supports intelligible privacy inspection and adjustment for data owners. Specifically, our proposal facilitates sample recommendation for joint privacy–performance training at cold start. Then it provides visual interpretation and attention rendering of privacy risks in view of multiple attacking channels and a holistic view. Further it supports interactive privacy enhancement involving both user initiative and differential privacy technique, and iterative trade-off with real-time inference accuracy estimation. We evaluate the effectiveness of the system and collect qualitative feedbacks from users. The results demonstrate that 96.7% of users acknowledge the benefits to privacy inspection and adjustment and 90.3% are willing to use our system. More importantly, 87.1% increase the willingness of contributing data for FL.     

Weakly supervised deep metric learning on discrete metric spaces for privacy-preserved clustering

With an increase in the number of data instances, data processing operations (e.g. clustering) requires an increasing amount of computational resources, and it is often the case that for considerably large datasets such operations cannot be executed on a single workstation. This requires the use of a server computer for carrying out the operations. However, to ensure privacy of the shared data, a privacy preserving data processing workflow involves applying an encoding transformation on the set of data points prior to applying the computation. This encoding should ideally cater to two objectives—first, it should be difficult to reconstruct the data, second, the results of the operation executed on the encoded space should be as close as possible to the results of the same operation executed on the original data. While standard encoding mechanisms, such as locality sensitive hashing, caters to the first objective, the second objective may not always be adequately satisfied.In this paper, we specifically focus on ‘clustering’ as the data processing operation. We apply a deep metric learning approach to learn a parameterized encoding transformation function with an objective to maximize the alignment of the clusters in the encoded space to those in the original data. We conduct experimentation on four standard benchmark datasets, particularly MNIST, Fashion-MNIST (each dataset contains 70K grayscale images), CIFAR-10 consisting of 60K color images and 20-Newsgroups containing 18K news articles. Our experiments demonstrate that the proposed method yields better clusters in comparison to approaches where the encoding process is agnostic of the clustering objective.     

Adaptive multi-feature fusion via cross-entropy normalization for effective image retrieval

Multi-feature fusion has achieved gratifying performance in image retrieval. However, some existing fusion mechanisms would unfortunately make the result worse than expected due to the domain and visual diversity of images. As a result, a burning problem for applying feature fusion mechanism is how to figure out and improve the complementarity of multi-level heterogeneous features. To this end, this paper proposes an adaptive multi-feature fusion method via cross-entropy normalization for effective image retrieval. First, various low-level features (e.g., SIFT) and high-level semantic features based on deep learning are extracted. Under each level of feature representation, the initial similarity scores of the query image w.r.t. the target dataset are calculated. Second, we use an independent reference dataset to approximate the tail of the attained initial similarity score ranking curve by cross-entropy normalization. Then the area under the ranking curve is calculated as the indicator of the merit of corresponding feature (i.e., a smaller area indicates a more suitable feature.). Finally, fusion weights of each feature are assigned adaptively by the statistically elaborated areas. Extensive experiments on three public benchmark datasets have demonstrated that the proposed method can achieve superior performance compared with the existing methods, improving the metrics mAP by relatively 1.04% (for Holidays), 1.22% (for Oxf5k) and the N-S by relatively 0.04 (for UKbench), respectively.     

位置服务中基于假数据的轨迹隐私度量方法

随着各种各样移动设备的普及,轨迹隐私保护问题变得日益严峻,作为轨迹隐私保护的一个方向,国内外开始慢慢建立起一些较为成熟的度量机制,常见的有基于k-匿名保护机制的隐私度量方法、基于跟踪的度量方法、针对特定匿名系统的轨迹隐私度量方法等。文章在现有研究的基础上构建了了一种基于假数据轨迹隐私保护技术的隐私度量标准,从发布假轨迹上点之间的关联性、发布数据所包含信息的精确度以及攻击者所拥有背景知识多少三个方面进行轨迹隐私保护度的全面度量。     

Color image retrieval technique based on color features and image bitmap

The field of color image retrieval has been an important research area for several decades. For the purpose of effectively retrieving more similar images from the digital image databases, this paper uses the color distributions, the mean value and the standard deviation, to represent the global characteristics of the image. Moreover, the image bitmap is used to represent the local characteristics of the image for increasing the accuracy of the retrieval system. As the experimental results indicated, the proposed technique indeed outperforms other schemes in terms of retrieval accuracy and category retrieval ability. Furthermore, the total memory space for saving the image features of the proposed method is less than Chan and Liu’s method.     

CIDER: Concept-based image diversification,exploration, and retrieval

Many of the approaches to image retrieval on the Web have their basis in text retrieval. However, when searchers are asked to describe their image needs, the resulting query is often short and potentially ambiguous. The solution we propose is to perform automatic query expansion using Wikipedia as the source knowledge base, resulting in a diversification of the search results. The outcome is a broad range of images that represent the various possible interpretations of the query. In order to assist the searcher in finding images that match their specific intentions for the query, we have developed an image organization method that uses both the conceptual information associated with each image, and the visual features extracted from the images. This, coupled with a hierarchical organization of the concepts, provides an interactive interface that takes advantage of the searchers’ abilities to recognize relevant concepts, filter and focus the search results based on these concepts, and visually identify relevant images while navigating within the image space. In this paper, we outline the key features of our image retrieval system (CIDER), and present the results of a preliminary user evaluation. The results of this study illustrate the potential benefits that CIDER can provide for searchers conducting image retrieval tasks.     

Geo-temporal distribution of tag terms for event-related image retrieval

Media sharing applications, such as Flickr and Panoramio, contain a large amount of pictures related to real life events. For this reason, the development of effective methods to retrieve these pictures is important, but still a challenging task. Recognizing this importance, and to improve the retrieval effectiveness of tag-based event retrieval systems, we propose a new method to extract a set of geographical tag features from raw geo-spatial profiles of user tags. The main idea is to use these features to select the best expansion terms in a machine learning-based query expansion approach. Specifically, we apply rigorous statistical exploratory analysis of spatial point patterns to extract the geo-spatial features. We use the features both to summarize the spatial characteristics of the spatial distribution of a single term, and to determine the similarity between the spatial profiles of two terms – i.e., term-to-term spatial similarity. To further improve our approach, we investigate the effect of combining our geo-spatial features with temporal features on choosing the expansion terms. To evaluate our method, we perform several experiments, including well-known feature analyzes. Such analyzes show how much our proposed geo-spatial features contribute to improve the overall retrieval performance. The results from our experiments demonstrate the effectiveness and viability of our method.     

A relational vector space model using an advanced weighting scheme for image retrieval

In this paper, we lay out a relational approach for indexing and retrieving photographs from a collection. The increase of digital image acquisition devices, combined with the growth of the World Wide Web, requires the development of information retrieval (IR) models and systems that provide fast access to images searched by users in databases. The aim of our work is to develop an IR model suited to images, integrating rich semantics for representing this visual data and user queries, which can also be applied to large corpora.     

Brain CT image database building for computer-aided diagnosis using content-based image retrieval

Content-based image retrieval for medical images is a primary technique for computer-aided diagnosis. While it is a premise for computer-aided diagnosis system to build an efficient medical image database which is paid less attention than that it deserves. In this paper, we provide an efficient approach to develop the archives of large brain CT medical data. Medical images are securely acquired along with relevant diagnosis reports and then cleansed, validated and enhanced. Then some sophisticated image processing algorithms including image normalization and registration are applied to make sure that only corresponding anatomy regions could be compared in image matching. A vector of features is extracted by non-negative tensor factorization and associated with each image, which is essential for the content-based image retrieval. Our experiments prove the efficiency and promising prospect of this database building method for computer-aided diagnosis system. The brain CT image database we built could provide radiologists with a convenient access to retrieve pre-diagnosed, validated and highly relevant examples based on image content and obtain computer-aided diagnosis.     

Advanced learning algorithms for cross-language patent retrieval and classification

We study several machine learning algorithms for cross-language patent retrieval and classification. In comparison with most of other studies involving machine learning for cross-language information retrieval, which basically used learning techniques for monolingual sub-tasks, our learning algorithms exploit the bilingual training documents and learn a semantic representation from them. We study Japanese–English cross-language patent retrieval using Kernel Canonical Correlation Analysis (KCCA), a method of correlating linear relationships between two variables in kernel defined feature spaces. The results are quite encouraging and are significantly better than those obtained by other state of the art methods. We also investigate learning algorithms for cross-language document classification. The learning algorithm are based on KCCA and Support Vector Machines (SVM). In particular, we study two ways of combining the KCCA and SVM and found that one particular combination called SVM_2k achieved better results than other learning algorithms for either bilingual or monolingual test documents.     

基于FIP的欧美在线隐私保护立法模式的比较研究

本文以美国联邦贸易委员会提出的公平信息原则（FP）为基础，对欧盟和美国在线隐私保护的立法模式及其相关法律进行比较分析；并在此基础上，对我国在线隐私保护的立法问题提出了作者的思考和建议。     

Integrating textual and visual information for cross-language image retrieval: A trans-media dictionary approach

This paper explores the integration of textual and visual information for cross-language image retrieval. An approach which automatically transforms textual queries into visual representations is proposed. First, we mine the relationships between text and images and employ the mined relationships to construct visual queries from textual ones. Then, the retrieval results of textual and visual queries are combined. To evaluate the proposed approach, we conduct English monolingual and Chinese–English cross-language retrieval experiments. The selection of suitable textual query terms to construct visual queries is the major issue. Experimental results show that the proposed approach improves retrieval performance, and use of nouns is appropriate to generate visual queries.     

基于工作过程的高职信息检索课学习情境设计——以信息资源类型为载体

结合信息检索课程性质与高职院校课程改革实际,以信息资源类型为载体,设计了基于工作过程的高职信息检索课学习情境,分析研究学习情境的构建、具体的学习任务及其相互之间的逻辑联系,以及理→实一体化教学组织形式的学法指导与考核评价体系。     

面向达斡尔族语言保护传承的内容图像检索技术应用研究

以达斡尔语言保护传承为目的,将基于内容的图像检索技术及智能技术应用于达斡尔双语移动学习中,通过调查与分析达斡尔双语学习现状和需求,提出了面向达斡尔双语移动学习基于内容的图像检索策略,构建了基于内容图像检索的达斡尔双语移动学习的学习环境,探讨了对基于内容图像检索的达斡尔双语移动学习推广应用的关键点。