A bias–variance evaluation framework for information retrieval systems

In information retrieval (IR), the improvement of the effectiveness often sacrifices the stability of an IR system. To evaluate the stability, many risk-sensitive metrics have been proposed. Since the theoretical limitations, the current works study the effectiveness and stability separately, and have not explored the effectiveness–stability tradeoff. In this paper, we propose a Bias–Variance Tradeoff Evaluation (BV-Test) framework, based on the bias–variance decomposition of the mean squared error, to measure the overall performance (considering both effectiveness and stability) and the tradeoff between effectiveness and stability of a system. In this framework, we define generalized bias–variance metrics, based on the Cranfield-style experiment set-up where the document collection is fixed (across topics) or the set-up where document collection is a sample (per-topic). Compared with risk-sensitive evaluation methods, our work not only measures the effectiveness–stability tradeoff of a system, but also effectively tracks the source of system instability. Experiments on TREC Ad-hoc track (1993–1999) and Web track (2010–2014) show a clear effectiveness–stability tradeoff across topics and per-topic, and topic grouping and max–min normalization can effectively reduce the bias–variance tradeoff. Experimental results on TREC Session track (2010–2012) also show that the query reformulation and increase of user data are beneficial to both effectiveness and stability simultaneously.     

Mining a Persian–English comparable corpus for cross-language information retrieval

Knowledge acquisition and bilingual terminology extraction from multilingual corpora are challenging tasks for cross-language information retrieval. In this study, we propose a novel method for mining high quality translation knowledge from our constructed Persian–English comparable corpus, University of Tehran Persian–English Comparable Corpus (UTPECC). We extract translation knowledge based on Term Association Network (TAN) constructed from term co-occurrences in same language as well as term associations in different languages. We further propose a post-processing step to do term translation validity check by detecting the mistranslated terms as outliers. Evaluation results on two different data sets show that translating queries using UTPECC and using the proposed methods significantly outperform simple dictionary-based methods. Moreover, the experimental results show that our methods are especially effective in translating Out-Of-Vocabulary terms and also expanding query words based on their associated terms.     

TOFIR: A tool of facilitating information retrieval – introduce a visual retrieval model

The paper introduces a new method for the visualization of information retrieval. Angle attributes of a document are used to construct the angle–angle-based visual space. The retrieved documents are perceived, several traditional information retrieval evaluation models are visualized and interpreted, and new non-traditional retrieval control means based on the model are explored in the two-dimensional angle display space. The impacts of different metrics on the visualization of information retrieval are discussed. Ambiguity, future research directions and other relevant issues are also addressed.     

Internet users’ perceptions of information sensitivity – insights from Germany

With the ever-increasing collection of user data, online privacy becomes an urgent matter for users and research across borders. The perception of information sensitivity is central to privacy attitudes and behaviors in different usage contexts. In an online questionnaire, n = 592 German internet users evaluated how sensitive they perceive 40 different data types. The German sensitivity evaluations revealed in this study are compared to results from the US and Brazil (Markos et al., 2017), in order to understand the cultural impact on evaluations. Additionally, we analyze how attitudes and demographic characteristics of the German sample influence the perception of sensitivity on an individual level. Some distinct differences in sensitivity perception between Germany, Brazil, and the US can be observed, but the rank orders of sensitivity of data types is very similar between the countries, indicating that there is a consensus on what constitutes sensitivity across nations. On an individual level, disposition to value privacy, risk propensity, and education level influence the perception of sensitivity. The findings contribute to an understanding of how to design information and communication strategies to inform internet users how to manage their data carefully.     

Bias–variance analysis in estimating true query model for information retrieval

The estimation of query model is an important task in language modeling (LM) approaches to information retrieval (IR). The ideal estimation is expected to be not only effective in terms of high mean retrieval performance over all queries, but also stable in terms of low variance of retrieval performance across different queries. In practice, however, improving effectiveness can sacrifice stability, and vice versa. In this paper, we propose to study this tradeoff from a new perspective, i.e., the bias–variance tradeoff, which is a fundamental theory in statistics. We formulate the notion of bias–variance regarding retrieval performance and estimation quality of query models. We then investigate several estimated query models, by analyzing when and why the bias–variance tradeoff will occur, and how the bias and variance can be reduced simultaneously. A series of experiments on four TREC collections have been conducted to systematically evaluate our bias–variance analysis. Our approach and results will potentially form an analysis framework and a novel evaluation strategy for query language modeling.     

New information‐communication technologies in scientific communication: Implications for third‐world users

Over the past decade, education reform and teacher training projects have spent a great deal of effort to create and support sustainable, scalable online communities of education professionals. For the most part, those communities have been created in isolation from the existing local professional communities within which the teachers practice. We argue that focusing on online technology solely as a mechanism to deliver training and/or create online networks places the cart before the horse by ignoring the Internet's even greater potential to help support and strengthen local communities of practice within which teachers work. In this article we seek guideposts to help education technologists understand the nature of local K-12 education communities of practice--specifically their reciprocal relationship with teacher professional development and instructional improvement interventions--as a prerequisite to designing online sociotechnical infrastructure that supports the professional growth of education professionals.     

Mobile users’ information privacy concerns and the role of app permission requests

Recent privacy-related incidents of mobile services have shown that app stores and providers face the challenge of mobile users’ information privacy concerns, which can prevent users from installing mobile apps or induce them to uninstall an app. In this paper, we investigate the role of app permission requests and compare the impact on privacy concerns with other antecedents of information privacy concerns, i.e., prior privacy experience, computer anxiety, and perceived control. To test these effects empirically, we conducted an online survey with 775 participants. Results of our structural equation modeling show that prior privacy experience, computer anxiety, and perceived control have significant effects on privacy concerns. However, concerns for app permission requests have approximately twice as much predictive value than the other factors put together to explain mobile users’ overall information privacy concerns. We expect that our findings can provide a theoretical contribution for future mobile privacy research as well as practical implications for app stores and providers.     

IS professionals’ information security behaviors in Chinese IT organizations for information security protection

Continued integration of technology for the purpose of connecting and exchanging data with other devices and systems over the Internet exposes information security (IS) to growing risks. Organizations can thus achieve a strategic advantage by securing IS as a pivotal information and intelligence asset. This study examined ways of motivating IS professionals to protect information security from potential risks, drawing on the theoretical frameworks of protection motivation theory (PMT) and the theory of planned behavior (TPB) as well as work-related organizational antecedents (e.g., organizational commitment and job satisfaction). This paper proposes structural equation modeling (SEM) in R as a framework for exploring relationships among the variables and determining the overall data fit to the hypotheses. SEM is a multivariate technique which simultaneously executes both factor analysis and aspects of multiple regression in order to estimate interrelated relationships while also allowing path analytic modeling to be performed with latent, unobserved variables. Using 804 questionnaires with SEM analysis, we find support for the following predictors’ associations: (a) information security attitudes and subjective norms, as constituents of TPB, significantly influenced information security protective behaviors; (b) the coping appraisals (self-efficacy and response cost) and threat appraisals (threat susceptibility and threat severity) of PMT were significantly predictive of information security protective behaviors; and (c) organizational commitment positively impacted information security protective behaviors. However, job satisfaction and perceived behavioral control as a construct of TPB were not associated with information security behaviors. The main theoretical contribution of this research is that the addition of organizational commitment allows the behavioral science model to offer a novel understanding of IS professionals’ protection motivation and actual behaviors in the Chinese context. This study has several practical implications for organizations. In order to encourage IS professionals to follow protective security behaviors, organizations should set up the belief that a close relationship with subordinates plays a vital role in ensuring information security, improve IS employees’ perception and cognition of their importance to the organization, constantly highlight the importance of information security protection, and emphasize the severe consequences of information security threats during trainings.     

Extending lead user theory to users’ innovation-related knowledge sharing in the online user community: The mediating roles of social capital and perceived behavioral control

The research on users as a source of innovation has been coming into blossom and the studies about the effect of users’ lead userness on their innovation-related activities are drawing more and more attention from both academic and business circles. However, there have been few empirical studies exploring the relationship between users’ lead userness and their innovation-related knowledge sharing behavior in the context of online user community and the mediating effects of users’ social capital and their perceived behavioral control on this relationship. By empirically analyzing the 140 data collected from an online user community that is used as an important source of innovation for a company with the structural equation modeling analysis through the partial least squares method, this study reveals that users’ lead userness has a positive relationship with their innovation-related knowledge sharing in the online user community and users’ social capital and perceived behavioral control jointly and fully mediate this positive relationship. Based on the new findings, this study is expected to provide useful implications which can contribute to widening and deepening the research stream about the effect of users’ lead userness on their innovation-related knowledge sharing in the online user community.     

A new look at Akaike’s Bayesian information criterion for inverse ill-posed problems

Akaike’s Bayesian information criterion (ABIC) has been widely used in inverse ill-posed problems. Little has been done to investigate its statistical aspects. We present an alternative derivation of the marginal distribution of measurements for ABIC under the assumption of normal distributions and show that the principle of ABIC is to statistically estimate the variances of measurements and prior data by maximizing the marginal distribution of measurements. The determination of the regularization parameter with ABIC is essentially equivalent to estimating the relative weighting between measurements and prior data. We prove that ABIC theoretically would produce a biased estimate of the variance of measurements. Since the prior mean is generally unknown but arbitrarily treated as zero in inverse ill-posed problems, ABIC is shown to fail to produce any reasonable estimate for the prior variance. Although ABIC is constructed under the Bayesian framework, it essentially plays more or less the same role as biased regularization from the frequentist’s point of view. ABIC error evaluation cannot be performed under the Bayesian framework but should be more appropriately done with the frequentist’s standpoint in terms of mean squared errors. ABIC is sensitive to prior distributions. In the case of non-informative prior distribution, ABIC leads to the conventional weighted least squares (LS) estimate of parameters and cannot be used to solve inverse ill-posed problems. It is not linked to the regularization parameter but only straightforwardly produces an unbiased estimator for the noise level of measurements, which is only applicable numerically for well-posed problems but not for inverse ill-posed problems. Numerical simulated examples are used to demonstrate the statistical performances of ABIC.     

Investigating consumers’ online social shopping intention: An information processing perspective

Online social media is transforming the way customers communicate and exchange product information with others. Consumers increasingly rely on the opinions and recommendations from social media members when making purchasing decisions. However, information received from social media may have different meanings and social implications for consumers. Based on the theory of informational social influence and heuristic-systematic model (HSM), we develop a model to understand the relative importance of informational social influence, normative social influence, and perceived information quality on the consumer’s social shopping intention under different levels of product involvement. The results of the structural equation modeling (SEM) using a sample of 503 consumers in the Facebook brand fan pages indicate that social influences have a greater impact on the consumer’s social shopping intention than perceived information quality. Three social interactional factors (perceived similarity, familiarity, and expertise) have a positive effect on social shopping intention via the mediation of informational, normative social influence and perceive information quality. The multiple-group analysis suggests that high product-involved consumers are motivated to exert more cognitive effort to evaluate the product information. In contrast, low product-involved consumers are more susceptible to informational social influence. We draw on these findings to offer implications for researchers and practitioners.     

Users’ ability to perceive misinformation: An information quality assessment approach

Digital information exchange enables quick creation and sharing of information and thus changes existing habits. Social media is becoming the main source of news for end-users replacing traditional media. This also enables the proliferation of fake news, which misinforms readers and is used to serve the interests of the creators. As a result, automated fake news detection systems are attracting attention. However, automatic fake news detection presents a major challenge; content evaluation is increasingly becoming the responsibility of the end-user. Thus, in the present study we used information quality (IQ) as an instrument to investigate how users can detect fake news. Specifically, we examined how users perceive fake news in the form of shorter paragraphs on individual IQ dimensions. We also investigated which user characteristics might affect fake news detection. We performed an empirical study with 1123 users, who evaluated randomly generated stories with statements of various level of correctness by individual IQ dimensions. The results reveal that IQ can be used as a tool for fake news detection. Our findings show that (1) domain knowledge has a positive impact on fake news detection; (2) education in combination with domain knowledge improves fake news detection; and (3) personality trait conscientiousness contributes significantly to fake news detection in all dimensions.     

Diverse needs and cooperative deeds: Comprehending users’ identities in online health communities

In recent years, the number of online health communities (OHCs) has increased rapidly as more patients seek to access alternate sources of health information and connect with other patients who have similar health concerns. However, insufficient attention has been paid to investigating user identities in OHCs. To address this potential research gap, by elaborating on the communication theory of identity, this study presents a multi-layered framework to analyze the different layers of user identities that are portrayed in OHCs. Through coding analysis, we discovered that the personal-layer identities that appear in OHCs are patients, partners, offspring, parents, friends and relatives, and others. Moreover, a series of detection models for the personal-layer identities of users were developed, which incorporated content features into machine learning approaches, and they achieved F1-scores above 0.88. Furthermore, we analyzed the features of enactment-layer identities presented by users’ posting behavior and content and the impact of the personal-layer identities of users on the features of the enactment-layer identities. The findings suggested that the features of the enactment-layer identities differed significantly among users with diverse personal-layer identities in terms of both behaviors and communication needs. Users who were identified as patients served as both information seekers and providers, whereas users with the personal-layer identities of parents tended to engage in the community continuously. Our findings extend the understanding of user identities within the context of OHCs.     

Luciano Floridi’s philosophy of information and information ethics: Critical reflections and the state of the art

I describe the emergence of Floridi’s philosophy of information (PI) and information ethics (IE) against the larger backdrop of Information and Computer Ethics (ICE). Among their many strengths, PI and IE offer promising metaphysical and ethical frameworks for a global ICE that holds together globally shared norms with the irreducible differences that define local cultural and ethical traditions. I then review the major defenses and critiques of PI and IE offered by contributors to this special issue, and highlight Floridi’s responses to especially two central problems – the charge of relativism and the meaning of ?entropy’ in IE. These responses, conjoined with several elaborations of PI and IE offered here by diverse contributors, including important connections with the naturalistic philosophies of Spinoza and other major Western and Eastern figures, thus issue in an expanded and more refined version of PI and IE – one still facing important questions as well as possibilities for further development.     

The capability approach and the ‘medium of choice’: steps towards conceptualising information and communication technologies for development

Amartya Sen’s capability approach has become increasingly popular in development studies. This paper identifies controllability and operationalisability as two key stumbling blocks which prevent the capability approach from being used even more widely in development practice. It discusses the origins and application of the Choice Framework, a conceptual tool designed to help operationalise the approach. The framework can be used to deconstruct embedded ideologies and analyse the appropriateness of development goals, to map development as a systemic process, and to plan interventions which can result in increased freedom of choice for people. Three examples of the application of the Choice Framework in the field of information and communication for development (ICT4D) are given. The three technologies which are examined, telecentres (Infocentros), Chilecompra and Fair Tracing, can be placed at different places of a determinism continuum, some reducing the spectrum of choices a user has. The paper argues that while frameworks such as the Choice Framework can be developed further to increase the operationalisability of the capability approach, it is up to development funders to accept the fact that people’s choices are never fully predictable and thus Sen’s ‘development as freedom’ will inevitably be a dynamic and open-ended process.     

Development of a holistic model for the management of an enterprise’s information assets

Organisations increasingly realise that they must transform into true digital enterprises to create competitive advantage and ensure corporate survival. However, many organisations do not realise that successful digital transformation (DT) requires much more than technology; it can only succeed if they manage their data, information and knowledge as true business assets. This paper describes collaborative research conducted by academic and industry partners, a mutually beneficial journey spanning the past ten years. The aim was to develop a Holistic Information Asset Management (HIAM) model indicating the important areas of information asset management (IAM) that support the DT journey. Interviews were conducted with C-level executives in organisations from all industries on three continents to investigate their IAM practices, the barriers to good IAM and the benefits of managing information assets (IAs) well. This paper proposes that organisations should focus on ten domains in their quest for effective IAM: i) business benefits, ii) business environment, iii) executive awareness, iv) leadership and management, v) information environment, vi) information systems, vii) information behaviour, viii) information attributes/quality, ix) information performance and x) justification.     

The reflection of offline activities on users’ online social behavior: An observational study

The ever increasing presence of online social networks in users’ daily lives has led to the interplay between users’ online and offline activities. There have already been several works that have studied the impact of users’ online activities on their offline behavior, e.g., the impact of interaction with friends on an exercise social network on the number of daily steps. In this paper, we consider the inverse to what has already been studied and report on our extensive study that explores the potential causal effects of users’ offline activities on their online social behavior. The objective of our work is to understand whether the activities that users are involved with in their real daily life, which place them within or away from social situations, have any direct causal impact on their behavior in online social networks. Our work is motivated by the theory of normative social influence, which argues that individuals may show behaviors or express opinions that conform to those of the community for the sake of being accepted or from fear of rejection or isolation. We have collected data from two online social networks, namely Twitter and Foursquare, and systematically aligned user content on both social networks. On this basis, we have performed a natural experiment that took the form of an interrupted time series with a comparison group design to study whether users’ socially situated offline activities exhibited through their Foursquare check-ins impact their online behavior captured through the content they share on Twitter. Our main findings can be summarised as follows (1) a change in users’ offline behavior that affects the level of users’ exposure to social situations, e.g., starting to go to the gym or discontinuing frequenting bars, can have a causal impact on users’ online topical interests and sentiment; and (2) the causal relations between users’ socially situated offline activities and their online social behavior can be used to build effective predictive models of users’ online topical interests and sentiments.     

How to approximate users’ values while preserving privacy: experiences with using attitudes towards work tasks as proxies for personal value elicitation

Software users have different sets of personal values, such as benevolence, self-direction, and tradition. Among other factors, these personal values influence users’ emotions, preferences, motivations, and ways of performing tasks—and hence, information needs. Studies of user acceptance indicate that personal traits like values and related soft issues are important for the user’s approval of software. If a user’s dominant personal value were known, software could automatically show an interface variant which offers information and functionality that best matches his or her dominant value. A user’s dominant personal value is the one that most strongly influences his or her attitudes and behaviors. However, existing methods for measuring a user’s values are work intensive and/or interfere with the user’s privacy needs. If interface tailoring for very large groups of users is planned, value approximation has to be achieved on a large scale to assign individualized software to all users of the software. Our work focuses on approximating the dominant values of a user with less effort and less impact on privacy. Instead of probing for a user’s values directly, we explore the potential of approximating these values based on the user’s preferences for key tasks. Producing tailored versions of software is a separate topic not in the focus here. In this paper we rather describe a method to identify user values from task preferences and an empirical study of applying parts of this method. We are proposing the method in this paper for the first time except for a preliminary version orally presented at a workshop. The method consists of a research process and an application process. In the research process a researcher has to identify key tasks occurring in a context under investigation which have a relationship to personal values. These key tasks can be used in the application process to approximate the dominant values of new users in a similar context. In this empirical study we show that the research process of our method allows us to determine key tasks which approximate values in the shared context of nursing. The majority of the nurses were found to have one of the three following dominant values: benevolence, self-direction, or hedonism. Data confirmed common expectations: that nurses with the value of benevolence, when compared to all other nurses, had a higher preference for tasks which helped people immediately or improved their circumstances of the treatment. In relation to all other nurses, participants with self-direction disliked tasks which affected their personal freedom, and users with hedonism had a lower preference for tasks which involved physical work and preferred tasks which promised gratification. Our findings advance measurement of personal values in large user groups by asking questions with less privacy concern. However, the method requires substantial efforts during the initial research process to prepare such measurements. Future work includes replicating our method in other contexts and identifying value-dependent tasks for users with other values than the three values our empirical study mainly focused on.     

Why free does not mean fair: Investigating users’ distributive equity perceptions of data-driven services

Individuals are supposed to perform a privacy risk-benefit analysis when deciding to transact with a free data-driven service provider. Building on equity theory, this article suggests that users incorporate the net value for providers in their trade-off. Based on two pre-studies and an experimental survey study among 200 free data-driven service users, we provide evidence that users’ balance their own net value (benefits minus risks) as well as providers’ net value from monetizing users’ data. This leads to distributive equity perceptions which, in turn, affect users’ satisfaction with the service and thus long-term success of the user-provider-relationship. In this vein, a distributive equity scale for the context of data-driven services is developed. Implications for research, providers and users are discussed.