A Defense of Publishing Nonsignificant (ns) Results

An argument is advanced for the propositions that a lack of statistically significant findings does not automatically justify rejecting a scientific article for publication or mean that the findings are necessarily uninformative. Systematically declining publication for nonsignificant results leads to negative consequences that include distorting scientific literatures, making hypotheses and theories less falsifiable, depriving meta-analyses of an accurate sample of prior findings, and encouraging questionable research practices. Providing effect sizes and confidence intervals can make findings more informative, regardless of whether the finding is p < .05.     

The value of goodness‐of‐fit indices in meta‐analysis: A comment on Hall and Rosenthal

In a useful article regarding testing for moderator variables in meta‐analysis, Hall and Rosenthal (1991) suggest, among other things, that when the sample of studies is large enough, it is acceptable to use ordinary inferential statistics such as the analysis of variance to test for moderator variables. However, because meta‐analytic data points (effect sizes) differ in important ways from those for which ordinary inferential statistics were designed, we recommend that analysts use conventional meta‐analytic statistics, which are at least potentially more informative about the literature in question. Hall and Rosenthal also suggest that tests of effect size variability should play a minimal role in meta‐analytic model tests. We concur that these tests of homogeneity cannot alone test for moderator variables and that at least tests between mean effect sizes should be performed. However, consistent with Hedges and Olkin's (1985) meta‐analytic framework, we show how homogeneity tests provide additional and valuable information regarding how completely a moderator (or set of moderators) explains the variation of effect sizes. Therefore, these goodness‐of‐fit statistics may prove quite valuable, especially in meta‐analyses of highly variable study outcomes. We further recommend that in meta‐analyses for which study outcomes are already consistent, analysts should continue to perform model tests if they have theoretical expectations about moderators. We discuss these and other meta‐analytic model testing concerns.     

Caveats for using statistical significance tests in research assessments

This article raises concerns about the advantages of using statistical significance tests in research assessments as has recently been suggested in the debate about proper normalization procedures for citation indicators by Opthof and Leydesdorff (2010). Statistical significance tests are highly controversial and numerous criticisms have been leveled against their use. Based on examples from articles by proponents of the use statistical significance tests in research assessments, we address some of the numerous problems with such tests. The issues specifically discussed are the ritual practice of such tests, their dichotomous application in decision making, the difference between statistical and substantive significance, the implausibility of most null hypotheses, the crucial assumption of randomness, as well as the utility of standard errors and confidence intervals for inferential purposes. We argue that applying statistical significance tests and mechanically adhering to their results are highly problematic and detrimental to critical thinking. We claim that the use of such tests do not provide any advantages in relation to deciding whether differences between citation indicators are important or not. On the contrary their use may be harmful. Like many other critics, we generally believe that statistical significance tests are over- and misused in the empirical sciences including scientometrics and we encourage a reform on these matters.     

Questionable Research Practices in Experimental Communication Research: A Systematic Analysis From 1980 to 2013

Questionable research practices (QRPs) pose a major threat to any scientific discipline. This article analyzes QRPs with a content analysis of more than three decades of published experimental research in four flagship communication journals: Journal of Communication, Communication Research, Journalism & Mass Communication Quarterly, and Media Psychology. Findings reveal indications of small and insufficiently justified sample sizes, a lack of reported effect sizes, an indiscriminate removal of cases and items, an increasing inflation of p-values directly below p < .05, and a rising share of verified (as opposed to falsified) hypotheses. Implications for authors, reviewers, and editors are discussed.     

Brief Report: Post Hoc Power,Observed Power,A Priori Power,Retrospective Power,Prospective Power,Achieved Power: Sorting Out Appropriate Uses of Statistical Power Analyses

This brief note hopes to dispel some apparent confusions concerning statistical power, especially ones stemming from power computations based on treating an observed sample effect size as the population effect of interest ("post hoc," "observed," or "retrospective" power). Some after-the-fact power analyses can be useful for presentation purposes but only when based on population effect sizes of independent interest. It is recommended that labels such as "post hoc power," "observed power," "retrospective power," and "a priori power" be avoided, that reported power figures be accompanied by specification and justification of the values used to compute power, and that results characteristically be described with effect sizes, confidence intervals, and p values.     

Peer teaching and information retrieval: the role of the NICE Evidence search student champion scheme in enhancing students’ confidence

Background

This research reports on the NICE Evidence search (ES) student champion scheme (SCS) first five years of activity (2011–2016) in terms of its impact on health care undergraduate students’ information search skills and search confidence.

Objectives

A review of students’ evaluation of the scheme was carried out to chart the changes in attitude towards NICE Evidence search as an online health care information source and to monitor students’ approach to information seeking.

Methods

This study is based on the results of questionnaires distributed to students before and after attending a training session on NICE Evidence search delivered by their own peers. The exercise was implemented in health related universities in England over a period of five consecutive academic years.

Results

(i) Students’ search confidence improved considerably after the training; (ii) ES was perceived as being an increasingly useful resource of evidence based information for their studies; (iii) the training helped students develop discerning search skills and use evidence based information sources more consistently and critically.

Conclusions

The NICE SCS improves confidence in approaching information tasks amongst health care undergraduate students. Future developments could involve offering the training at the onset of a course of study and adopting online delivery formats to expand its geographical reach.     

国内外环境扫描研究综述

[目的/意义] 对环境扫描现有研究进行评述,指出未来研究内容,在丰富环境扫描领域研究成果的同时,为相关学者更好地把握环境扫描研究问题提供理论基础。[方法/过程] 针对国内外环境扫描的研究现状,对环境扫描领域的国内外核心文献进行系统的梳理和分析,重点从环境扫描的研究概况、概念、环境分类及研究维度方面进行综述。[结果/结论] 国外有关环境扫描的研究较多,研究内容全面,我国的环境扫描研究起步较晚,主要集中在图书情报学领域;3种环境分类方法和3种环境扫描维度被广泛地应用于环境扫描研究中;环境扫描的相关研究有待于得到进一步关注。     

Comparing Theories of Media Learning: Cognitive Mediation,Information Utility,and Knowledge Acquisition from Cancer News

Determining what factors predict media learning is an important avenue of research for the field of mass communication. The present study provides a comparative investigation of two models of media learning: the cognitive mediation model and the information utility model. Participants (N = 1,076) read a news article related to scientific discoveries relevant to cancer prevention and responded to all constructs of the two models. Recognition and comprehension were used to measure knowledge acquisition. Results generally support previous predictions of each model, though predicted variance remains small. In addition to testing the existing models, a modified cognitive mediation model using a key construct related to information utility—perceived relevance—was tested. The refined cognitive mediation model offered a more nuanced understanding of certain causal mechanisms but did not result in a meaningful change in predictive power of the model. Implications of the theoretical comparison and integration are discussed.     

Measuring Emotional Appeals in Israeli Election Campaigns

The use of emotions in political campaigns is known and welldocumented. From its earliest days in the Roman Empire or laterin Crusader times to its extensive use by the Nazi regime, politicalpropaganda attempted to seduce and convince, attract and influencetargeted publics by appealing to their emotions. Appeals tofear, hope, pride, anger, and anxiety are frequent in moderncampaigns too. Yet no attempts appear to have been made to developa methodological tool for the objective and systematic identificationof the use of such appeals. This research is based on the design,testing, and validation of a research instrument intended tomeasure quantitatively and comparatively the use of emotionalappeals in two Israeli election campaigns (1996 and 1999). Italso reports findings from an application of the new measureto test specific hypotheses on the use of emotional appealsacross campaigns and parties.     

A study of clinical questions posed by hospital clinicians.

OBJECTIVE: This article presents the results of complementary research studies on the behaviors of hospital clinicians in asking clinical questions and the relationship between asking of questions, outcome of information searches, and success in problem solving. METHODS: Triangulation in research methods--a combination of mailed questionnaires, interviews, and a randomized controlled study--was employed to provide complementary views of the research problems under study. RESULTS: The survey and interviews found that clinical problems (concerned mainly with therapy and equipment or technology) were expressed as statements rather than questions (average number of concepts = 1.7), that only slightly more than half (higher for doctors) of problems could be solved, and that the majority of clinical questions were not well formed. An educational workshop however improved clinicians' formulation of questions, but the use of structured prompting was found to improve building of hypotheses in the doctors' group without training. The workshop also improved satisfaction with the obtained information and success in problem solving. Nonetheless, for both the experimental and control groups, more structured and complete questions or statements did not mean higher success rates in problem solving or higher satisfaction with obtained information. CONCLUSION: The triangulation methods have gathered complementary evidence to reject the hypothesis that building well-structured clinical questions would mean higher satisfaction with obtained information and higher success in problem solving.     

Quantitative evaluation of persuasive appeals using comparative meta-analysis

Message appeals are a key component of communication campaigns and an important source of campaign influence. However, research on them is heavily fragmented and it is difficult to generalize findings from the many diverse field studies. Based on a large and unique data set using quantitative and qualitative meta-analyses, this research provides measures of the relative impact of each type of appeal, as well as the major differences among them, and identifies the moderating variables that lead to a better understanding of each. This investigation is based on persuasion research, which provides a framework for understanding the unique characteristics of these advertising appeals. It highlights empirical gaps in the academic literature and acts as a conceptual guide for our research hypotheses. Results reveal weaker effect sizes than those previously reported in the literature and show important differences among appeals that lead to a “hierarchy of appeals.” Specifically, emotional appeals, led by sex and humor, appear to be more effective than fear and rational appeals. The study finds new theoretical and empirical generalizations; some results are counterintuitive and differ from findings generated from single appeal studies. Findings are of theoretical and practical importance.     

Online Contact,Face-to-Face Contact,and Multilingualism: Young Swedish-Speaking Finns Develop Trilingual Identities

Based on the tenets of the social context model of second language acquisition, the present article examined the combined effect of online and face-to-face contact in developing multilingual skills and identities among young Swedish-speaking Finns (N = 304). The hypotheses were tested for Finnish as a second language and English as a third language using parallel models. The results were largely identical for both languages. Specifically, online contact enhanced language confidence which, in turn, contributed to language identity. However, online contact had a more substantial effect on confidence as well as identity among those who had little face-to-face contact with speakers of the given language. Findings and their implications are discussed.     

A Tutorial on Testing,Visualizing, and Probing an Interaction Involving a Multicategorical Variable in Linear Regression Analysis

Empirical communication scholars and scientists in other fields regularly use regression models to test moderation hypotheses. When the independent variable X and moderator M are dichotomous or continuous, the practice of testing a linear moderation hypothesis using regression analysis by including the product of X and M in a model of dependent variable Y is widespread. However, many research designs include multicategorical independent variables or moderators, such as in an experiment with three or more versions of a stimulus where participants are randomly assigned to one of them. Researchers are less likely to receive training about how to properly test a moderation hypothesis using regression analysis in such a situation. In this tutorial, we explain how to test, visualize, and probe interactions involving a multicategorical variable using linear regression analysis. While presenting and discussing the fundamentals—fundamentals that are not software specific—we emphasize the use of the PROCESS macro for SPSS and SAS, as it greatly simplifies the computations and potential for error that exists when doing computations by hand or using spreadsheets based on formulas in existing books on this topic. We also introduce an iterative computational implementation of the Johnson-Neyman technique for finding regions of significance of the effect of a multicategorical independent variable when the moderator is continuous.     

The Effect of Personal and Academic Variables on Student Adoption of Twitter as an Information Source

ABSTRACT

This study investigated the use of Twitter in terms of personal and academic variables of students. These included gender, grade point average (GPA), and year level. Data were collected from undergraduate students at the College of Social Sciences at Kuwait University. A structured questionnaire was used as the data collection instrument. The findings show that the majority of respondents use social media, 42.8% use Twitter “sometimes,” and 44.4% use it “always.” There is no significant difference between males and females, except for social influence (SI). There is a statistically significant difference in the use of Twitter as an information source among students for different academic year levels, with the highest mean for senior students. However, no significant differences were found by students' GPA scores. Further investigation focusing on variables such as students' level of confidence and skills of using Twitter, such as the effect of course content, etc., would provide more insight into the use of social media platforms in higher education.     

Meta-Analysis of the Persuasive Effects of Metaphorical vs. Literal Messages

This study uses meta-analysis to compare the persuasive effects of metaphorical and literal messages and assess various theoretical explanations for the relatively greater persuasiveness of metaphor. The meta-analysis includes studies from 2001 to 2015 that use various message formats and topics. Results indicate an overall effect size of r = 0.09, p < .001, 95% CI [0.06, 0.12], which supports the conclusion that metaphorical messages are more persuasive than literal messages. Moderator analyses indicate differences in effect sizes based on metaphor target familiarity, message topic, and message format. A meta-regression using the theoretical moderators indicates that message format was the strongest predictor of variation in effect size. This study provides implications related to the theoretical mechanism behind the greater persuasive effects of metaphor.     

Opinions and experiences of university faculty regarding library research instruction: Results of a web-based survey at the University of Southern Colorado

In March 2000, all 188 University of Southern Colorado (USC) nonlibrary faculty members were surveyed to determine their needs and opinions regarding library research instruction. Forty-four (23.40%) of the faculty responded to questions concerning their past experiences with library research instruction, the frequency with which they use various library and Internet resources, the types of library research instruction they currently use with their students, and their level of support for proposed new programs. Results showed an overwhelming interest on the part of respondents for an online library research skills tutorial. The survey also revealed the need for professional development opportunities for faculty. A higher proportion of respondents with 10 years or less of teaching experience than those with more than 10 years of experience indicated they had received formal library instruction from a librarian. However, no direct correlation could be shown between years of teaching experience and respondents' perceived value of library research instruction. Nor did a correlation exist to show that respondents with fewer years of teaching experience were more likely to ask a librarian to provide formal library instruction to their students. In fact, respondents with more than 20 years of teaching experience provided most types of library instruction more often than less experienced respondents. Overall, USC faculty rated their students' abilities to conduct library research very low. However, they had slightly more confidence in their students' abilities to find, evaluate, and use information they found on the Internet over information from traditional print sources.     

iSchools: 为21世纪信息挑战做好准备——iSchools发展历程及最新进展

本文通过梳理iSchools的发展背景,阐述了iSchools学院跨学科的特征;回顾历届年会以及聚焦2012年第七届I-conference,展现了社会上关于信息、技术与人们具体应用的问题。并总结了iSchools的发展特点及趋势。     

How Can the University Library Better Meet the Information Needs of Research Students? Experiences From Ulster University

This article investigates academic library needs of doctoral students. The study identifies PhD students' information literacy training needs and explores current levels of library engagement, barriers to use, and gaps in existing services. First-year PhD students at Ulster University (UU) were surveyed and interviews were undertaken with three students. Findings show that just over half the respondents start their research from the Library's e-journal interface or library databases and these resources are also their main research tools. Minimal use was made of social media or apps. Students' ability to correctly identify different source types within a reference did not match the confidence they expressed in their own abilities, and two thirds were not aware of basic e-book capabilities. However, over 90% of the respondents strongly agreed that the university library service was essential for their research. The article concludes with a number of recommendations on improving library services for PhD students.     

An experimental study of public trust in AI chatbots in the public sector

This study investigates the public's initial trust in so-called “artificial intelligence” (AI) chatbots about to be introduced into use in the public sector. While the societal impacts of AI are widely speculated about, empirical testing remains rare. To narrow this gap, this study builds on theories of operators' trust in machines in industrial settings and proposes that initial public trust in chatbot responses depends on (i) the area of enquiry, since expectations about a chatbot's performance vary with the topic, and (ii) the purposes that governments communicate to the public for introducing the use of chatbots. Analyses based on an experimental online survey in Japan generated results indicating that, if a government were to announce its intention to use “AI” chatbots to answer public enquiries, the public's initial trust in their responses would be lower in the area of parental support than in the area of waste separation, with a moderate effect size. Communicating purposes that would directly benefit citizens, such as achieving uniformity in response quality and timeliness in responding, would enhance public trust in chatbots. Although the effect sizes are small, communicating these purposes might be still worthwhile, as it would be an inexpensive measure for a government to take.     

Application of adaptive boosting (AdaBoost) in demand-driven acquisition (DDA) prediction: A machine-learning approach

Demand-driven acquisition (DDA) programs are playing an increasingly important role in academic libraries. However, the literature surrounding this topic illustrates the wide-ranging, and frequently unpredictable, results of DDA implementation. As uncertainty abounds, librarians continue to seek out deeper understandings of those processes driving the use and purchase of DDA materials. Implicit in this search is a desire to understand how local environmental factors and user preferences dictate broader collection use and purchasing patterns. A small number of these studies have sought deeper insights through predictive modeling, though success has been limited. Following this line of inquiry, this study explores how machine learning might enable more effective collection development and management strategies through the predictive modeling of complex collection use and purchasing patterns. This research describes a replicable implementation of an adaptive boosting (AdaBoost) model that predicts the likelihood of DDA titles being triggered for purchase. The predictive capacity of this model is compared against a more traditional logistic regression model. This study's results show that the AdaBoost model possesses much higher predictive capacity than a regression-based model informed by the same set of predictors. The AdaBoost algorithm, once trained with local DDA data, provides accurate predictions in 82% of cases.