Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (REF)

Altmetrics have been proposed as a way to assess the societal impact of research. Although altmetrics are already in use as impact or attention metrics in different contexts, it is still not clear whether they really capture or reflect societal impact. This study is based on altmetrics, citation counts, research output and case study data from the UK Research Excellence Framework (REF), and peers’ REF assessments of research output and societal impact. We investigated the convergent validity of altmetrics by using two REF datasets: publications submitted as research output (PRO) to the REF and publications referenced in case studies (PCS). Case studies, which are intended to demonstrate societal impact, should cite the most relevant research papers. We used the MHq’ indicator for assessing impact – an indicator which has been introduced for count data with many zeros. The results of the first part of the analysis show that news media as well as mentions on Facebook, in blogs, in Wikipedia, and in policy-related documents have higher MHq’ values for PCS than for PRO. Thus, the altmetric indicators seem to have convergent validity for these data. In the second part of the analysis, altmetrics have been correlated with REF reviewers’ average scores on PCS. The negative or close to zero correlations question the convergent validity of altmetrics in that context. We suggest that they may capture a different aspect of societal impact (which can be called unknown attention) to that seen by reviewers (who are interested in the causal link between research and action in society).     

Exploring e-readers to support clinical medical education: two case studies

Question:

Can e-readers loaded with medical textbooks and other relevant material benefit medical students, residents, and preceptors in clinical settings?

Settings:

The settings are North Carolina community clinics served by Duke University Medical Center and St. Joseph''s Hospital in Bryan, Texas, and Scott and White Memorial Hospital in Temple, Texas.

Methods:

Duke University: Twenty second-year medical students and fourteen family medicine clerkship preceptors used Kindle e-readers in clinics during eight months of rotations. Students and preceptors provided feedback through an anonymous online survey. Texas A&M University: Nine fourth-year medical students in an elective compared medical textbooks in print, online, and on a Kindle. Six residents at a local hospital completed an anonymous online survey after a three-week loan of a Kindle loaded with medical textbooks.

Results:

The e-reader''s major advantages in clinical settings are portability and searchability. The selected e-reader''s limitations include connection speed, navigation, and display. User preferences varied, but online resources were preferred. Participants suggested additional uses for Kindles in medical education.

Conclusions:

The selected e-reader''s limitations may be resolved with further development of the device. Investigation of other e-readers is needed. Criteria for evaluating e-readers in clinical settings should include portability, searchability, speed, navigation, and display. Research comparing e-readers and mobile devices in clinical education is also warranted.     

Distinct Factors Associated With Head Start Mothers’ Self-Report of Perceived Low Positive and High Negative Maternal Expressiveness

Research Findings: There is growing acknowledgment of the need for parenting interventions to address early-onset behavior and emotional concerns. Favorable child outcomes have been linked to parents’ responsiveness and positive expressiveness. Given the theoretical and empirical link between perceptions and actual behavior, Head Start mothers (n = 114) participated in an investigation to examine factors that may be associated with level of self-reported maternal expressiveness. Participants completed the Self-Expressiveness in the Family Questionnaire (Halberstadt, Cassidy, Stifter, Parke, & Fox, 1995). Analysis suggested that self-reported high negative expressiveness was linked with raising a preschooler with perceived internalizing and externalizing behaviors, high parenting stress, and obtaining a post–high school degree. A perceived view of being low in positive expressiveness, in contrast, correlated with different variables, including raising a preschooler with a diagnosed delay, not having any child receive specialized services, raising only 1 child, dropping out of high school, and not receiving behavior advice from Head Start staff. Practice or Policy: Implications for examining self-reported low positive and high negative expressiveness as separate constructs with possibly different intervention pathways are discussed.     

Implementing web 2.0 tools in the classroom: Four teachers’ accounts

In this paper, four teachers shared their experiences using the following free Web 2.0 tools with their students: Jing, Wix, Google Sites, and Blogger. The teachers found that students reacted positively to lessons in which these tools were used, and also noted improvements they could make when using them in the future.     

The h index research output measurement: Two approaches to enhance its accuracy

The h index is a widely used indicator to quantify an individual's scientific research output. But it has been criticized for its insufficient accuracy—the ability to discriminate reliably between meaningful amounts of research output. As a single measure it cannot capture the complete information on the citation distribution over a scientist's publication list. An extensive data set with bibliometric data on scientists working in the field of molecular biology is taken as an example to introduce two approaches providing additional information to the h index: (1) h² lower, h² center, and h² upper are proposed, which allow quantification of three areas within a scientist's citation distribution: the low impact area (h² lower), the area captured by the h index (h² center), and the area of publications with the highest visibility (h² upper). (2) Given the existence of different areas in the citation distribution, the segmented regression model (sRM) is proposed as a method to statistically estimate the number of papers in a scientist's publication list with the highest visibility. However, such sRM values should be compared across individuals with great care.     

The citation speed index: A useful bibliometric indicator to add to the h index

The scientific impact of a publication can be determined not only based on the number of times it is cited but also based on the citation speed with which its content is noted by the scientific community. Here we present the citation speed index as a meaningful complement to the h index: whereas for the calculation of the h index the impact of publications is based on number of citations, for the calculation of the speed index it is the number of months that have elapsed since the first citation, the citation speed with which the results of publications find reception in the scientific community. The speed index is defined as follows: a group of papers has the index s if for s of its N_p papers the first citation was at least s months ago, and for the other (N_p ? s) papers the first citation was ≤s months ago.