Peer assessment of class participation: applying peer nomination to overcome rating inflation

Having students actively engaged with each other in discussions has become an increasingly important and common aspect of the classroom environment. This increased emphasis has also meant that instructors need to find ways to effectively and efficiently evaluate class participation. In this paper, we describe the most common method used for these assessments and highlight some of its inherent challenges. We then propose an alternative method based on peer nominations. Two case studies illustrate the advantages of this method; we find that it is both easy for students to complete and provide instructors with valuable diagnostic information with which to provide feedback and assign grades.     

‘The constant rejections hurt’: Skills and personal attributes needed to successfully complete a thesis by publication

The thesis by publication (TBP) is an increasingly popular approach to doctoral education that sees candidates publishing their research during their candidature, and including these outputs in their final thesis submission. Responsive to the realities of modern academia, a TBP provides doctoral candidates with opportunities to engage in the scholarly publication process, and to build their research portfolio. As the TBP gains further traction in many fields and countries, there is a need to better understand the skills and attributes needed to complete a TBP successfully. This paper draws on the views of 246 recent doctoral graduates of Australian universities to identify the skills and attributes they felt supported their success. Their responses have implications for the development and refinement of institutional policies and programs that are sensitive to the needs and experiences of doctoral candidates seeking to publish during candidature and create a TBP. As editors, peer‐reviewers, and publishers see a growing influx of papers sole‐ or co‐authored by doctoral candidates in response to the trend toward TBP, they may find themselves involved in supporting students to develop the skills and attributes needed to successfully publish during candidature.     

Applying computerized-scoring models of written biological explanations across courses and colleges: prospects and limitations

Our study explored the prospects and limitations of using machine-learning software to score introductory biology students’ written explanations of evolutionary change. We investigated three research questions: 1) Do scoring models built using student responses at one university function effectively at another university? 2) How many human-scored student responses are needed to build scoring models suitable for cross-institutional application? 3) What factors limit computer-scoring efficacy, and how can these factors be mitigated? To answer these questions, two biology experts scored a corpus of 2556 short-answer explanations (from biology majors and nonmajors) at two universities for the presence or absence of five key concepts of evolution. Human- and computer-generated scores were compared using kappa agreement statistics. We found that machine-learning software was capable in most cases of accurately evaluating the degree of scientific sophistication in undergraduate majors’ and nonmajors’ written explanations of evolutionary change. In cases in which the software did not perform at the benchmark of “near-perfect” agreement (kappa > 0.80), we located the causes of poor performance and identified a series of strategies for their mitigation. Machine-learning software holds promise as an assessment tool for use in undergraduate biology education, but like most assessment tools, it is also characterized by limitations.     

Analysis of integrated STATCOM-SMES based on three-phase three-level multi-pulse voltage source inverter for high power utility applications

This paper is aimed to investigate the operating characteristics of a static synchronous compensator (STATCOM) integrated with superconducting magnetic energy storage (SMES) for high power applications in the transmission network level. The STATCOM controller topology comprises multi-level multi-pulse neutral-point clamped-type (NPC) voltage source inverters (VSIs) using the harmonics cancellation technique, and incorporates a SMES coil. An innovative two-quadrant multi-level dc-dc converter is proposed to effectively interface the STATCOM with the superconducting coil using a buck-boost topology with neutral point voltage control capabilities; thus enabling to simultaneously control both active and reactive power exchange with the high voltage power system. A detailed analysis of major system variables is presented, including analytical results and digital simulations using the MATLAB/Simulink environment. Moreover, a three-level control scheme is designed, including a full decoupled current control strategy in the d-q reference frame with a novel controller to prevent the STATCOM dc bus capacitors voltage drift/imbalance and an enhanced power system frequency controller.     

Conveying quality and value in emerging industries: Star scientists and the role of signals in biotechnology

This paper addresses the role that scientific status plays in initial public offerings of technology focused firms. The paper builds on the literature of the sociology of science as well as the work of Spence (1974) and Podolny (1993) and argues that the presence of a Nobel laureate affiliated with a firm making an IPO provides a signal of firm quality to potential investors. Moreover, and building on the work of Podolny and Scott Morton (1999) and Stuart et al. (1999) we hypothesize that the importance of status diminishes as other measures of firm quality become available. We test our hypothesis for two periods of initial public offerings in biotechnology. We document that there is a clear difference in “maturity” of the firms across the two windows on a number of metrics. Consistent with our hypothesis that Nobel laureates play an important role as a non-financial signal of firm quality, we find that first-window firms with a Nobel laureate affiliate realize greater IPO proceeds in the amount of $24 million. In the second window the amount of money raised is not significantly different between Nobel and non-Nobel firms. This finding is consistent with the signaling literature that argues that the importance of a signal is inversely related to the availability of cogent information on firm quality. Consistent with this view, we also find a change between the two windows in the importance of other non-financial metrics used to convey value. Our research is one of the first to examine the dynamic nature of signals. Because we are unable to distinguish the extent to which the reduction in uncertainty at the firm level is correlated with the reduction of uncertainty at the industry level, the question remains as to the extent to which the diminished importance of signals in our second period is due to a change in market uncertainty versus a change in firm uncertainty.     

Searching for randomised controlled trials and clinical controlled trials in Thai online bibliographical biomedical databases

Background: Thailand is a poor but highly literate country in South‐East Asia with over 60 million people. A lot of biomedical research is undertaken but dissemination is limited. Objectives: To identify relevant Thai bibliographic databases and investigate accessibility, functionality and content, particularly in relation to randomised controlled trials (RCTs) and clinical controlled trials (CCTs). Methods: A systematic search for institutions productive of research and the databases in their libraries. Search each accessible database in both Thai and English, recording the functionality and content. Assess accessibility of the retrieved RCTs or CCTs by comparing to Pub Med holdings. Results: We found 32 different databases (29 accessible in UK) of various sizes, coverage and functionality but many with unique records of RCTs and CCTs (total, n = 781). Two hundred and nine of 781 trials were accessible on Pub Med (27%). However, 641 of the 781 trial records contain text in both English and Thai (82%) and 112 records were solely English (14%). Conclusions: Those undertaking comprehensive searches for RCTs/CCTs should, in addition to a Pub Med , search the Thai Medical Index and Thai Index Medicus databases, and the Khon Kaen University Library Catalogue.     

A validated search assessment tool: assessing practice-based learning and improvement in a residency program

Objective:

The objective of this study was to validate an assessment instrument for MEDLINE search strategies at an academic medical center.

Method:

Two approaches were used to investigate if the search assessment tool could capture performance differences in search strategy construction. First, data from an evaluation of MEDLINE searches from a pediatric resident''s longitudinal assessment were investigated. Second, a cross-section of search strategies from residents in one incoming class was compared with strategies of residents graduating a year later. MEDLINE search strategies formulated by faculty who had been identified as having search expertise were used as a gold standard comparison. Participants were presented with a clinical scenario and asked to identify the search question and conduct a MEDLINE search. Two librarians rated the blinded search strategies.

Results:

Search strategy scores were significantly higher for residents who received training than the comparison group with no training. There was no significant difference in search strategy scores between senior residents who received training and faculty experts.

Conclusion:

The results provide evidence for the validity of the instrument to evaluate MEDLINE search strategies. This assessment tool can measure improvements in information-seeking skills and provide data to fulfill Accreditation Council for Graduate Medical Education competencies.

Highlights

• The University of Michigan MEDLINE Search Assessment tool can be used to assess search skills in residency education.
• Five elements were identified as critical elements in the development of an effective MEDLINE search strategy: inclusion of all search concepts, appropriate use of Medical Subject Headings, appropriate use of search limits, successful combination of all concepts, and search efficiency.

Implications

• This validated assessment tool can serve as an effective means to measure improvements in residents'' information-seeking skills and provide data to fulfill Accreditation Council for Graduate Medical Education competencies.

     

Validation of MET estimates and step measurement using the ActivPAL physical activity logger

The purpose of this study was to determine the validity of the metabolic equivalent (MET) equation and step rate function of the ActivPAL? physical activity logger in a group of females. Using a standard treadmill protocol, 62 females aged 15-25 years walked on a treadmill at speeds between 3.2 and 7.0 km · h(-1) while wearing an ActivPAL. Oxygen consumption was measured using expired gas analysis at each speed and METs for each speed were estimated based on each participant's own resting metabolic rate. A sub-set of 18 participants also wore an Actigraph. Results showed that the in-built equation in the ActivPAL significantly underestimated (P < 0.001) METs under treadmill conditions at higher intensities. The ActivPAL equation is based on step rate yet the relationship between counts and measured METs (r = 0.76; P < 0.001) is stronger than that between steps and measured METs (r = 0.59; P < 0.001). Both the ActivPAL and Actigraph step functions showed no significant difference (P > 0.05) to video recorded step rate except at the slowest walking speed where the Actigraph significantly underestimated steps (P < 0.05). The development of a new equation based on the counts-METs relationship that includes a variety of speeds and activities would be useful. The ActivPAL step function performs better than the Actigraph at the slowest walking speed under treadmill conditions.