An update on peer review and research data

Key points

• Technological advances in the amounts of data that researchers generate and use are causing problems for the scholarly communication system.
• How, when and by whom should quality checks and assurance be integrated into this – already overloaded – ecosystem?
• This paper outlines the challenges, illustrates some current initiatives and posits possible directions for the future.

     

Study experiences and the post-study intentions of female international undergraduate students

The number of female international students is increasing exponentially, and whilst international study may engender many benefits and challenges, little is known about their intentions once they complete their studies. This article reviews the literature on female international students with specific focus on exploring post-study intentions. A scoping review of four electronic databases was completed. After applying criteria to determine suitability, 30 publications were included in the final review. Analysis revealed three key foci: (1) the rationale for studying internationally; (2) the study experience; and (3) post-study intentions. The literature illustrates that an international study experience has the potential to be a powerful transformative opportunity if positive experiences outweigh the negatives. The findings also indicate that the post-study intentions of female international students are under-researched. The article contends that attention should be given to supporting the needs of this group, with a view to maximizing post-study opportunities.     

Methodological concerns: the feeling-of-knowing task affects resolution

In traditional feeling-of-knowing procedures, participants make judgments on unrecalled items only (e.g. Hart 1965). However, many researchers elicit feeling-of-knowing judgments (FOKs) on all items. When FOKs are made on all items, participants may use recall as a basis for judgments, leading to higher magnitude judgments for recalled items, but causing a relative floor effect for judgments for unrecalled items. We suspected that resolution (relative accuracy) would be better when FOKs are made on all items than when they are made on unrecalled items only. We examined the issue by comparing across studies, reanalyzing data from another experiment, and by conducting an original experiment. In the literature review, we included 83 conditions across 52 studies. We found that feeling-of-knowing judgments made on all items showed higher resolution than feeling-of-knowing judgments made on unrecalled items. This was replicated in the reanalysis of existing data of a single study that used both methods. In the original experiment, we collected feeling-of-knowing judgments for general-information questions. The experiment confirmed that resolution for predicting recognition was higher when feeling-of-knowing judgments were made on all items than when they were made only on unrecalled items. We discuss both methodological and theoretical implications of these data.     

Job satisfaction among academic staff: An international perspective

This study examined aspects of academics'satisfaction with theirjob across the eight nations (Australia, Germany, Hong Kong, Israel, Mexico,Sweden, UK, USA). Interesting patterns emerged across countries reflectingdifferences in the international academic climate. The study also exploredpatterns of job satisfaction and dissatisfaction more closely for theAustralian data, and examined the impact of context elements, includingworking climate and atmosphere, on general levels of job satisfaction.Results indicated that factors related to the environment in which academicswork, including university atmosphere, morale, sense of community, andrelationships with colleagues, are the greatest predictors of jobsatisfaction. Implications for university management and governing bodiesare discussed.     

Electronic portfolios and learner identity: an ePortfolio case study in music and writing

Although the employability of graduates is of concern across further and higher education it is particularly problematic in the arts disciplines, from which few students transition to a traditional, full-time position. Arts graduates shape their work to meet personal and professional needs, and the successful negotiation of this type of career requires a strong sense of identity and an awareness of diverse opportunities. The challenge for educators is how we might develop these capacities whilst being mindful of students’ dreams, which are often focused on artistic excellence and recognition. This paper reports findings from a collaborative study undertaken at four Australian universities. With a focus on developing an electronic portfolio (eP), the study involved students in classical and contemporary music, music education, music technology, creative writing and professional writing. The combination of music and writing provided points of comparison to identify issues specific to music, and those that might apply more generally. This paper reports findings related to learner identity, drawing evidence from survey and interview data. The study, which was driven by the learning process rather than the technological tool, revealed that students’ use of eP transitions from archive to self-portrait. Moreover, the eP emerged as a vehicle through which identity is negotiated and constructed. Indeed, the process of developing of an eP prompted students to adopt future-oriented thinking as they began to redefine their learning in relation to their future lives and careers. These findings were common to all students, regardless of discipline or technological platform.     

Support for assessment practice: developing the Assessment Design Decisions Framework

There are many excellent publications outlining features of assessment and feedback design in higher education. However, university educators often find these ideas challenging to realise in practice, as much of the literature focuses on institutional change rather than supporting academics. This paper describes the conceptual development of a practical framework designed to stimulate educators’ thinking when creating or modifying assessments. We explain the concepts that underpin this practical support, including the notions of ‘assessment decisions’ and ‘assessment design phases’, as informed by relevant literature and empirical data. We also present the outcome of this work. The Assessment Design Decisions Framework. This provides key considerations in six categories: purposes, contexts, tasks, interactions, feedback processes and learning outcomes. By tracing the development of the Framework, we highlight complex ways of thinking about assessment that are relevant to those who design and deliver assessment to tertiary students.     

The research grant application process. Learning from failure?

Increasing competition for federal government research funds has resulted in a large number of good projects not being funded. This situation is unlikely to change in the near future and has generated uncertainty and frustration amongst many who are dependent on external funding for their research. In this context it is particularly important that the aims of federal government funding agencies are communicated effectively and that the procedures they establish to allocate research funds are seen as credible by the academic research community. This article reports the results of a survey which investigated the research grant process from the point of view of unsuccessful applicants from four universities for large 1991 initial Australian Research Council grants. The findings identify a number of limitations in the operations of the peer review mechanism as used by this Council and question the adequacy of the advice and instructions provided by the ARC to those nominated to review research proposals. The findings also raise questions concerning how the lists of external assessors are compiled as well as how these external assessors are later matched with individual applications.     

Transitions in Leisure Careers and Their Parallels in Work Careers: The Effect of Constraints on Choice and Action

Careers are increasingly characterized by multiple jobs and employers, and thus the issue of transitions between career components and through career stages is of greater importance than in the past. Transitions have also been extensively studied in the context of serious leisure. This paper uses findings from the literature on serious leisure and leisure constraints to develop propositions about the role of transitions in work careers and the effects that constraints have on transitions.