Assessment and “The Global Research Village”: Adapting Psychological Instruments

Authors involved in adapting career assessment instruments generally recognize the meaning and nature of the pitfalls and hurdles they encounter in making a test ready for use in a foreign country, culture or language. This article discusses some major obstacles related to the sampling of situations and functions and their ecological relatedness, the compatibility between the conceptual and the operational definitions of variables, the psychometric properties of the test, and the cross-national evidence. In addition there is the question of whether the new reality of globalisation should determine a shift towards intercultural research, and whether common and regional competencies should be integrated to achieve added value and usefulness of assessment techniques?     

The psychometric properties of the Bem sex‐role inventory: Questions concerning reliability and validity

This investigation examined criticisms of the Bem Sex‐Role Inventory, a measure of gender orientation, and further examined its reliability and validity. Study 1 produced a two factor solution which was similar to other analyses. Study 2 demonstrated what adjectives best described males and females in American society. From the two studies, a revision of the Bem Sex‐Role Inventory was proposed with 10 items each measuring masculinity and femininity.     

Academic Self‐concept,Implicit Theories of Ability,and Self‐regulation Strategies

The purpose of the present study is to explore how academic self‐concept and implicit theories of ability are related to four self‐regulation strategies—motivation/diligence, concentration, information processing, and self‐handicapping. The hypothesis is that academic self‐concept and an incremental theory of ability are (1) positively related to motivation/diligence, concentration, and information processing strategies, and (2) negatively related to self‐handicapping strategies. On the basis of inventories 168 teacher students and 60 sport students (a total of 178 females and 50 males) were scored on academic self‐concept, incremental and fixed theories of ability and the four self‐regulation strategies. Multiple regression analysis was used for each self‐regulation strategy as dependent variable, and with academic self‐concept and the ability theories as independent variables. Results revealed that an incremental theory had, as predicted, a positive relation with motivation/diligence and concentration, but had only trivial relations with information processing and self‐handicapping, whereas a fixed theory had only the predicted relation with self‐handicapping. As hypothesised, a high academic self‐concept was positively related to motivation/diligence, conception, and information processing and negatively to self‐handicapping. The findings may indicate that, in order to promote meta‐theoretical processing and prevent student from self‐handicapping, it is important to strengthen academic self‐concept, and to foster an incremental conception of ability among students.     

Assessing the psychometric properties of a scenario‐based measure of achievement guilt and shame

In this study, the psychometric properties of the scenario‐based Achievement Guilt and Shame Scale (AGSS) were established. The AGSS and scales assessing interpersonal guilt and shame, high standards, overgeneralization, self‐criticism, self‐esteem, academic self‐concept, fear of failure, and tendency to respond in a socially desirable manner were completed by 322 undergraduate students. A confirmatory factor analysis indicated that a 12‐scenario model had an acceptable fit to the data, with guilt and shame items forming separate, weakly correlated subscales. Each of the guilt and shame subscales of the AGSS demonstrated good internal and test–retest reliability. Good construct validity was also evident, with each subscale uniquely correlating with constructs in ways that were consistent with predictions. Acceptable discriminant validity was also evident. These outcomes provide support for the utility of the AGSS in assessing guilt and shame reactions in achievement situations.     

Assessing test‐taking strategies of university students: developing a scale and estimating its psychometric indices

Test‐taking strategies are important cognitive skills that strongly affect students’ performance in tests. Using appropriate test‐taking strategies improves students’ achievement and grades, improves students’ attitudes toward tests and reduces test anxiety. This results in improving test accuracy and validity. This study aimed at developing a scale to assess students’ test‐taking strategies at university level. The scale developed was passed through several validation procedures that included content, construct and criterion‐related validity. Similarly, scale reliability (internal reliability and stability over time) was assessed through several procedures. Four samples of students (50, 828, 553 and 235) participated by responding to different versions of the scale. The scale developed consists of 31 items distributed into four sub‐scales: Before‐test, Time management, During‐test and After‐test. To the researcher’s knowledge, this is the first comprehensive scale developed to assess test‐taking strategies used by university students.     

Student‐Teachers' Ability to Read Nature: Reflections on their own learning in ecology

This paper addresses student‐teachers' ability to read nature in a woodland habitat before and after a 10‐week ecology course. Reading nature is our definition of the ability to observe, describe and explain basic ecology in the field. Data consists of field‐based pre‐course and post‐course interviews followed up by metacognitive interviews where students analyse their own learning. A bi‐dimensional coding scheme is adopted to examine the range and development of students' ability to read nature. Students find it important to know the ecology of a few key species and they recognize the importance of having learned the language of ecology — ecologish — helping them to describe and discuss ecology. Students generally recognize the excursions as key learning situations in ecology education but they give different reasons for finding excursions so important. This variation will be elaborated in the paper together with the implications for teaching ecology.     

The little emperor: Chinese parents’ assessment of their own,their partner’s and their only child’s intelligence

This study set out to examine whether Chinese parents, more than people from other nations, over-estimate the intelligence of their son (little emperor) compared to their daughter. In this study, 155 pairs of married couples from mainland China estimated their own, their partner’s and their only child’s overall intelligence and 13 “multiple intelligences.” They also completed a short measure of the Big Five personality traits. Replicating previous studies, fathers rated themselves higher than mothers for almost all types of intelligence. Results revealed, however, no parental sex differences in the estimates of children’s multiple intelligences. Both parents attributed higher overall, verbal, and musical intelligence, but lower existential intelligence to their child than to themselves. Multiple regressions indicated that estimates of verbal, logical–mathematical, spatial and intrapersonal intelligence were the best predictors of estimates of overall intelligence. Participants’ openness and neuroticism were significant predictors of self-rated intelligence. The results were interpreted in relation to specific Chinese social and cultural influences, though there seemed to be no “little emperor” effect.     

Ability Grouping: year group differences in self‐concept and attitudes of secondary school pupils

The vast international literature on the effects of different types of ability grouping on self‐concept and pupils' attitudes towards school has focused on pupils from single year groups. This article aimed to explore year group differences in pupils' self‐concept and attitudes towards school as influenced by ability grouping. The study was cross‐sectional in design and measured attitudes towards school and mathematics, general and school self‐concept and preferences for different kinds of grouping in 234 pupils from years 7 through 10 attending a mixed comprehensive school. The findings indicated that mathematics self‐concept was consistently lower than school and general self‐concept; all aspects of self‐concept increased up to Year 9 and then declined in Year 10, pupils in the higher sets tended to have higher self‐concepts; pupil preferences for setting were greater in the higher year groups, as was the extent to which the top set was perceived to be the best. A substantial proportion of pupils wanted to move set, usually to a higher set. Set placement itself did not appear to have a consistent effect on attitudes towards mathematics. This seemed to be mediated by the quality of teaching.     

Multiple‐goal learners and their differential patterns of learning

The hypothesis in this study was that different types of multiple‐goal learners would have different patterns of learning. A sample of 797 adult distance learners enrolled in different programs offered by a distance learning university in Hong Kong completed a questionnaire assessing their goals, use of strategies, motivational beliefs, and attitudes towards the course they were doing. Two‐stage cluster analyses found a group of single‐goal learners (mastery focused) and three groups of multiple‐goal learners with different focuses in their goal profiles: performance focus, work focus, and multiple focuses. These four clusters of learners differed in terms of use of learning strategies, regulatory strategies, motivational beliefs, and attitudes towards the course. Learners focusing on work‐related goals or performance goals achieved better examination results than did those focusing on multiple goals or solely on mastery goals.     

The Local Benefits and Burdens of Large‐scale Portfolio Assessment

This paper examines research on the use of portfolios in large‐scale assessments in the US. Overall, the research indicates that portfolio assessments produce many of the desired instructional benefits. However, the assessments also place considerable burdens on teachers, and these burdens do not appear to lessen during the first two or three years of implementation.

The author raises concerns about the long‐term sustainability of portfolio assessments without improvement in student performance, the use of portfolios in high stakes accountability contexts, and the difficulty of aligning specific scoring procedures with broader curricular goals.     

Differences in Self‐Concept among Children with Mathematics Disabilities and their Average and High Achieving Peers

Self‐concept ratings of children with mathematics disabilities (MD), average mathematics achievement (AA), and high mathematics achievement (HA) who attended regular classes in grades 4 through 6 were compared. Twenty‐four children in each group, who were selected from an original pool of 811 children, and who were matched one‐to‐one by grade, gender, and nonverbal ability participated in the study. The Culture‐Free Self‐Esteem Inventories (Battle, 1992 Battle J 1992 Culture‐Free Self‐Esteem Inventories: Examiner's manual (2nd ed.), Austin, TX:, Pro‐Ed  [Google Scholar]), a nonverbal ability test, and mathematics achievement tests were administered to all children. Results indicated a marginally significant difference in mathematics self‐concept between the MD and AA groups but no difference between the AA and HA groups. However, children with MD rated themselves more negatively than their HA peers on mathematics, academic, and global subscales. Interestingly, the group differences on academic and general self‐concept measures did not emerge when variations in mathematics self‐concept were controlled, limiting the difference between the MD and HA groups to mathematics self‐concept only. The results suggest that by focusing on children with a specific learning disability and by using domain‐specific academic self‐concept measures, the group differences could be shown to be limited to a subject‐specific self‐concept that is pertinent to the children's area of disability.     

Artistic and Verbal Creative Capacity of 6‐year‐old Children and their Psychopedagogic and Social Conditioning

The subject of the study comprised individual differences of creative achievements among 6‐year‐old children in two basic and qualitatively different kinds of child's activity: verbal and artistic. The aim of the research was both the analysis of the creative capacity level differentiation and pointing out the hierarchy of factors which determine certain inter‐ and intra‐individual as well as inter‐ and intro‐group differences.

The study is based on the concepts of the creativity theory of Wallach and Kogan (1970) and the capacity theory of Renzulli (1986) and Monks (1981) developed within the framework of humanistic and cognitive psychology.

The researcher analysed 6,000 drawings and the same number of oral statements produced by 643 children attending two types of pre‐schools and living in three different habitats. The research also covered teachers working with the 6‐year‐old children.

The following were used in the study: the test ‘Draw‐a‐Man’ of Goodenough; Guilford's tests of divergent thinking; Uszynska's Questionnaire of self‐estimation; The Schaefer's and Aaronson's Classroom Behaviour Inventory Pre‐school to Primary Scale; Popek's Questionnaire of Creative Behaviour ‘KANH’, and Uszynska's Questionnaire for parents.

The results of the research confirm a great differentiation of creative capacity levels of children and suggest that the most significant factors determining them are the pedagogic ones, which may directly or indirectly be influenced by the preschool teacher.     

Assessment preferences,learning orientations,and learning strategies of pre‐service and in‐service teachers

The importance of developing effective learning strategies and motivational beliefs has been widely acknowledged as a way of meeting the demand to acquire lifelong learning capabilities for successful functioning as professionals in the ‘information age’. The study reported in this paper examined the learning orientations and strategies of prospective teachers as well as their assessment preferences and compared them with those of in‐service teachers. The Motivated Learning Strategies Questionnaire (MSLQ) and the Assessment Preferences Inventory (API) were used to test the hypothesis that in‐service teachers will exhibit a deeper approach to learning and assessment due to their constant engagement in meaningful learning experiences. The results confirmed the hypothesis and their implications for teacher education programmes were then discussed.     

Group Assessment in Systems Analysis and Design: a comparison of the performance of streamed and mixed‐ability groups

The results from a controlled experiment in methods of group formation for a group assessment exercise on a second‐year systems analysis and design module are presented. The experiment was undertaken in response to analysis of module results from the previous 3 years. In the group assessments the members of each group received the same group grade. The experiment compared the performance of streamed and mixed‐ability groups. The results are analysed and discussed. It shows that high ability students obtain considerably lower grades in mixed‐ability groups than in streamed groups whereas the reverse is the case for students at the lower end of the ability range. The study also indicates that lower ability students do better in subsequent examinations after having worked in mixed‐ability groups than those in streamed groups whereas the reverse is the case for higher ability students. The implications of the results and directions for future research are discussed.     

Students’ confidence in their performance judgements: a comparison of different response scales

We report results of two studies on metacognitive accuracy with undergraduate education students. Participating students were asked to judge their personal performance in a multiple-choice exam as well as to state their confidence in their performance judgement (second-order judgement [SOJ]). In each study, we compared four conditions that differed in the type of the presented 5-point confidence scale for SOJs. In Study 1, four bipolar scales with different labels were applied; in Study 2, unipolar and bipolar scales were implemented. The results of Study 1 with N?=?420 students show that undergraduates on average provided accurate performance estimations. However, students were not aware of their judgement accuracy, shown by a low fit of SOJ and judgement accuracy. In addition, the type of provided scale significantly influenced the SOJs. Study 2 with N?=?348 students replicated the findings of the first study and gave further insight into the effects of uni- vs. bipolar response scales.     

Accuracy of students' self‐assessment and their beliefs about its utility

The purpose of the two studies presented here was to evaluate the accuracy of students' self‐assessment ability, to examine whether this ability improves over time and to investigate whether self‐assessment is more accurate if students believe that it contributes to improving learning. To that end, the accuracy of the self‐assessments of 3588 first‐year students enrolled in a post‐secondary institution was studied throughout a semester during which each student made approximately 80 self‐assessments about his or her own learning process. These self‐assessments were then compared with multiple judgements by peers and tutors. The overall correlations between the scores of self‐, peer and tutor assessments suggest weak to moderate accuracy of student self‐assessment ability. The findings also reveal an ability effect; students judged as more academically competent were able to self‐assess with higher accuracy than their less competent peers. Comparing the accuracy of student self‐assessment averaged over four consecutive periods indicates that the accuracy does not improve over time. In a second study, a questionnaire aimed at eliciting student's beliefs about the effects of self‐assessment on their learning was administered to 936 first‐year students. Based on their responses, sub‐groups of students were identified: those who either believed in the usefulness of self‐assessment or did not. Results suggest that there is no significant association between student beliefs about the utility of self‐assessment and the accuracy of their self‐assessments.     

Self‐concepts of students with learning disabilities and their normally achieving peers: a review

In an attempt to test the assumption that children with learning disabilities (LD) have deficient self‐concepts, a number of studies have compared the self‐concepts of students with learning disabilities and their normally achieving (NA) peers. The purpose of this paper is to review recent studies that investigated the academic, social and general self‐concepts of students with LD and their NA peers and compare the results with those of a previous meta‐analysis of relatively older studies, by ­Chapman. Consistent with earlier findings, results of the present review indicated that the academic self‐concept of LD students is more negative than that of their NA peers. Unlike Chapman’s conclusion, however, the evidence is less clear for general self‐concept. This is also true for social self‐concept. Because the evidence that shows no group differences outweighs that indicating better ­social and general self‐concept scores for NA children, the conclusion that children with LD hold more negative social and general self‐concepts than do NA children is not warranted.     

Bodily practices and discourses of hetero‐femininity: girls' constitution of subjectivities in their social transition between childhood and adolescence

In foregrounding how girls use their bodies when negotiating subjectivities as adolescents, this paper explores possible ways of doing, being and becoming an adolescent girl in urban, multicultural school contexts. Drawing on interview material generated through a longitudinal qualitative study of children's social transitions between childhood and adolescence, the paper more specifically examines how girls constitute subjectivities through bodily practices which position them according to dominant discourses of hetero‐femininity. The paper goes on to argue that the way in which girls negotiate these discourses through practices on and with their bodies has a significant impact on how they subsequently experience their social transition between childhood and adolescence.