THE CONCURRENT VALIDITY OF STANDARDIZED ACHIEVEMENT TESTS BY CONTENT AREA USING TEACHERS' RATINGS AS CRITERIA

To assess the concurrent validity of standardized achievement tests using teachers' ratings (and rankings) of pupils' academic achievement as criteria, 42 teachers evaluated each of their students (n = 1,032) in each of five major curricular areas prior to the administration of a battery of standardized achievement tests. The teachers were directed to rate each student's proficiency disregarding attendance, attitude, deportment, and so on. Within-class correlation coefficients were computed to eliminate rater leniency bias. The standardized achievement tests were found to have substantial concurrent validity in reading, math, language arts, science, and social studies. The normalized teacher ranks yielded significantly higher validity coefficients than did the ratings, although the magnitude of the difference was small. The concurrent validity coefficients for language arts, reading, and math were significantly higher than those in science and social studies.     

Likes and Dislikes of Indonesian Pupils

Abstract

An investigation of the hypothesis: Children who select unconventional (U) careers have more creative ability than those who select conventional (C) ones. C or U careers were determined by frequency of occupational choice of 609 sixth grade children. Measures of creativity were the Minnesota Tests of Creative Thinking. Using t tests at the .05 level of confidence, U boys were compared against C boys and U girls against C girls on four different measures of creativity. While ail U groups scored higher than C groups on the creativity measures (with one exception), the results were statistically significant only for nonverbal elaboration. The results partially confirm the hypothesis and simultaneously provide evidence of the validity of the Minnesota tests. The norms on occupational choice contribute to identifying the developmental sequence of career aspirations.     

Are Multiple-choice Items Too Fat?

ABSTRACT

The evidence is mounting regarding the guidance to employ more three-option multiple-choice items. From theoretical analyses, empirical results, and practical considerations, such items are of equal or higher quality than four- or five-option items, and more items can be administered to improve content coverage. This study looks at 58 tests, including state achievement, college readiness, and credentialing tests. The evidence here supports previous assertions. The article also clarifies distractor functioning criteria and offers a typology of items via distractor functioning.     

Disposition assessment in teacher education: A framework and instrument for assessing technology disposition

Abstract

This study proposes and verifies the concept of a technology disposition in teacher education, which is comprised of predisposition and competence. It examines the reliability and validity of a measure of the technology disposition of teacher education students, the Technology Disposition Scale for Teacher Education Students (TDS‐T). The TDS‐T is comprised of two subscales: technology predisposition and technology competence. In this study, the TDS‐T was completed by 656 teacher education students. Results showed the proposed disposition model of predisposition and competence had significant relationships with gender and class year. Male students had significantly higher overall technology disposition scores than did female students, but these differences were due to the male students' strong self‐concept, especially self‐confidence. Students technology competence level was significantly higher for seniors than for sophomores, and the overall technology disposition scores significantly increased from students' junior to senior years. The TDS‐T demonstrated content validity through factor analysis and convergent and discriminate validity through item analysis. The value for Cronbach's alpha was .93, indicating highly satisfactory reliability.     

Reading Frustration Level as Indicated by the Polygraph1

Abstract

A stratified sample of 62 elementary school pupils was monitored by a polygraph as they were administered an informal reading inventory. Evidence was obtained that for these children the mean of ten percent oral reading errors was consistent with the polygraph indication of frustration. Tentative evidence suggests that somewhat more than 50 percent error in comprehension is needed to correspond more closely with polygraph indication of frustration. These results appear in general to support the conventional criteria for determining reading frustration levels for elementary school children.     

Posttraumatic stress disorder and emotion dysregulation among Syrian refugee children and adolescents resettled in Lebanon and Jordan

BackgroundSince the outbreak of the conflict in Syria, many people, including children and adolescents, have fled their homes into neighboring countries. Little research exists on the psychosocial adjustment of refugee children and adolescents resettled in Lebanon and Jordan.ObjectiveThis study investigated the prevalence and predictors of PTSD and emotion dysregulation in Syrian refugee children and adolescents who resettled in Lebanon and Jordan. It was hypothesized that a combination of pre-trauma variables (age and gender), trauma-specific variables (traumatic events and time spent in host country, and host country), and post-trauma variables (coping strategies, family relationships, and school environment) would be associated with PTSD and emotion dysregulation.Participants and SettingParticipants were 1000 Syrian refugee children and adolescents aged 7–18 years attending formal and non-formal schools representing various governorates in Lebanon and Jordan.MethodsThe trauma exposure scale, DSM-IV criteria for the assessment of PTSD, the Difficulties in Emotion Regulation Scale Short Form, Kidcope, Family relationship scales, and school environment scale were administered in an interview format with children and adolescents at school by two trained psychologists. Multivariate binary logistic regression was used to predict PTSD whereas hierarchical multiple regression was used to predict emotion dysregulation.ResultsResults indicated that 45.6% of the refugees have developed PTSD with excessive risk for comorbidity with emotion dysregulation. Emotion dysregulation was reported by older refugee children and adolescents. The prevalence of PTSD was higher in refugee children and adolescents who had resettled in Lebanon than for those who had resettled in Jordan. Refugee children and adolescents who were exposed to higher levels of war atrocities evidenced the greatest prevalence of PTSD and emotion dysregulation. PTSD symptomatology and emotion dysregulation in children and adolescents varied according to coping styles, family relationships, and school environment. Both decreased significantly with the passage of time spent in host country.ConclusionsThe results may be used to formulate cognitive–behavioral coping interventions that can lead to optimal developmental outcomes in the posttrauma environment.     

The Relationship between Assignment Completion and the Attrition and Achievement in Correspondence Courses

Abstract

The relationships between the number of assignments completed by students, whether or not they completed a correspondence course, and their final grades were investigated. The subjects were two separate groups of students enrolled in a correspondence course in fundamental accounting during two consecutive years at the School of Continuing Education, University of Toronto. Students who did more than half of the assignments were significantly more likely to complete the course. Furthermore, the more assignments they submitted, the higher their final examination grade tended to be. Explanation for the observed relationships and suggestions to improve the generally low completion rate in correspondence courses are offered.     

A validation study of the thinking styles inventory: Implications for gifted education

This study examined internal and external validity of the Thinking Styles Inventory (TSI) developed by Sternberg and Wagner (1991) within the framework of Sternberg's (1988) theory of mental self‐government. Participants were 96 adolescent students who were in a summer residential program at the time of testing. The results provide evidence of the external discriminant validity, but only lend partial support to the internal validity of the instrument, suggesting the need to theoretically clarify the relationships between and among posited thinking styles as well as empirically test the validity of these thinking styles measures. Implications of thinking styles for gifted education are discussed.     

Assessing students’ perceptions of fit between secondary and higher education: a validation study of the SPFQ

ABSTRACT

Supporting first-year students to adjust to their new academic environment is a crucial task in higher education. Investigating students’ perceptions of fit between secondary and higher education could give higher education institutions valuable information for student feedback and support, when captured in a reliable and valid way. This study examines the construct validity, reliability, and criterion-related validity of the Students’ Perceived Fit Questionnaire (SPFQ) by using a longitudinal dataset (N?=?930, first-year students). The results of exploratory and confirmatory factor analysis supported a three-factor structure measuring students’ perceptions of: 1) the extent to which secondary education prepared them for higher education in terms of content knowledge; 2) resemblance in the teaching approach between secondary and higher education, and 3) the need to adapt to their higher education study programme. Concerning the criterion-related validity, it was found that first-year students who experienced similarities in teaching approach are more likely to persist in their studies. Furthermore, our results indicated that first-year students who experienced less need to adapt to their higher education study programme felt more self-efficacious regarding their own learning and, consequently, were more likely to persist in their studies.     

The Relationship between General Job Satisfaction and Specific Work-Activity Satisfaction of Community College Faculty

Abstract

Data on job satisfaction of community college faculty, as obtained from five state and national surveys, are analyzed in this study. Results of this analysis indicate that many instructors are generally satisfied with their jobs but somewhat dissatisfied with their working conditions. Examination of the methods used in those studies to measure job satisfaction suggests that the measures may be tapping different dimensions of satisfaction, and that they may not be related to one another or to the criteria measures they are designed to predict. Data from a survey of two-year college humanities faculty (N = 1,493) and nonhumanities chairpersons (N =505) revealed that a measure of general job satisfaction was relatively independent of a measure of specific work-activity satisfaction, and that the former measure of satisfaction was a much better predictor of several indicators of faculty desire to remain at their college than the latter. These findings indicate that different methods of measuring job satisfaction may yield highly discrepant reports, which in turn can pose serious problems of interpretation.     

Scholastic Aptitude and Achievement as Predictors of Performance on Teacher Competency Tests: A Further Investigation

Abstract

In an extension of an earlier study (McPhee & Kerr, 1985), the authors investigated the relationships among scholastic aptitude, achievement, and performance on the secondary form of the Curriculum Proficiency Exam, a state-mandated teacher competency test. Results were similar to findings with the elementary form of the test, indicating little criterion-related validity and some areas in which apparent validity is questionable. Further investigation and/or revision of the test was recommended.     

Relationships Between Teaching and Research

Abstract

This paper reports a study conducted amongst heads of department at a British university into their perspectives of teaching and research. It explores the reasons why research is given higher status, even though academics believe that, in principle, both aspects of the lecturer's role are equally valuable and mutually supportive. The links between teaching and research are investigated, and the categories of ‘teaching’ and ‘research’ are found to be somewhat inadequate concepts for distinguishing between different aspects of the academic role. The article concludes that closer relationships between the two can provide the basis for a programme to improve the quality of university teaching. Such an approach is underpinned by the view that students’ understandings should be taken seriously. They can make a significant contribution to the lecturer's research, as well as to student learning.     

Potential impact of the validated Predicting Abusive Head Trauma (PredAHT) clinical prediction tool: A clinical vignette study

BackgroundThe validated Predicting Abusive Head Trauma (PredAHT) tool estimates the probability of abusive head trauma (AHT) in children <3 years old with intracranial injury.ObjectiveTo explore the impact of PredAHT on clinicians’ AHT probability estimates and child protection (CP) actions, and assess inter-rater agreement between their estimates and between their CP actions, before and after PredAHT.Participants and SettingTwenty-nine clinicians from different specialties, at teaching and community hospitals.MethodsClinicians estimated the probability of AHT and indicated their CP actions in six clinical vignettes. One vignette described a child with AHT, another described a child with non-AHT, and four represented “gray” cases, where the diagnosis was uncertain. Clinicians calculated the PredAHT score, and reported whether this altered their estimate/actions. The ‘think-aloud’ method was used to capture the reasoning behind their responses. Analysis included linear modelling, linear mixed-effects modelling, chi-square tests, Fisher’s exact tests, intraclass correlation, Gwet’s AC₁ coefficient and thematic analysis.ResultsOverall, PredAHT significantly influenced clinicians’ probability estimates in all vignettes (p < 0.001), although the impact on individual clinicians varied. However, the influence of PredAHT on clinicians’ CP actions was limited; after using PredAHT, 9/29 clinicians changed their CP actions in only 11/174 instances. Clinicians’ AHT probability estimates and CP actions varied somewhat both before and after PredAHT. Qualitative data suggested that PredAHT may increase clinicians’ confidence in their decisions when considered alongside other associated clinical, historical and social factors.ConclusionsPredAHT significantly influenced clinicians’ AHT probability estimates, but had minimal impact on their CP actions.     

Overcoming Some Addition Difficulties

Abstract

In literature relating to elementary school reading it is commonly contended that attitudes toward reading influence achievement in reading and that teachers’ attitudes (and achievement) affect pupils’ attitudes and achievement. As yet there is little research support for this contention. In opposition is the view expressed in teacher effectiveness literature that there may be a degree of incompatibility between maximizing in pupils both desired cognitive (or achievement) outcomes and desired affective outcomes (for example, favorable attitudes). The validity of these two positions was assessed by administering reading attitude and achievement tests to 251 final-year student teachers and to the grade 4 through 6 pupils (approximately 900) of 48 of these teachers twice during the following year. One-way analyses of variance were used to investigate teacher-pupil relationships. Findings indicated that high achievement and high attitude in teachers were positively associated with high achievement and high attitudes in pupils; in the pupils of low achievement and low attitude teachers, achievement was low, but attitudes were more favorable than those expressed by the pupils of middle achievement and middle attitude teachers.     

The Effect of Mental-Set or Attitude on the Reading Performance of High-school Pupils

Abstract

This study explored relationships of performance assessments for student teachers among three groups of raters. A 78-item evaluation instrument was administered to 47 student teachers and to their academic and field supervisors. Analysis of the seven subsections of the instrument revealed that student teachers’ self-evaluations were significantly higher, for most categories, than either academic or field supervisors’ ratings. The high degree of agreement between the two types of supervisory ratings on all categories suggest the presence of halo effects.     

Borrowing Norms for the Human Figure Drawing Test: How to validate this practice?

Too often tests are used with clients for whom the validity of the test has not been established. As a case in point we studied the use of the Human Figure Drawing (HFD) test with children living in Curaçao, a small island in the Caribbean. In this community no time and money are available for developing tests and establishing their validity and norms. We suggest that borrowing such information can be a relatively good, inexpensive alternative, provided that clinicians make the best of choices. This paper formulates three requirements, which should be met by the group of clients a clinician is working with. As an example we explored to what extent the requirements are being satisfied by 96 Curaçaoan Grade 4 school children. With regard to these children we conclude that clinicians using the HFD test can best use US representative frequency tables for scoring.     

The potential of anchoring vignettes to increase intercultural comparability of non-cognitive factors

ABSTRACT

We investigate whether Anchoring Vignettes (AV) improve intercultural comparability of non-cognitive student-directed factors (e.g., procrastination). So far, correlation analyses for anchored and non-anchored scores with a criterion have been used to demonstrate the effectiveness of AV in improving data quality. However, correlation analyses are often used to investigate external validity of a scale. Nonetheless, before testing for validity, the reliability of the measurement of a construct should be examined. In the present study, we tested for measurement invariance across countries and languages and compared anchored and non-anchored student-directed self-reports that are highly relevant for the students’ self and their behaviour and performance. In addition, we apply further criteria for testing reliability. The results indicate that the data quality for some of the constructs can – in fact – be improved slightly by anchoring; whereas, for other self-reports, anchoring is less successful than was hoped. We discuss with regard to possible consequences for research methodology.     

Validation of a touch screen tablet assessment of early literacy skills and a comparison with a traditional paper-based assessment

ABSTRACT

Touch screen tablets are being increasingly used in schools for learning and assessment. However, the validity and reliability of assessments delivered via tablets are largely unknown. The present study tested the psychometric properties of a tablet-based app designed to measure early literacy skills. Tablet-based tests were also compared with traditional paper-based tests. Children aged 2–6 years (N?=?99) completed receptive tests delivered via a tablet for letter, word, and numeral skills. The same skills were tested with a traditional paper-based test that used an expressive response format. Children (n?=?35) were post-tested 8 weeks later to examine the stability of test scores over time. The tablet test scores showed high internal consistency (all α’s?>?.94), acceptable test-retest reliability (ICC range?=?.39–.89), and were correlated with child age, family SES, and home literacy teaching to indicate good predictive validity. The agreement between scores for the tablet and traditional tests was high (ICC range?=?.81–.94). The tablet tests provides valid and reliable measures of children’s early literacy skills. The strong psychometric properties and ease of use suggests that tablet-based tests of literacy skills have the potential to improve assessment practices for research purposes and classroom use.     

Some Effects of a Five-Year Developmental Experiment Sponsored by a Private Social Agency in a Public School

The present study investigated the relative validity of the conventional pretest-posttest design and the retrospective pretest-posttest design using self-report measures as outcome criteria. As predicted the experimental training in Seeing Problems Strategy was effective in improving subjects’ task performance. Traditional self-report pre-post comparison did not reflect this actual change of performance, whereas the retrospective self-report approach did. Comparison of pretest mean scores and retrospective pretest mean scores indicated that subjects initially over-estimated their performance. This result represents further evidence on the validity of the response-shift notion as introduced by Howard et al. (8) and supports earlier research in showing that the retrospective pretest is a valid means to control for this source of contamination.     

Child abuse: Betrayal and disclosure

ObjectiveThe current study tested several hypotheses about disclosure of childhood sexual, physical, and emotional abuse derived from Betrayal Trauma Theory [Freyd, J. J. (1996). Betrayal trauma: The logic of forgetting childhood abuse. Cambridge, MA: Harvard University Press]. We predicted that the duration of time from abuse to its disclosure would vary as a function of victim–perpetrator closeness.MethodsData collected from 202 undergraduate participants using a survey methodology were submitted to logistic regression analyses. The relative variance explained by other variables was also examined.ResultsCompared to survivors of emotional abuse (EA) who were in not very close (NVC) victim–perpetrator relationships, EA survivors in very close (VC) victim–perpetrator relationships were significantly more likely to wait 1 or more years to disclose, or never to disclose, than to wait a period of time less than 1 year (OR = 2.65). Further, survivors of physical abuse (PA) in VC victim–perpetrator relationships were significantly more likely to wait 1 or more years to disclose their abuse, if it was disclosed at all, than PA survivors of NVC victim–perpetrator relationships (OR = 3.99). Results for sexual abuse were not significant.ConclusionsFor EA and PA, VC victim–perpetrator relationships predicted longer durations of time from abuse to its disclosure than NVC victim–perpetrator relationships.Practice implicationsAlthough delayed disclosure may support necessary (albeit abusive) attachments with caregivers, it may also prolong the abuse and prevent receipt of support. Increased awareness that VC victim–perpetrator relationships may predict longer durations of time from abuse to its disclosure, and that these delays may serve a functional purpose, can help guide supportive and empathic responses to traumatic disclosures.