The validity and reliability of focus groups as a research method in adult education

The article examines the theory and practice of focus groups in adult education research. Three theoretical positions are described: radical hermeneutic, moderate interpretative and pragmatic realistic position trying to bridge the gap with positivist research. This last position has been chosen as the departure point for a further analysis of the focus group elements, which relate to validity and reliability. The research examines four European research projects in the field of adult education. The most important and surprising data are presented with reference to both socio-psychological and technical problems in the use of this method. The results highlight the importance and dynamics of all the elements discussed.     

Establishing the validity and reliability of a modified tool for assessing innovative thinking of engineering students

Abstract

The goal of this study was to establish the validity and reliability of a self-report tool for assessing individual differences in innovative thinking, based on a modification of the ‘innovative behavior scale’. A five-stage study was conducted among engineering students worldwide, who enrolled in a massive open online course in Nanotechnology and Nanosensors. Content validity was established by a panel of experts in engineering education. Construct validity was established through exploratory and confirmatory factor analysis, indicating a four-factor solution with 13 items loaded above critical level. Known-groups validity revealed differences among learners with distinct expertise. The scale’s stability across populations and over time was confirmed, providing there has been no intentional intervention. Finally, the scale’s concurrent validity was reinforced through data triangulation with actual practice - the fabrication of innovative nanosensors.     

Alternate form reliability of the PPVT-R for white and black preschool children

Alternate forms of the PPVT-R were compared for 72 preschool children. Pearson product moment correlation coefficients for whites, blacks, males, females, and for the total sample were significant; values ranged from .74 to .86. Using standard scores for analyses, differences between Form L and Form M mean scores were nonsignificant for whites, males, females, and for the total group. However, for black preschoolers, Form L was apparently more difficult to complete successfully than Form M.     

A method for assessing the treatment validity of tests in special education

Several tests in special education are used as a basis for describing education treatment. When tests are used in this fashion, it is necessary to assess their treatment validity, i.e., to assess the degree to which they enable educators to accurately prescribe an effective treatment. This article outlines a procedure for estimating the treatment validity of a test and uses two tests as examples.     

Generalization of the Child Observation Record: a validity study for diverse samples of urban,low-income preschool children

The present investigation addressed the construct validity of the Child Observation Record (COR) with low-income urban preschool children. From two separate samples representing low-income preschool children, COR ratings were analyzed using multivariate techniques. Independent analyses from these two urban sites yielded a three-dimensional structure: Cognitive Skills, Social Engagement, and Coordinated Movement. Further analyses cross-validated this structure for males and females and across ethnic groups. Concurrent assessments provided convergent and discriminant validity for the Social Engagement dimension and convergent validity for Cognitive Skills dimension. Analyses of item distributions of the 5-point developmental sequences represented by the 30 COR items were used to examine the assumption that all the distributions were continuous unimodal distributions. Findings did not universally support this assumption revealing some irregular distributions with troughs in the mid-range of continua. Implications of the findings for early childhood assessment of vulnerable children and future research were discussed.     

幼儿教育新的价值取向：幼儿教育与生活的融合

一、教育和生活的关系一个人一出生，就在这个世界上生活，从他的诞生、发展、成熟、延续直至死亡是一个生长的过程，是一个人生活历程的全部内容。当然生活并不仅仅是生理学意义上的，并不仅仅是指人的生命的内在潜能自然展开的过程，而且还包括人的知识、能力、个性，直至整个精神世界经验的过程。因此生活对于个体来说就可简化为被经验到的一切，个体的生活获得各种经验。而这种经验的增长、组织、更新就是广义的教育。因此教育离不开生活。杜威曾说“教育应当是生活本身，而不是生活的准备”。在杜威看来，教育并不是强制儿童静坐听讲和…     

The reliability and validity of the pupil behaviour questionnaire: a child classroom behaviour assessment tool

ABSTRACT

This paper assesses the reliability and validity of the teacher-completed Pupil Behaviour Questionnaire (PBQ), by comparing it to the already extensively validated teacher-completed Strengths and Difficulties Questionnaire (SDQ). Participants included 2074 primary school children participating in a universal school-based trial and 41 vulnerable children who were taking part in a study exploring the impact of exclusion from school. Exploratory factor analysis results (first factor accounts for 80.8% of the variation in the items) and the high Cronbach’s alpha value of 0.85 indicate that the PBQ consists of one substantive factor/dimension. Strong correlations between the total PBQ score and the conduct sub-scale (Spearman’s correlation coefficient (r_s) = 0.67) and total difficulties score (r_s = 0.59) of the SDQ indicate convergent validity. This study suggests that the PBQ is a reliable measure, and provides some evidence of validity. Further work is needed to test the PBQ in an older, more diverse populations and to measure sensitivity to change.     

Evaluating the reliability and validity of a learning styles inventory: a classroom-based study

Background Psychology and educational theory has a long tradition of research into learning styles. However, the current educational policy and practice interest in learning styles in the UK has resulted in concepts and practices being adopted with little rigorous empirical evaluation.

Purpose This small-scale, experimental study aimed to test the reliability and validity of an available inventory designed to identify learning styles (visual, auditory and kinaesthetic).

Sample, design and methods Nineteen children, aged 7–10 years, with different styles as identified by the inventory, participated in a class experimental evaluation of their differential response to the teaching of word spelling using different teaching methods.

Results The study found that the visual and auditory scales, but not the kinaesthetic scale, were reliable (internally and re-test). The three groups of pupils with different learning styles—visual only, auditory only and mixed visual and auditory—showed different gains to teaching that matched these styles (visual and auditory teaching approaches). Retention of word spelling was higher one week after the teaching when the teaching matched the learning style.

Conclusions The findings in this exploratory study suggest the significance of learning style for classroom-based teaching.     

A validity study of two preschool screening instruments: The LAP-D and DIAL-R

This study compared the Developmental Indicators for the Assessment of Learning—Revised (DIAL-R) and the Learning Accomplishment Profile—Diagnostic (LAP-D) for a sample of 121 children in an urban Head Start program. To examine validity, guidelines suggested by the multitrait-multimethod model (MTMM) were employed. The results indicated significant correlations between like-named scales, providing evidence for convergent validity. However, the within-method correlations often equaled or exceeded the validity coefficients, as did the between-method correlations of dissimilar scales. Thus the simultaneous requirements of both convergent and discriminant validity were not met. From the viewpoint of professional practices, the results suggest that profiles cannot be interpreted with confidence. New instruments and procedures are needed that reflect research and theory associated with the principles of psychosocial Change.     

Interscorer reliability of the modified version of the bender-gestalt test for preschool and primary school children

This study extended the reliability research of the Qualitative Scoring System for the Modified Version of the Bender-Gestalt Test (Modified B-G Test). The test was administered to 48 kindergartners and first graders by a licensed psychologist. The 48 test protocols were scored independently by two psychologists using the Qualitative Scoring System. The two sets of scores were significantly correlated. Results indicated that the scoring system for the Modified B-G is highly reliable.     

The Parent Threat Inventory: development, reliability, and validity

OBJECTIVE: The goal was to develop a retrospective inventory of parental threatening behavior to facilitate a better understanding of such behavior's role in the etiology of psychological distress. METHOD: Inventory items were developed based on theory and 135 students' responses to a question eliciting examples of threatening parental behavior. Following item development, two additional student samples (n = 200 and n = 603) completed batteries of self-report measures. Responses were used to eliminate unstable or redundant items from the inventory and to examine the inventory's psychometric properties. RESULTS: Factor analysis of the inventory revealed three factors, accounting for 66.2% of variance; this factor structure is compatible with theory, and consistent across maternal behavior scores, paternal behavior scores, and combined maternal and paternal scores. Cronbach's coefficient alphas indicated acceptable internal consistency; Pearson correlation coefficients indicated acceptable 4-week test-retest reliability. Moderate intercorrelations with two retrospective measures of childhood experiences suggested construct validity. Regression analyses demonstrated the ability of the inventory to predict both anxious and depressive symptomatology and lifetime symptoms of anxiety and depressive disorder. Normative data on combined parent scores, maternal scores, and paternal scores are also presented. CONCLUSIONS: Initial psychometric testing of the Parent Threat Inventory (PTI) suggests it is a reliable and valid tool for investigating the developmental antecedents of adult psychological distress. Further research should focus on addressing two limitations: (1) lack of normative and psychometric data on men and women suffering from clinical disorders, and (2) lack of validation by parental reporting.     

The assessment of lesson plans in teacher education: a case study in assessment validity and reliability

This paper forms part of an exploration of assessment on one part‐time higher education (HE) course: an in‐service, professional qualification for teachers and trainers in the learning and skills sector which is delivered on a franchise basis across a network of further education colleges in the north of England. This paper proposes that the validity and reliability of portfolio‐based assessment, a key component of many HE programmes in addition to the course being researched here, is contestable. Analysis of the processes of compiling portfolios for assessment, through the conceptual framework of the New Literacy Studies, suggests that the ways in which portfolios are assessed and the ways in which the crucial requisites of validity and reliability are assigned to them, mask complexities and contradictions in their creation by the student. This paper argues for a new, critical analysis of portfolio production and raises a number of questions about the validity, reliability and authenticity of the assessment process that the portfolios reify.     

The reliability and validity of an interactive arithmetic test

A sample of 331 primary school children from seven primary schools, from Years 3 through 7, participated in this study to determine the reliability of a computer resident interactive arithmetic test, using both the method of a single sitting and that of parallel forms. Reliability coefficients up to .96 were calculated for scoring rates by the single sitting methods and up to .87 by the method of parallel forms. The study also attempted to determine the validity of the interactive test by comparing it with the arithmetic subtests of the Key Math Test (KMT, Connolly et al ., 1971). Positive, but weak, correlations up to .65 were recorded, reflecting differences in the medium of administration and assessment methodology of the two tests.