Measuring the relationship between two questionnaires: appropriate definitions of intuitive understandings

A number of recent articles have claimed strong relationships—i.e., very high ‘proportions of shared variance’—between pairs of teaching and learning questionnaires. These claims have been the subject of debate and it has emerged that the proportion of shared variance is defined as the complement of Wilks’ lambda. The present article argues that this is not an appropriate measure for the data being analyzed. First some intuitive understandings of the situation are developed. A case study finds that the (Wilks‐based) proportion of shared variance between two questionnaires is 79%, but that the first questionnaire explains only 27% of the variance in the second and the second explains only 40% of the variance in the first. The technical literature is then considered and alternatives to the complement of Wilks’ lambda are recommended as measures of the relationship between two sets of variables.     

Emotionally and motivationally supportive classrooms: A state-trait analysis of lesson- and classroom-specific variation in teacher- and student-reported teacher enthusiasm and student engagement

Teacher enthusiasm and student engagement are often interrelated and have important implications for student learning and students' and teachers' well-being. However, results on the lesson-specific variation of teachers' and students' affective-motivational experiences and their interplay are scarce. This study investigated variation in teacher enthusiasm and student engagement, each rated by teachers (n = 70) and students (n = 1537), as indicators of a shared affective-motivational climate in ninth-grade math classrooms across five consecutive lessons. Multitrait-multistate analyses revealed substantial “trait-like” consistency in all four affective-motivational measures. However, there was also a substantial degree of “state-like” lesson-specific variance that was shared across the four measures. This indicates that teachers' and students’ affective-motivational experiences are shaped by situation-specific influences and person-by-context interactions, which are shared between teachers and students. Teacher gender, teaching experience, class-level achievement, and the availability of motivationally supportive instructional interventions failed to explain substantial variance in these associations.     

Toward a further understanding of teachers’ reports of early teacher–child relationships: Examining the roles of behavior appraisals and attributions

This study examined teachers’ reports of early teacher–child relationships by focusing on their assessments of the severity and the causes of children's social behaviors. Eighty-one kindergarten teachers filled out questionnaires about socially inhibited, hyperactive, and average children (n = 237) selected from their own classes. Multilevel analyses indicated that teachers reported less close and more dependent relationships for the inhibited and hyperactive versus the average children, and more conflictual relationships for the hyperactive versus the average children. These differences were largely mediated by teachers’ perceptions of children's personal behavior problems. In addition, we found that the teachers’ control attributions for children's social behaviors increased the link between children's perceived (personal and social) problems and relationship closeness. Results further support the idea that teachers’ relationship reports are personal, evaluative accounts rather than objective measures of teacher–child interactions.     

Classroom effects are as large as grade-level effects on curriculum-based measurement maze reading scores of secondary school students with and without special educational needs

Background

Previous studies used curriculum-based measurement (CBM) maze scores as an indicator of the reading comprehension level of secondary school students with and without special educational needs in multiple grades, pinpointing a high influence of both student- and context-related variables. However, studies on cumulative influence are necessary for better understanding of data-based decision-making.

Methods

We examined a sample of 1066 secondary school students using four linear mixed-effect models: How much variance in maze scores exists between multiple student characteristics (i.e., gender, immigration background, learning disability and developmental language disorder) and context variables (i.e., classroom, grade and school type) across Grades 5–8?

Results

The intra-class correlation (ICC) results show that the influence by the context-related variable classroom (ICC = .094) is almost as large as by the variable grade level (ICC = .126). School type (i.e., inclusive school vs. special school) has the least influence (ICC = .02). In addition, the effects of student-related variables explain only a small proportion of the variance (marginal R² = .114).

Conclusions

Maze scores can be used as a screening instrument for students with multiple characteristics across grades; they also show that it makes no difference which type of school students attend. As teachers and further classroom-related variables have almost as much influence as grade level, we discuss that teachers can minimise classroom effects by using maze scores as a formative approach.     

Daily fluctuations in young children’s persistence

Children's behavior changes from day to day, but the factors that contribute to its variability are understudied. We developed a novel repeated measures paradigm to study children's persistence by capitalizing on a task that children complete every day: toothbrushing (N = 81; 48% female; 36–47 months; 80% white, 14% Multiracial, 10% Hispanic, 2% Asian, 1% Black; 1195 observations collected between January 2019 and March 2020). Children brushed longer on days when their parents used more praise (d = .23) and less instruction (d = −.22). Sensitivity to mood, sleep, and parent stress varied across children, suggesting that identifying the factors that shape an individual child's persistence could lead to personalized interventions.     

Kindergarten Children's Executive Functions Predict Their Second-Grade Academic Achievement and Behavior

Whether and to what extent kindergarten children's executive functions (EF) constitute promising targets of early intervention is currently unclear. This study examined whether kindergarten children's EF predicted their second-grade academic achievement and behavior. This was done using (a) a longitudinal and nationally representative sample (N = 8,920, M_age = 97.6 months), (b) multiple measures of EF, academic achievement, and behavior, and (c) extensive statistical control including for domain-specific and domain-general lagged dependent variables. All three measures of EF—working memory, cognitive flexibility, and inhibitory control—positively and significantly predicted reading, mathematics, and science achievement. In addition, inhibitory control negatively predicted both externalizing and internalizing problem behaviors. Children's EF constitute promising targets of experimentally evaluated interventions for increasing academic and behavioral functioning.     

Language,ambiguity, and executive functions in adolescents' theory of mind

Executive functions' (EF) role in adolescents' advanced theory of mind (aToM) was examined. In Study 1, adolescents (N = 189 in 2017, M_age = 13.1 years, 55.6% female from racially/ethnically diverse schools) completed the Flexibility and Automaticity of Social Cognition task (FASC), and shifting and inhibition measures. Study 2 (N = 289 in 2018 and 2019, M_age = 15.7 years, 59.9% female, 56.4% Hispanic/Latino) replicated Study 1 in older adolescents using automated scoring of FASC flexibility. Flexibility of social cognition varied based on ambiguity and language use; automaticity of social cognition varied by ambiguity. The role of EF was less conclusive; shifting and inhibition predicted some flexibility and automaticity measures. Ambiguity, language, and EF, particularly shifting, influence aToM into adolescence.     

How Heritable are Parental Sensitivity and Limit-Setting? A Longitudinal Child-Based Twin Study on Observed Parenting

We examined the relative contribution of genetic, shared environmental and non-shared environmental factors to the covariance between parental sensitivity and limit-setting observed twice in a longitudinal study using a child-based twin design. Parental sensitivity and parental limit-setting were observed in 236 parents with each of their same-sex toddler twin children (M_age = 3.8 years; 58% monozygotic). Bivariate behavioral genetic models indicated substantial effects of similar shared environmental factors on parental sensitivity and limit-setting and on the overlap within sensitivity and limit-setting across 1 year. Moderate child-driven genetic effects were found for parental limit-setting in year 1 and across 1 year. Genetic child factors contributing to explaining the variance in limit-setting over time were the same, whereas shared environmental factors showed some overlap.     

Optimizing quality of dental carving by preclinical dental students through anatomy theory reinforcement

Knowledge of dental anatomy is of great importance in the practice of dentistry, especially in oral rehabilitation, because without this knowledge, professional practice is not possible. Dental carving plays a major role in training dental students as it develops their manual dexterity. This randomized controlled trial aimed to evaluate the influence of didactic‐theoretical reinforcement on the theoretical and practical knowledge of dental anatomy of preclinical students by examining the quality of the anatomical restorations performed by these students before and after a didactic‐theoretical reinforcement. For the evaluation of theoretical knowledge, a questionnaire with closed questions about dental anatomy was used. To evaluate the effect of didactic reinforcement on dental carvings, two groups of 15 preclinical students were assessed. Experimental group (G1) received a three‐hour theoretical tutoring on dental anatomy, while the control group (G2) did not. The dental carving scores obtained by the two different groups were compared using Student's t‐test. Cohen's d was used to estimate the effect sizes between groups. The frequency of correct answers given for each theoretical knowledge question was compared in each group using Fisher's exact test. T‐test was also used to compare the means of the two groups' final scores of theoretical evaluations. To compare these final scores obtained in both carving and theoretical tests, a principal component analysis was performed with different items assessed in each test to obtain factor loading scores and a final weighted score, where factor loadings were considered for each item. Weighted scores were compared using t‐test. Also, scores obtained during the head and neck course were assessed and compared using t‐test. Spearman's correlation test was used to assess the correlation between scores obtained prior to the anatomy course and scores obtained in the dental carving exercise. The theoretical evaluation revealed no significant difference between the grades (mean ± SD) of G1 (85.1 ± 6.6%) and G2 (86.2 ± 9.1%) with the grades of a baseline test that was previously obtained when students submitted to the study (P = 0.725). Regarding the tooth carving assessment, the dental carving quality by students of G1 has significantly improved, except for tooth #23 (P = 0.096). Theoretical reinforcement of dental anatomy seems to improve the students' carving performance but does not enhance their knowledge about dental anatomy. Anat Sci Educ 11: 377–384. © 2017 American Association of Anatomists.     

A quantitative and qualitative inquiry into the attitudes toward science of nonscience college students

The attitudes toward science of nonscience college students were investigated using quantitative and qualitative forms of inquiry. Quantitative methods were used to determine (a) how attitudes toward science of nonscience college students compare with attitudes of science majors, and (b) whether attitudes toward science change with instruction. Qualitative assessment was used to investigate attitude development as it relates to science. The subjects were 102 nonscience students and 81 science students. Six attitudinal variables were investigated using the Attitudes Toward Science Inventory (ATSI) as the quantitative instrument. Hotelling's T² showed a significant difference (p = 0.0001) in attitudes between the two groups. T tests revealed significant differences between the two groups for all six variables. A significant difference (p = 0.0001) was found between pretest and posttest results for the nonscience students. T tests showed significant differences between the two sets of scores for all six variables, indicating a favorable change in attitudes. An interview questionnaire was used to investigate factors contributing to attitude development. The interview results suggested that attitudes toward science are formed by interactions of both school and nonschool variables.     

Young Children's Theory of Mind Predicts More Sharing With Friends Over Time

Children (predominantly white and middle class) between 3 and 6 years (M = 55.12 months, N = 145 at Time 1, N = 102 at Time 2) participated in the prosocial choice test at two time points approximately 10 months apart. Children could share with strangers, close friends, nonfriends, and in a control, no recipient condition. Children shared more rewards with friends over time. Age interacted with recipient type such that older children had a higher probability of prosocial allocations toward friends and strangers compared to younger children. Theory of mind (ToM) predicted more prosocial allocations to friends over time, and the youngest children with higher ToM scores showed the largest increase in sharing with friends over time.     

Children and Adults Selectively Attribute Shared Cultural Knowledge to Speakers of the Same Language

Children's and adults’ attributions of shared knowledge of and shared preference for songs were investigated across two prominent social categories: language and gender. Both attributions indicate similarity among individuals but shared cultural knowledge can be more informative about common social history than shared preference, as it is mainly transferred through social interactions within cultures, while preferences can have various sources. Both 5- to 6-year-old children (N = 60) and adults (N = 160) generalized knowledge of songs across individuals who speak the same-language rather than same-gender individuals. In contrast, preference for songs was not systematically generalized across either category. Thus, individuals selectively infer shared cultural knowledge among same-language speakers, suggesting an early emerging link between shared knowledge and cultural boundaries.     

A differential item functioning study of counseling competencies scale-revised scores

Our study examines the differential item functioning of the Counselor Competencies Scale—Revised (CCS-R) scores due to respondents’ gender, the type of evaluation, and a combination of these two variables using a large sample (N = 1614). Implications of the findings are offered to inform counselor educators and supervisors using the CCS-R and similar supervisee assessment measures.     

An Exploratory Study of the Effects of Time Compressed Animated Delivery Multimedia Technology on Student Learning in Reproductive Physiology

Two experiments examined the effects of a multimedia technology referred to as “Time Compressed Animated Delivery” (TCAD), on student learning in a junior-level reproductive physiology course. In experiment 1, participating students received one of two presentations of the same instructional material: TCAD and a lecture captured on video. At the completion of each presentation a test was administered and group mean test scores computed and compared. The results were statistically significant (df = 362, t = 10.623, p < 0.05), favoring the TCAD treatment group. The effect size estimate was 1.14. In experiment 2, student learning from three groups were compared on the same reproductive physiology unit used in experiment 1: (1) TCAD, (2) TCAD without the 3-D component, and (3) video-lecture. After removal of three poor functioning items, a one-way analysis of variance was computed. The results were statistically significant (df = 2, F = 2.351, p < 0.10). Results of post hoc comparisons showed that differences in mean test scores between the TCAD and control groups were statistically significant (p < 0.05). The effect size estimate was 0.25. These findings provide preliminary evidence for use of TCAD.     

Teaching neuroanatomy using computer‐aided learning: What makes for successful outcomes?

Computer‐aided learning (CAL) is an integral part of many medical courses. The neuroscience course at Oxford University for medical students includes CAL course of neuroanatomy. CAL is particularly suited to this since neuroanatomy requires much detailed three‐dimensional visualization, which can be presented on screen. The CAL course was evaluated using the concept of approach to learning. The aims of university teaching are congruent with the deep approach—seeking meaning and relating new information to previous knowledge—rather than to the surface approach of concentrating on rote learning of detail. Seven cohorts of medical students (N = 869) filled in approach to learning scale and a questionnaire investigating their engagement with the CAL course. The students' scores on CAL‐course‐based neuroanatomy assessment and later university examinations were obtained. Although the students reported less use of the deep approach for the neuroanatomy CAL course than for the rest of their neuroanatomy course (mean = 24.99 vs. 31.49, P < 0.001), deep approach for CAL was positively correlated with neuroanatomy assessment performance (r = 0.12, P < 0.001). Time spent on the CAL course, enjoyment of it, the amount of CAL videos watched and quizzes completed were each significantly positively related to deep approach. The relationship between deep approach and enjoyment was particularly notable (25.5% shared variance). Reported relationships between deep approach and academic performance support the desirability of deep approach in university students. It is proposed that enjoyment of the course and the deep approach could be increased by incorporation of more clinical material which is what the students liked most. Anat Sci Educ 10: 560–569. © 2017 American Association of Anatomists.     

Does spatial awareness training affect anatomy learning in medical students?

Spatial ability (SA) is the cognitive capacity to understand and mentally manipulate concepts of objects, remembering relationships among their parts and those of their surroundings. Spatial ability provides a learning advantage in science and may be useful in anatomy and technical skills in health care. This study aimed to assess the relationship between SA and anatomy scores in first- and second-year medical students. The training sessions focused on the analysis of the spatial component of objects' structure and their interaction as applied to medicine; SA was tested using the Visualization of Rotation (ROT) test. The intervention group (n = 29) received training and their pre- and post-training scores for the SA tests were compared to a control group (n = 75). Both groups improved their mean scores in the follow-up SA test (P < 0.010). There was no significant difference in SA scores between the groups for either SA test (P = 0.31, P = 0.90). The SA scores for female students were significantly lower than for male students, both at baseline and follow-up (P < 0.010). Anatomy training and assessment were administered by the anatomy department of the medical school, and examination scores were not significantly different between the two groups post-intervention (P = 0.33). However, participants with scores in the bottom quartile for SA performed worse in the anatomy questions (P < 0.001). Spatial awareness training did not improve SA or anatomy scores; however, SA may identify students who may benefit from additional academic support.     

Effects of Home Literacy,Parents' Beliefs,and Children's Task-Focused Behavior on Emergent Literacy and Word Reading Skills

We examined the effects of home literacy (shared book reading, teaching activities, and number of books), children's task-focused behavior, and parents' beliefs and expectations about their child's reading and academic ability on kindergarten children's (N = 61) phonological sensitivity and letter knowledge and on Grade 1 word reading. The results showed that, after controlling for nonverbal IQ and vocabulary, home literacy instruction prior to kindergarten, parents' beliefs about their children's reading ability, and children's task-focused behavior were significant predictors of two or more of the dependent variables. Storybook reading did not account for unique variance in any of the dependent variables.     

Teacher education and the choice to enter the teaching profession: A prospective study

This two-wave survey study aimed at testing a hypothetical model of teacher education graduates’ decisions about whether or not to take a teaching position upon graduation. The model focuses on the relationship between teacher education and graduates’ choice on job entrance. Using path analysis and logistic regression, this model was tested in a sample of student teachers (n = 436), subsequently graduates (n = 251) of teacher training for secondary education. The results validate the relationship between teacher education variables and nearly graduates’ intention to enter the teaching profession. Furthermore, this intention proves an imperative predictor of graduates’ actual entrance.     

The impact of bilingual dictionaries on lexical sophistication and lexical accuracy in tests of L2 writing proficiency: A quantitative analysis

Writing assessment essentially juxtaposes two elements: how ‘good writing’ is to be defined, and how ‘good measurement’ of that writing is to be carried out. The timed test is often used in large-scale L2 writing assessments because it is considered to provide reliable measurement. It is, however, highly inauthentic. One way of enhancing authenticity in a way that might also promote good writing is to include access to support resources in the test. This article reports a study into whether allowing a bilingual dictionary enhances good writing in German writing tests. Participants (n = 47) took two tests, one with and one without a bilingual dictionary. The essays of sub-sets of the participants were compared in terms of lexical sophistication, lexical accuracy and test score. Lexical sophistication was measured by means of the Lexical Frequency Profile (Laufer & Nation, 1995). Lexical accuracy was determined through a quantitative analysis of words looked up in the dictionary. The results revealed an increase in lexical sophistication in ‘with dictionary’ tests but frequent misuse of look-ups. There was also no improvement in test scores. Subsequent issues for the use of dictionaries in writing tests are raised.