A comparison of the approaches of generalizability theory and item response theory in estimating the reliability of test scores for testlet-composed tests

Previous assessments of the reliability of test scores for testlet-composed tests have indicated that item-based estimation methods overestimate reliability. This study was designed to address issues related to the extent to which item-based estimation methods overestimate the reliability of test scores composed of testlets and to compare several estimation methods for different measurement models using simulation techniques. Three types of estimation approach were conceptualized for generalizability theory (GT) and item response theory (IRT): item score approach (ISA), testlet score approach (TSA), and item-nested-testlet approach (INTA). The magnitudes of overestimation when applying item-based methods ranged from 0.02 to 0.06 and were related to the degrees of dependence among within-testlet items. Reliability estimates from TSA were lower than those from INTA due to the loss of information with IRT approaches. However, this could not be applied in GT. Specified methods in IRT produced higher reliability estimates than those in GT using the same approach. Relatively smaller magnitudes of error in reliability estimates were observed for ISA and for methods in IRT. Thus, it seems reasonable to use TSA as well as INTA for both GT and IRT. However, if there is a relatively large dependence among within-testlet items, INTA should be considered for IRT due to nonnegligible loss of information.     

Mapping online social networks of Korean politicians

Korean politicians, particularly National Assembly members, have embraced the internet as a major means of communication for managing election campaigns and maintaining relationships with (potential) supporters during non-election periods. An increasing number of National Assembly members have become aware of the considerable influence that the internet has on their constituents, and thus, this study determines whether there are significant differences in online networks of National Assembly members and examines the relationship among National Assembly members and that between those members and their constituents to determine whether Twitter is used as a tool for communication as well as deliberation. The data were drawn from homepages, blogs, and Twitter pages of the members of the 18th National Assembly in April 2010. The results indicate that National Assembly members tended to have a more equal relationship in the Twitter network than in the homepage or blog network. More importantly, the members were linked mainly to fellow politicians in the same party, and thus, they used Twitter to communicate more with fellow politicians than with their constituents.     

Multimodal Genre of Science Classroom Discourse: Mutual Contextualization Between Genre and Representation Construction

Research in Science Education - This paper argues that meaning-making with multimodal representations in science learning is always contextualized within a genre and, conversely, what constitutes...     

Course evaluation: reconfigurations for learning with learning management systems

The introduction of online delivery platforms such as learning management systems (LMS) in tertiary education has changed the methods and modes of curriculum delivery and communication. While course evaluation methods have also changed from paper-based in-class-administered methods to largely online-administered methods, the data collection instruments have remained unchanged. This paper reports on a small exploratory study of two tertiary-level courses. The study investigated why design of the instruments and methods to administer surveys in the courses are ineffective measures against the intrinsic characteristics of online learning. It reviewed the students' response rates of the conventional evaluations for the courses over an eight-year period. It then compared a newly developed online evaluation and the conventional methods over a two-year period. The results showed the response rates with the new evaluation method increased by more than 80% from the average of the conventional evaluations (below 30%), and the students' written feedback was more detailed and comprehensive than in the conventional evaluations. The study demonstrated the possibility that the LMS-based learning evaluation can be effective and efficient in terms of the quality of students' participation and engagement in their learning, and for an integrated pedagogical approach in an online learning environment.     

Early childhood practicum students' professional growth in the USA: areas of confidence and concern

This research examines specific areas of confidence and concern as expressed by 40 American undergraduate early childhood students on a practicum (supervised field-based internships); if their beliefs changed over the course of their practicum, and if prior teaching experience had an impact on their confidence levels. Areas of confidence and concern were determined through questionnaires, reflections and interviews at three points in time during the semester. Findings indicate that students were more confident when arranging the physical environment and least confident in managing children's behaviour. Significant changes in understanding and interaction with children, planning and implementing activities and overall confidence were observed over time periods. Experienced students initially showed and maintained higher confidence levels than inexperienced students who were much less confident at the start of the practicum, but improved significantly. Interestingly, by the end of the semester, this latter group was as confident as more experienced students. Here the factors that affected confidence, the factors that boosted student confidence at different time periods and differences in confidence among students based on prior experience are discussed.     

Transformation of classroom spaces: traditional versus active learning classroom in colleges

Educational environment influences students’ learning attitudes, and the classroom conveys the educational philosophy. The traditional college classroom design is based on the educational space that first appeared in medieval universities. Since then classrooms have not changed except in their size. In an attempt to develop a different perspective of educational environment, a new design of classroom, the active learning classroom (ALC), was established at SoongSil University in Korea. Two questionnaire surveys were conducted for diagnosing the educational effects of students’ learning in the ALC and comparing the results with those obtained regarding the traditional classroom. The result proved the existence of a ‘golden zone’ and a ‘shadow zone’ in the traditional classroom, which discriminate students’ learning experiences depending on seating positions. On the contrary, the ALC did not produce such positional discrimination. Students perceived the ALC environment as more inspirational, especially in regards to active class participation. Students with more emphasis on academic achievement showed greater tendency to share information and to create new ideas in the ALC. However, in the traditional classroom setting, only students with high GPAs were more motivated to learn while the gap in learning attitudes was offset in the ALC setting. In-depth discussions about research findings were undertaken and four suggestions were provided in support of school administrators and relevant institutional personnel, faculty members, and researchers for future utilization of the ALC.     

A developmental study on a SPAT design model for mobile learning

Educational technology research and development - This paper explores another format for mobile learning (mLearning) content consisting of Still Pictures, Audio, and Text (SPAT; Rha, Instructional...     

Student–Faculty Interaction and Discrimination from Faculty in STEM: The Link with Retention

Research in Higher Education - Previous studies have documented student–faculty interaction in STEM, but fewer studies have specifically studied negative forms of interaction such as...     

What Happens When a Teacher’s Science Belief Structure Is in Disequilibrium? Entangled Nature of Beliefs and Practice

Research in Science Education - This qualitative case study examined the process of change in an experienced elementary teacher’s belief structure during implementation of an inquiry-based...     

Seeing the Forest through the Trees Using Network Analysis: Exploring Student Responses to Conceptual Physics Questions

Journal of Science Education and Technology - This study explored how scientific ideas interacted in college students’ scientific explanations using text analysis techniques and network...