Learning approaches: examination type,discipline of study,and gender

A sample of 248 Australian university students participated in a study that investigated the influence of assessment type and discipline of study on students' learning approaches. Participants (93 psychology and 155 business students) responded to the Study Process Questionnaire in a context of preparation for a hypothetical examination. The hypothetical examination is in either a multiple‐choice or an essay format. Results indicated that assessment type had no significant influence on how students approach their learning, while discipline did have significant influence on student learning.     

Faculty and Students Conceptions of Assessment in Higher Education

Assessment in higher education serves multiple purposes such as providing information about student learning, student progress, teaching quality, and program and institutional accountability. Yet, little is known about faculty and students’ attitudes regarding different aspects of assessment that have wide-ranging implications for policy and practice in tertiary institutions. To investigate these views, parallel surveys of conceptions of assessment were administered to faculty and undergraduate students across four tertiary institutions including universities, an indigenous tertiary institution, and an institute of technology. A mean and covariance structures approach was used to test for measurement invariance and latent means differences between faculty and students regarding their conceptions of assessment. Results revealed differences in the latent means across the two groups. Faculty were likely to view assessment as a trustworthy process aiding teaching and learning, whereas students viewed assessment as focussed primarily on accountability and perceived assessment as irrelevant or even ignored in the teaching and learning process. These findings highlight the importance of ensuring that assessment policy and practices are fit for purposes, and are being carried out with integrity in ways that are transparent to and understood by both staff and students. While these results show how staff and students view assessment practices, one should keep in mind that while the sample was large and did incorporate different types of tertiary institutions, the inclusion of a broader range of disciplines would make the conclusions more generalizable.     

The overall effects of end-of-course assessment on student performance: A comparison between multiple choice testing, peer assessment, case-based assessment and portfolio assessment

This study investigates the effect of method of assessment on student performance. Five research conditions go together with one of four assessment modes, namely: portfolio, case-based, peer assessment, and multiple choice evaluation. Data collection is done by means of a pre-test/ post-test-design with the help of two standardised tests (N=816). Results show that assessment method does make a difference: assessments do not produce overall effects on student performance. Moreover, student-activating instruction efforts do not automatically result in more extensive learning gains. Finally, test results show, when compared to other assessments, a statistically significant positive effect of the multiple choice test on students' test scores. However, students' preparation level and the closed book format of the tests might serve explanatory purposes.     

Psychometric models of student conceptions in science: Reconciling qualitative studies and distractor-driven assessment instruments

We stand poised to marry the fruits of qualitative research on children's conceptions with the machinery of psychometrics. This merger allows us to build upon studies of limited groups of subjects to generalize to the larger population of learners. This is accomplished by reformulating multiple choice tests to reflect gains in understanding cognitive development. This study uses psychometric modeling to rank the appeal of a variety of children's astronomical ideas on a single uniform scale. Alternative conceptions are captured in test items with highly attractive multiple choice distractors administered twice to 1250 8th through 12th-grade students at the start and end of their introductory astronomy courses. For such items, an unusual psychometric profile is observed—instruction appears to strengthen support for alternative conceptions before this preference eventually declines. This lends support to the view that such ideas may actually be markers of progress toward scientific understanding and are not impediments to learning. This method of analysis reveals the ages at which certain conceptions are more prevalent than others, aiding developers and practitioners in matching curriculum to student grade levels. This kind of instrument, in which distractors match common student ideas, has a profoundly different psychometric profile from conventional tests and exposes the weakness evident in conventional standardized tests. Distractor-driven multiple choice tests combine the richness of qualitative research with the power of quantitative assessment, measuring conceptual change along a single uniform dimension. © 1998 John Wiley & Sons, Inc. J Res Sci Teach 35: 265–296, 1998.     

A Multiple Choice Test that Rewards Partial Knowledge

A new multiple choice test format is presented that allows examinees to select more than one answer to a question if they are uncertain of the correct one. Negative marking is used to penalise incorrect selections. The aim is to explicitly reward examinees who possess partial knowledge as compared with those who are simply guessing. The result is a test method that forces examinees to think more carefully about their answers, and that yields results of a higher resolution than standard multiple choice tests. After describing the new format, the paper presents and critiques several existing methods which have the same or similar aims. The paper ends with a discussion of the feedback and experience gained to date in using the new format.     

A stratified study of students’ understanding of basic optics concepts in different contexts using two‐tier multiple‐choice items

A large scale study involving 1786 year 7–10 Korean students from three school districts in Seoul was undertaken to evaluate their understanding of basic optics concepts using a two‐tier multiple‐choice diagnostic instrument consisting of four pairs of items, each of which evaluated the same concept in two different contexts. The instrument, which proved to be reliable, helped identify several context‐dependent alternative conceptions that were held by about 25% of students. At the same time, students’ performance on the diagnostic test correlated with the location of the schools, students’ achievement in school science and their attitudes to science learning. However, students’ grade levels had limited influence on their understanding of basic concepts in optics as measured by the instrument.     

Relationships between learning patterns and attitudes towards two assessment formats

The study examined the relationships between learning patterns and attitudes towards two assessment formats: open‐ended (OE) and multiple‐choice (MC), among students in higher education. Sixteen Semantic Differential scales measuring emotional reactions, intellectual reactions and appraisal of each assessment format, along with measures of learning processes, academic self‐concept and test anxiety, were administered to 58 students. Results indicated two patterns of relationships between the learning‐related variables and the assessment attitudes: high scores on the self‐concept measure and on the three measures of learning processes were related to positive attitudes towards the OE format but negative ones towards the MC format; low scores on the test anxiety measures were related to positive attitudes towards the OE format. In addition, significant gender differences emerged with respect to the MC format, with males having more favourable attitudes than females. Results were discussed in light of an adaptive assessment approach.     

Developing,Evaluating and Demonstrating the Use of a Conceptual Survey in Mechanical Waves

An understanding of mechanical waves is a pre‐requisite for the study of many topics in advanced physics, and indeed in many other disciplines. There have been many research studies in mechanical waves, all of which have revealed that students have trouble with the basic concepts. Therefore, in order for teachers to prepare appropriate instruction for their classes, it is useful to diagnose their students' conceptions—if possible before they enter class. It is for this purpose that many diagnostic instruments have been developed, often in the form of multiple‐choice tests. In this study, we have used the open‐ended Wave Diagnostic Test to develop a multiple‐choice conceptual survey in an evolutionary manner. The two‐year development procedure included open‐ended surveys and interviews involving 299 Thai students and 88 Australian students. The final version, called the Mechanical Waves Conceptual Survey, has been administered to 632 Australian students from high school to second‐year university and 270 Thai high school students. Standard statistical analyses show that the survey is reliable and valid. Further validity checks, including consultation with experts, were also carried out. The survey has four subtopics—propagation, superposition, reflection, and standing waves—and the teachers can choose the subtopics relevant for their students. In this paper we also demonstrate the use of a typical survey question to test students' conceptual understanding and identify common alternative conceptions.     

Justifying the selection of answers in multiple choice items

Although multiple choice tests, as commonly used, can be rightly criticized, their structure, when wisely used, makes them an excellent diagnostic tool for identifying students’ conceptions, including misconceptions. This paper offers a number of useful guidelines regarding the analysis of multiple choice items. A special reference is made to the use of the justifications that students provide for their choices. Examples of the procedures, which may be used in analysing justifications and the kinds of diagnostic information that may be obtained, are provided.     

Formal music education not only enhances musical skills, but also conceptions of teaching and learning: a study with woodwind students

This paper studies conceptions of woodwind students regarding learning and teaching music, within the framework of implicit theories. It analyzes whether there are differences related to participants’ instruction level, and whether it is possible to identify profiles according to their conceptions. Sixty-eight students from three different levels at 14 Spanish conservatories took part voluntarily in this study. A multiple-choice questionnaire was used, which comprised 16 dilemmas on different situations typical of learning and teaching music. Three answers were presented for each dilemma, corresponding to the three implicit theories identified in recent years: direct, interpretative, and constructive. For each dilemma, participants were requested to choose the option they most agreed with and the one they least agreed with. Data were analyzed using chi-square tests of independence, multiple correspondence analysis, and ascending hierarchical classification. From the results obtained we can state that as students’ age and level of instruction increase, so does the level of sophistication of their conceptions. Three conceptual profiles were identified: direct, direct-interpretative, and constructive, which are consistent with prior results on conceptions of teaching and learning in other domains. Implications for education are discussed.     

Analogical reasoning for understanding solution rates: students’ conceptual change and chemical explanations

The study aims to demonstrate evidence of (a) students’ conceptual change on solution rates; (b) students’ sub‐microscopic explanations of dissolution; and (c) retention of the concepts of solution rates. The sample consists of 44 Grade 9 students (18 boys and 26 girls) drawn purposively from two different classes (22 each) in the city of Trabzon, Turkey. The current study incorporates multiple methods of data collection: items from a solution concept test, clinical interviews and examination of students’ self‐assessment tasks. The results reveal that there is a statistically significant difference between scores in pre‐test and post‐test and between pre‐test and delayed test (p < 0.05). This research indicates that the intervention has improved students’ understanding of the conceptual relationship between solution rates and the sub‐microscopic explanation of dissolution. It has also helped in overcoming students’ alternative conceptions to some extent. However, alternative conceptions have not been eliminated completely. In the light of the results, this current study suggests that a simple model such as the four‐step constructivist teaching (4E) is efficient.     

Confidence versus Performance as an Indicator of the Presence of Alternative Conceptions and Inadequate Problem‐Solving Skills in Mechanics

This study investigated the use of performance–confidence relationships to signal the presence of alternative conceptions and inadequate problem‐solving skills in mechanics. A group of 33 students entering physics at a South African university participated in the project. The test instrument consisted of 20 items derived from existing standardised tests from literature, each of which was followed by a self‐reported measure of confidence of students in the correctness of their answers. Data collected for this study included students’ responses to multiple‐choice questions and open‐ended explanations for their chosen answers. Fixed response physics and confidence data were logarithmically transformed according to the Rasch model to linear measures of performance and confidence. The free response explanations were carefully analysed for accuracy of conceptual understanding. Comparison of these results with raw score data and transformed measures of performance and confidence allowed a re‐evaluation of the model developed by Hasan, Bagayoko, and Kelley in 1999 for the detection of alternative conceptions in mechanics. Application of this model to raw score data leads to inaccurate conclusions. However, application of the Hasan hypothesis to transformed measures of performance and confidence resulted in the accurate identification of items plagued by alternative conceptions. This approach also holds promise for the differentiation between over‐confidence due to alternative conceptions or due to inadequate problem‐solving skills. It could become a valuable tool for instructional design in mechanics.     

Students’ perceptions about evaluation and assessment in higher education: a review1

In educational contexts, understanding the student’s learning must take account of the student’s construction of reality. Reality as experienced by the student has an important additional value. This assumption also applies to a student’s perception of evaluation and assessment. Students’ study behaviour is not only determined by the examination or assessment modes that are used. Students’ perceptions about evaluation methods also play a significant role. This review aims to examine evaluation and assessment from the student’s point of view. Research findings reveal that students’ perceptions about assessment significantly influence their approaches to learning and studying. Conversely, students’ approaches to study influence the ways in which they perceive evaluation and assessment. Findings suggest that students hold strong views about different assessment and evaluation formats. In this respect students favour multiple‐choice format exams to essay type questions. However, when compared with more innovative assessment methods, students call the ‘fairness’ of these well‐known evaluation modes into question.     

Anonymity as an instructional scaffold in peer assessment: its effects on peer feedback quality and evolution in students’ perceptions about peer assessment skills

Although previous research has indicated that providing anonymity is an effective way to create a safe peer assessment setting, continuously ensuring anonymity prevents students from experiencing genuine two-way interactive feedback dialogues. The present study investigated how installing a transitional approach from an anonymous to a non-anonymous peer assessment setting can overcome this problem. A total of 46 bachelor’s degree students in Educational Studies participated in multiple peer assessment cycles in which groups of students assessed each other’s work. Both students’ evolution in peer feedback quality as well as their perceptions were measured. The content analysis of the peer feedback messages revealed that the quality of peer feedback increased in the anonymous phase, and that over time, the feedback in the consecutive non-anonymous sessions was of similar quality. The results also indicate that the transitional approach does not hinder the perceived growth in peer feedback skills, nor does it have a negative impact on their general conceptions towards peer assessment. Furthermore, students clearly differentiated between their attributed importance of anonymity and their view on the usefulness of a transitional approach. The findings suggest that anonymity can be a valuable scaffold to ease students’ importance level towards anonymity and their associated need for practice.     

Extended matching sets questions for online numeracy assessments: a case study

Extended matching sets questions (EMSQs) are a form of multiple‐choice question (MCQ) consisting of a stem (the question or scenario) with an extended number of possible answers. Although there is no consensus on their absolute format, for the purpose of this paper a multiple‐choice question with ten or more alternative answers is considered to be an EMSQ. Faced with the limitations imposed by virtual learning environment software, I have conducted a case study into the use of the EMSQ format in online assessment of numerical and statistical ability which shows that properly constructed questions of this type can play a valuable role in assessment of numeracy. The extended format was found to work well on screen and resulted in an increase in both student marks and student satisfaction when compared with other answer input formats. This case study indicates that the EMSQ format has much more widespread applicability for online assessment that its traditional uses.     

Changing approaches to classroom assessment: An empirical study across teacher career stages

Research indicates that there is considerable variability in teachers' approaches to assessment resulting in different learning cultures for students. The primary purpose of the study is to examine the relationship between teachers' approaches to assessment across a set of dimensions (including their conceptions of assessment purposes, processes, fairness, and measurement theory) and career stage. The results of this paper illustrate nuanced impacts of career stage on teachers' approaches to multiple dimensions of assessment and enable the generation of assessment profiles that provide empirical support for differences in teachers’ approaches to assessment both within and between career stages.     

Consistency of Students’ Ideas across Evaporation, Condensation, and Boiling

Existing research on students’ conceptions contain competing philosophical positions concerning the nature of students’ ideas—whether those ideas are coherent, systematic and theory-like, or fragmented and incoherent. Existing research has also focused primarily on studies of individual conceptions rather than investigating multiple, related conceptions. Nevertheless, there is wide agreement among researchers and teachers alike that the ideas students bring to a learning situation are fertile ground for investigation, and that students’ ideas should be taken into consideration when planning science instruction. The purpose of this study was to examine the representational, conceptual framework, and contextual consistency aspects of two students’ ideas across concepts of evaporation, condensation, and boiling. Knowing the consistency students express for each specific concept, and how well they integrate these related concepts, would offer insights that could potentially impact student learning. We present two case studies here that highlight the degree of consistency expressed by two students across different representations for each target concept and in instances where these conceptions are related to one another. Findings from this study highlight the need for attention to the consistency of students’ ideas across multiple, related concepts. Implications from this study support our recommendation for metaconceptual teaching strategies that would help students examine different representations for the same concept and also to examine the consistency of their ideas across multiple conceptions.     

Post-graduate student performance in ‘supervised in-class’ vs. ‘unsupervised online’ multiple choice tests: implications for cheating and test security

This research explores differences in multiple choice test (MCT) scores in a cohort of post-graduate students enrolled in a management and leadership course. A total of 250 students completed the MCT in either a supervised in-class paper and pencil test or an unsupervised online test. The only statistically significant difference between the nine test scores was for one test where the students scored significantly lower in the unsupervised online test. There was no increase in mean test scores over time and the mean test scores for the unsupervised online test were not significantly higher than the mean test scores for the supervised in-class test. The study suggests that unsupervised online MCTs can be a viable tool for assessing knowledge in post-graduate students provided they meet best practice standards for online assessment. Concerns about increased cheating in unsupervised online testing are not supported.     

Engineering Design and Conceptual Change in Science: Addressing thermal energy and heat transfer in eighth grade

The purpose of this research was to investigate the impact of engineering design classroom activities on middle‐school students’ conceptions of heat transfer and thermal energy. One eighth‐grade physical science teacher and the students in three of her classes participated in this mixed‐methods investigation. One class served as the control receiving the teacher’s typical instruction. Students in a second class had the same learning objectives, but were taught science through an engineering design curriculum that included demonstrations targeting specific alternative conceptions about heat transfer and thermal energy. A third class also used the engineering design curriculum, but students experienced typical demonstrations instead of targeted ones. Conceptual understandings of heat transfer and thermal energy and attitudes towards engineering were assessed prior to and after the interventions through interviews, observations, artefact analysis, a multiple choice assessment, and a Likert scale assessment. Results indicated that the engineering design curriculum with targeted demonstrations was significantly more effective in eliciting desired conceptual change than the typical instruction and also significantly more effective than the engineering curriculum without targeted demonstrations. Implications from this study can inform how teachers should be prepared to use engineering design activities in science classrooms for conceptual change.