Focus: Design and Evaluation of a Software Tool for Collecting Reader Feedback

Reader feedback is generally considered to be valuable input for writers who want to optimize their documents, but a reader-focused evaluation is often time-consuming. For this reason, we have developed Focus, a software tool for collecting reader comments more efficiently. The design and rationale of the software are described in this article. In a small-scale evaluation study, the results we obtained using Focus were compared to the reader feedback collected under the plus-minus method. It appeared that the number of problems detected per participant did not differ, but there were differences in the types of problems found. Focus participants appeared to comment more from a reviewer's and less from a user's perspective. Although the two methods are not interchangeable, Focus can be said to be a promising evaluation tool, deserving further research.     

Leveraging new media skills in a peer feedback tool

Today's youth thrive in informal participatory communities where they not only consume but also act as contributors or producers. Through active participation they develop new media skills encompassing cultural competencies and social skills. In a participatory culture of learning, students' active contributions to their learning are stressed and peer feedback is considered an important component. In this paper we consider how to leverage aspects of these skills and the environments in which they are developed in the design of a peer feedback tool for students engaged in collaborative inquiry learning. The tool needs to be lightweight, and “playful” and draw on these participatory skills in order to entice and motivate students to give and receive feedback.     

The “least intrusive intervention” strategy for behavior change procedures: The use of public and private feedback in school classrooms

Many classroom behavior modification procedures have failed to be adopted by practicing classroom teachers because the procedures are overly intrusive into the regular classroom routine. A strategy for teachers and consultants which moves from less to more intrusive interventions is described and demonstrated with a case example. Private feedback was ineffective in reducing a child's inappropriate behavior, but the introduction of public feedback resulted in a decrease in inappropriate behavior to below the class average. Attitudinal measures of the teacher's evaluation of the child and his selfevaluation were related to the child's behavior during the public feedback procedure.     

Moral Education or Indoctrination in South Africa? A Brief Response to Potgieter

Abstract

This paper is a brief and informal response to Professor P. C. Potgieter's paper Moral Education in South Africa which appeared in the January 1980 edition of this Journal (Vol. 9 No. 2, pp. 130‐3). In response to Potgieter the author attempts to present some of the more obvious philosophical and sociological inconsistencies and problems appearing in Potgieter's paper. He concludes, basically, that Potgieter has assumed a marked consensual model of South African society and, therefore, his analysis serves only to misinform the reader as to the complexities of moral education in South Africa.     

Evaluating students' science notebooks as an assessment tool

The idea of using science notebooks as a classroom assessment tool is not new. There is general agreement that science notebooks allow teachers to assess students' conceptual and procedural understanding and to provide the feedback students need for improving their performance. In this study we examined the use of science notebooks as an unobtrusive assessment tool that can also be used by individuals outside the classroom (for example, school district personnel), and as a means for obtaining information about students' learning and their opportunities to learn. More specifically, in this study students' science notebooks were used as a source of data about the (a) implementation of a curriculum's intended activities, (b) students' performance, and (c) quality of teachers' feedback. Our results indicated that: (1) Students' science notebooks can be reliably scored. Unit implementation, student performance, and teacher feedback scores were highly consistent across raters and units. (2) High and positive correlations with other performance assessment scores indicated that the student performance score can be considered as an achievement indicator. And (3) low performance scores across the two units revealed that students' communication skills and understanding were far away from the maximum score and did not improve over the course of instruction during the school year. This result may be due, in part, to the fact that no teacher feedback was found in any of the students' notebooks across the six classrooms studied. This may reflect some characteristics of the teachers' assessment practices that may require further professional development.     

An innovative concept map approach for improving students' learning performance with an instant feedback mechanism

Concept maps have been widely employed for helping students organise their knowledge as well as evaluating their knowledge structures in a wide range of subject matters. Although researchers have recognised concept maps as being an important educational tool, past experiences have also revealed the difficulty of evaluating the correctness of a concept map. It usually takes days or weeks for teachers to manually evaluate the concept maps developed by students; consequently, the students cannot receive timely feedback from the teachers, which not only affects their learning schedules, but also significantly influences the students' learning achievements. In this paper, a computer‐based concept map‐oriented learning strategy with real‐time assessment and feedback is proposed in order to cope with the problems mentioned above. Our approach provides immediate evaluation of concept maps and gives also real‐time feedback to the students. An experiment has been conducted to evaluate the effectiveness of this new strategy in comparison with the conventional computer‐based concept map approach. It is found that our innovative approach can be significantly beneficial to promote learning achievements as well as the learning attitudes of students.     

Pupils’ Information Processing and Its Implications for Learning and Assessment: A Think-Aloud Study

A think-aloud study investigated 7th graders’ (N?=?30) information processing and seeking of feedback while reading four multiple partly contradictory texts. An extreme-case analysis shows that a highly-skilled reader uses deeper strategies, such as elaboration, evaluation, and paraphrasing of less contradictory information, than a less-skilled reader. Elaboration strategies are frequently used by the less-skilled reader, but often consist of irrelevant elaboration strategies, which seem to preclude comprehension and deeper learning. The highly-skilled reader seeks internal feedback on task-, process-, and self levels, while the less-skilled reader seeks feedback related to task- and self levels. The results provide useful knowledge regarding pupils’ trajectories into deeper learning and self-regulation processes in information-processing tasks.     

A Study of the Influence of Some Factors in Style of Composition on the Interest,Comprehension, and Rate of Reading of Fourth-Grade Pupils

The research hypothesis was that memory strategy deficits can occur because of students' failure to understand the task and to evaluate their own performance. The effect of performance feedback on memory strategy use, performance evaluation, and recall was assessed with students with and without mild mental retardation with a mental age of approximately 8.5 years. For students with retardation, feedback resulted in more accurate performance evaluation and recall, but memory strategy use did not increase. By contrast, for students without retardation, feedback resulted in more accurate performance evaluation and greater recall accuracy and memory strategy use. We concluded that students with retardation may not have the appropriate memory strategies in their repertoire and, therefore, feedback does not result in strategy use. It seems likely that memory strategy use in this population would be increased by a training package that includes strategy instruction as well as feedback.     

Small Talk: Identifying communication problems in maltreated children

Development of speech and language is rapid in early years, yet if developmental problems in speech and language are not addressed they are likely to continue and impact negatively on a child's overall development and their life trajectory. Children who have experienced abuse and or neglect are particularly vulnerable. The aim of this study was to develop a tool to assist in identifying a child's need for assessment by a speech pathologist so that there could be early identification of problems. A culturally sensitive tool was developed to be completed by the child's carer included questions on language, speech and hearing, voice, fluency, understanding sentences, vocabulary and expression. Sixty-five children aged between 4 and 8 years, who had experienced abuse and/or neglect participated in the study. Fourteen percent were Aboriginal. A speech pathologist undertook an assessment for each child and the results were compared with the information on the Small Talk tool. The Tool was found to be high in sensitivity but low in specificity, requiring further refinement. However, it has the potential to assist non speech pathologists to identify a child's need for speech and language assessment with the findings identifying the Tool as promising practice.     

Learning by Seeing by Doing: Arithmetic Word Problems

"Learning by doing" in pursuit of real-world goals has received much attention from education researchers but has been unevenly supported by mathematics education software at the elementary level, particularly as it involves arithmetic word problems. In this article, we give examples of doing-oriented tools that might promote children's ability to "see" significant abstract structures in mathematical situations. The reflection necessary for such seeing is motivated by activities and contexts that emphasize affective and social aspects. Natural language, as a representation already familiar to children, is key in these activities, both as a means of mathematical expression and as a link between situations and various abstract representations. These tools support children's ownership of a mathematical problem and its expression; remote sharing of problems and data; software interpretation of children's own word problems; play with dynamically linked representations with attention to children's prior connections; and systematic problem variation based on empirically determined level of difficulty.     

STUDENT EVALUATION OF TEACHING IN AN AUSTRALIAN UNIVERSITY ‐‐ STAFF PERCEPTIONS AND REACTIONS

At the University of Queensland staff have had the opportunity since 1982 to make use of a standardised questionnaire to students to get feedback on their teaching. The questionnaire was developed and is organised in such a way that it may be used in promotion applications, as evidence of teaching ability in the annual appraisal of probationary staff by the head of department, and for personal feedback. After one year of operation interviews were held with all those who had used the questionnaire. Their attitude to evaluation of teaching was overwhelmingly positive and differed sharply from the way they perceived their department's and the University's attitude to be. Staff, on the whole, acted on the feedback received from their students by changing the course organisation and content, aspects of classroom teaching or their attitude. Many felt more confident about their teaching as a result of the evaluation, and for many the questionnaires were the only source of feedback and encouragement.     

Web site accessibility: what logo will we use today?

The Special Educational Needs and Disability Act has amended the Disability Discrimination Act and requires that students with a disability not be discriminated against or substantially disadvantaged by establishments of further and higher education. Academic web site developers must take steps to ensure that their sites fulfil this requirement, and guidance on compliance is provided by the World Wide Web Consortium (W3C) Web Accessibility Initiative's (WAI) Web Content Accessibility Guidelines, which offer three levels for assessing accessibility. Tools which give developers feedback on their site are available, but following their recommendations alone will not guarantee SENDA compliance. An audit of UK academic web sites has shown discrepancies between the level of compliance that sites are claiming and that which has been achieved. This would seem to demonstrate that there is a misunderstanding of the requirements necessary to create accessible web sites. This problem is exacerbated by the academic sector's widespread reliance on software accessibility tools, the use of which tends to lead developers to rely on the tools’ quantitative approach when what is required is the inclusion and development of a more qualitative view which prioritises inclusive design from the outset.     

Read-Think-Aloud Protocols: A New Data-Source for Formative Evaluation

Traditional approaches to formative evaluation include one-to-one trials which provide some information about revisions that are needed in instructional materials, but typically fail to provide sufficient information about the learner's cognitive processing problems while studying the materials. Read-think-aloud methods are a viable alternative to traditional one-to-ones and provide valuable information about the learner's cognitive processing while reading instructional text. In this paper we describe the read-think-aloud method of formative evaluation and offer suggestions for its use.     

“It's all coming together”: an encounter between implied reader and actual reader in the Australian rainforest

In this paper I discuss how taking a particular literary theory— the implied reader—serves to offer a focus for the teacher's initial reading of a text and provides a formative assessment tool. Iser's Implied Reader theory is discussed, after which a picture book, Where the Forest Meets the Sea by Jeannie Baker, is analysed from this perspective. An account of a series of reading events with Year 4 children then follows. An analysis of these readings highlights where the pupils were able to take up the invitation offered by the text and where there were difficulties. Finally, implications for teachers' subject knowledge are discussed.     

Affordances and constraints of computers in science education

Throughout the 1970s and 1980s, computers in science teaching were seen as a panacea for many problems plaguing the domain. While considerable research has been done to determine cognitive achievements of students who interact with computers during their science learning, more basic questions have not yet been addressed. This study was designed to investigate how computers and a modeling software contributed to students' interactions and learning in a physics course. The interpretations focused on the microworld as a tool that supported but also limited students' sense-making activities. First, the computer microworld contributed in significant ways to the maintenance and coordination of students' physics conversations. Second, the computer environment (a) was sometimes “unready to hand” so that students spent more time learning the software rather than physics, and (b) limited the interactions within groups. It was concluded that while computer environments have some potential as learning tools, they also limit interactions in significant ways, rendering them less than ideal for everyday classroom use. With the use of software … students can be provided with the necessary tools and experiences to practice the investigative skills used by scientists and mathematicians… [Students] can pursue specific topics of their own interest and deal with this information in sufficient depth to construe personal meaning to various concepts. (Barman, 1993, p. viii) In educational applications, user interface design has received little attention, despite the fact that the interface is particularly important for educational software… This concern goes much deeper than the nebulous concept most often represented by the buzz phrase, ‘user friendliness.’ (Jackson, Edwards, & Berger, 1993b, p. 414) © 1996 John Wiley & Sons, Inc.     

Selecting educational apps for preschool children: How useful are website app rating systems?

In the present paper, we assess whether website rating systems are useful for selecting educational apps for preschool age children. We selected the 10 highest scoring and 10 lowest scoring apps for 2–4-year-olds from two widely used websites (Good App Guide; Common Sense Media). Apps rated highly by the two websites had a higher educational potential as assessed by a validated questionnaire for evaluating the educational potential of apps and were more likely to include a learning goal and feedback compared to low scoring apps. However, high scoring apps scored on average just 9/20 for indicators of educational potential, and both high and low scoring apps had poor language quality as determined by psycholinguistic and construction type analyses. We argue that website rating systems should also include quality of feedback, adjustable content, social interactions, storyline and a more fine-grained analysis of language in their assessments.

Practitioner notes

What is already known about this topic

• Appropriately designed apps for preschool age children have the potential to teach early school readiness skills.
• Selecting high quality educational apps for preschool age children is challenging.
• The children's app marketplace is currently unregulated.

What this paper adds

• We assess whether two leading app rating websites are useful for selecting educational apps for preschool age children.
• Children's apps rated highly by two app website rating systems had a higher educational potential than low rated apps as measured by a research informed app evaluation tool.
• In-depth analysis of the language in apps shows that highly rated children's apps on app rating websites may not enrich a child's early language environment.

Implications for practice and/or policy

• Children's app rating website assessments should include potential for feedback, language, adjustable content, storyline and social interactions.
• Policy should be implemented for app ratings in the app stores or on website app rating systems.