Assessing personnel practices in higher education: A misleading note of caution

Williams and I provided a model of the ideal or random flow of people from a pre-selection pool of applicants to a post-selection work force. Dometrius and Sigelman claim that our test of the ideal versus actual work force composition lacks statistical power. They also claim that our assumption of random terminations is unreasonable. I show that their two claims are wrong, because they did not use relative measures of female representation in the work force, and because they base their modification of our model on unsupported supposition.     

Influence of Previous Knowledge, Language Skills and Domain-specific Interest on Observation Competency

Many epoch-making biological discoveries (e.g. Darwinian Theory) were based upon observations. Nevertheless, observation is often regarded as ‘just looking’ rather than a basic scientific skill. As observation is one of the main research methods in biological sciences, it must be considered as an independent research method and systematic practice of this method is necessary. Because observation skills form the basis of further scientific methods (e.g. experiments or comparisons) and children from the age of 4 years are able to independently generate questions and hypotheses, it seems possible to foster observation competency at a preschool level. To be able to provide development-adequate individual fostering of this competency, it is first necessary to assess each child’s competency. Therefore, drawing on the recent literature, we developed in this study a competency model that was empirically evaluated within learners (N = 110) from different age groups, from kindergarten to university. In addition, we collected data on language skills, domain-specific interest and previous knowledge to analyse coherence between these skills and observation competency. The study showed as expected that previous knowledge had a high impact on observation competency, whereas the influence of domain-specific interest was nonexistent. Language skills were shown to have a weak influence. By utilising the empirically validated model consisting of three dimensions (‘Describing’, ‘Scientific reasoning’ and ‘Interpreting’) and three skill levels, it was possible to assess each child’s competency level and to develop and evaluate guided play activities to individually foster a child’s observation competency.     

Research and policy on the language of instruction issue in Africa

Since 1995 I have been engaged in consultancies and since 2001 in research dealing with the language of instruction policies in Africa. To what extent are the policies being implemented? What is the role of research and consultancy when it comes to language policy? What happens when researchers or consultants come up with recommendations which do not please donors or the people of power? My paper builds on a consultancy on the language policy of Namibia from 1995, the stock-taking exercise of ADEA/GTZ/UIL of 2006, as well as on research results and experiences gathered within the LOITASA (Language of Instruction in Tanzania and South Africa) project. In the first part of the article I give two examples where the policy recommendations by the consultants/researchers have not been followed up. I give an example of a consultancy that has had some impact on the policy formulation in some African countries. Recommendations from only one of the four studies/consultancies looked at have been thoroughly followed up. These recommendations were in line with what the donors wanted, but did not logically follow from the research done. In the second part of the article I deal with the challenge of making policy research relevant for those most greatly affected by the policy. This means working for a paradigm shift in the research and consultancy on language policies in Africa. It also means looking at the power of misconceptions held by lay people. Towards the end of the article I discuss changing a belief system which has become common knowledge. I show examples of how Tanzanian colleagues in the LOITASA project are trying to do this.     

Investigating the Effect of Different Selected-response Item Formats for Reading Comprehension

While previous research has identified numerous factors that contribute to item difficulty, studies involving large-scale reading tests have provided mixed results. This study examined five selected-response item types used to measure reading comprehension in the Pearson Test of English Academic: a) multiple-choice (choose one answer), b) multiple-choice (choose multiple answers), c) re-order paragraphs, d) reading (fill-in-the-blanks), and e) reading and writing (fill-in-the-blanks). Utilizing a multiple regression approach, the criterion measure consisted of item difficulty scores for 172 items. 18 passage, passage-question, and response-format variables served as predictors. Overall, four significant predictors were identified for the entire group (i.e., sentence length, falsifiable distractors, number of correct options, and abstractness of information requested) and five variables were found to be significant for high-performing readers (including the four listed above and passage coherence); only the number of falsifiable distractors was a significant predictor for low-performing readers. Implications for assessing reading comprehension are discussed.     

Testgütekriterien der „Profilanalyse nach Grießhaber“. Evaluation eines Verfahrens zur Erfassung grammatischer Fähigkeiten von ein- und mehrsprachigen Grundschulkindern

A diagnostic goal of the “Profile analysis according to Grießhaber” is the assessment of the grammatical ability to place verbs in German sentences in the correct syntactical position. To evaluate the test quality for the first time, 403 monolingual and multilingual primary school children were randomly assigned to three different test materials: Test material that specifically stimulated the target competence led to the best test results. This indicates a low objectivity of application. In addition, ceiling effects for primary school children were detected, which means that the profile analysis can only slightly differentiate among the grammatical abilities of the children. Retest reliability and objectivity of analysis are also too small. In summary, the test quality of the assessment tool should be improved with respect to all of the test quality criteria considered. Hence, we recommend standardizing the profile analysis better and adding more sensitive diagnostic criteria for primary school aged children. Other aspects of profile analysis, for example the improvement of diagnostic competencies of teachers, are recommended for future research.