Dave Bartram (SHL Group, United Kingdom)
The challenge of setting standards for testing and test use in a rapidly changing online assessment environment
Together, globalization and the Internet have fundamentally changed the way tests can be used in occupational assessment. They have also impacted on the types of tests that can be used. In this presentation I will consider some of the key changes in the nature of the occupational assessment environment that globalization and the internet have brought about and how test designers have risen to the challenge of development tests and test delivery mechanisms to cope with these. (Much of what I will cover is also applicable to educational and health-related assessment.) Issues that have been addressed include test design, the use of multimedia, test security and data protection and a host of other matters. The speed of change has posed a real challenge for those involved in trying to set standards of good practice as such standards are typically backward-looking and pose the danger of stifling innovation. Key to these changes has been a shift in the relationship between test provider and test user and the increasing level of control the provider can exercise over delivery. Some thoughts on what the next ten years might bring will be provided.
Eckhard Klieme (German Institute for International Educational Research, Germany)
Current challenges of international student assessment
International comparative assessments of student achievement have been around for about 50 years. While in the early years researchers struggled with validity, curriculum coverage, translation and sampling issues, as well as with operational difficulties, modern psychometrics and testing technology allow for regular rapid assessment, e.g., of reading, mathematics and science, around the globe. Especially, the Programme for International Student Assessment (PISA), commissioned by OECD and mainly run by ACER (Adams et al.), since 2000 has met very high technical standards. However, PISA and other studies are challenged by more and more diverse participation (with now about 70 countries from all continents), demanding constructs (assessing literacy and life skills instead of subject matter knowledge), and challenging policy demands (understanding causes and effects of policies and practices). The presentation will give an overview of current issues in international educational measurement, including the following topics:
- Defining and modeling student competencies that are relevant both from an educational and from a life skill-perspective.
- Using the opportunities of state-of-the-art technology to improve measurement and enhance construct coverage, e.g. by assessing problem solving and collaboration skills.
- Establishing measures - both cognitive and noncognitive - without cultural bias, also allowing for trend analysis, and at the same time understanding - rather than suppressing - cultural differences.
- Building an analytical framework that makes use of assessment data for educational policy making and practice, but also takes into account the limits of cross-sectional surveys.
Ric Luecht (University of North Carolina at Greensboro, USA)
Engineering the design of assessment task models and templates
Assessment engineering (AE) offers a principled approach to assessment design and development practices meant to systematically control and monitor the quality of measurement information produced from the response data. This paper will describe two of the core technologies within the AE framework: task modeling and templating. Task modeling is used to concretely define a unique combination of skills and knowledge objects required to support proficiency claims within a specific region of the construct-based measurement scale. Multiple item-generation templates are then created for each task model making it possible to generate numerous, possibly thousands of test items with similar statistical characteristics. AE task models and templates provide enormous assessment design and development cost and resource efficiencies and lend themselves nicely to sophisticated, hierarchical quality control, calibration and scoring systems.
Marten Roorda (Cito, The Netherlands)
The exciting future of educational testing
In the future educational testing will become more and more computerized. In the future educational testing will return to the classrooms. It will be a tool in the hands of teachers, and will make their life easier. In the future assessment of learning will become assessment for learning. Classroom testing will become a continuous process, inseparable from the electronic learning environment. Evaluation and feedback from testing will become a vital part of the learning process. In the 21st century we must be able to assess 21st century skills. In the future educational testing will be stealth, intelligent and flexible. Large synchronous standardized tests have no future. Global experts agree on all of these trends. Shaping our business to the future will require groundbreaking innovations. Major investments are needed in testing technology. Major inventions are needed in psychometric models and methods. But market size and market forces in educational testing are quite limited. How can we face these exciting challenges?
Fons van de Vijver (Tilburg University, The Netherlands)
Recent developments in international testing
Recent developments to assessment in cross-cultural context will be described. I will describe and illustrate two approaches: (i) the use of instruments that have well established psychometric properties, such as measures of intelligence and personality; (ii) the use of methods that combine emic and etic procedures. Strengths and weaknesses of both are described. In addition, I will deal with the renewed interest in response styles and social desirability in cross-cultural perspective. An overview is given of the cross-cultural patterning of differences in acquiescence, extreme response styles, middle response styles, and social desirability. It will be illustrated that these styles differ systematically across cultures, that these styles are interrelated at country level and that some social and economic indicators are good predictors of these styles at country level. Implications are discussed.
Important Dates and Deadlines
July 3-5, 2012
July 2, 2012 (Pre-Conference Workshops)
Submissions are now closed since 20 January 2012
Early bird registration has been closed on 15 April 2012
Second announcement of conference: