This Learning Module explores basic concepts in assessment, applying these concepts to the latest educational assessment technologies.
Intelligence or Standards - what are the differences in assessment paradigm? A good place to start is to explore the history of intelligence testing (which was actually, the first modern form of testing):
Standards-based approaches are quite different, compare for instance:
And if you would lile to read deeper into a contemporary version of this debate, contrast Gottfredson and Phelps with Shenk in the attached extracts.
Comment: What are the differences between testing intelligence and testing to standards? When might each approach be appropriate or innappropriate?
Item-based, standardized tests have epistemological and social bases.
Their epistemological basis is an assumption that there can be right and wrong answers to the things that matter in a discipline (facts, definitions, numerical answers to problems), and from the sum of these answers we can infer deeper understanding of a topic or discipline. (You must have understood something if you got the right answer?) Right answers are juxtaposed beside 'distractors'—plausible, nearly right answers or mistakes it would be easy to make. The testing game is to sift the right from the (deceptively) wrong.
The social basis of item-based tests is the idea of standardization, or tests which are administered to everyone in the same way for the purposes of comparison measured in terms of comparative success or failure.
Psychometrics is a statistical measurement process that supports generalizations from what is at root survey data. (An item-based test is essentially, a kind of psychological survey, whose purpose is to measure knowledge and understanding.)
Today, some standardized tests, such as PISA and TIMMS aim to evaluate higher order disciplinary skills.
Comment: When are standardized tests at their best? And/or worst?
Criterion referenced, norm-referenced and self-referenced assessments have fundamentally different logics and social purposes. In the following image from Chapter 10 of our New Learning book, we attempt to characterize the different logics. But what are the different social assumptions?
Comment: What are the social assumptions of each kind of assessment? What are the consequences for learners? For better and/or for worse, in each case?
Now, here's a provocative thought! John B. Watson founded the behaviorist tradition in psychology based on the idea that consciousness could not be measured. That would be too self-referential. Instead, he argued, let's examine tangible behaviors. Behaviorism fell out of favor in the last quarter of the twentieth century, and for good reason. We returned to pychologies of mental states once derided by the behaviorists.
But is it time to think again about this? What are the difficulties of assessments that focus on cognition? In these approaches, long term memory becomes a proxy for cognition. Another proxy for cognition is an individual's capacity to reason as evidenced in the calculation of correct answers. However, following are some alternatives to this perhaps excessive focus on mental states—and how do we assess these? It may be the artifact that we assess, not the stuff that is remembered but stuff that is compiled from many sources. It may be collaborative works that are assessed, where the knowledge of a working group is greater than the sum of the knowledge of its members. It may be disciplinary practice and complex knowledge performance that we measure.
Comment: How do we assess these higher-order competencies, and evidence in knowledge artifacts?
This week's topics:
Make your own update on one of these topics, and also, comment below: What do you think about the assumption that all learners in a class should be more or less the same? It was, still is, a pragmatic necessity? Created more problems than it solved? Works less as a strategy than it used to? What role can/should assessment play?
Here's a paper on big data that we recently completed.
Comment: What are the possibilities, and the worrying things, about big data in education?
Write a utopian or dystopian scenario (imaginary or based on a reality you have experienced), a snapshot of the experience of assessment in an ideal or disastrous future.
Write a wiki-like entry defining an assessment concept. Define the concept, describe how the concept translates into practice, and provide examples. Concepts could include any of the following, or choose another concept that you would like to define. Please send a message to both admins through Scholar indicating which you would like to choose - if possible, we only want one or two people defining each concept so, across the group, we have good coverage of concepts.
If you are new to Scholar, do view the introductory videos here.
Analyze an assessment practice. This could be a description of a practice in which you are or have been involved, or plans you have to implement an assessment practice, or a case study of an interesting assessment practice someone else has applied and that you would find beneficial to research and analyze. Use as many of the theory concepts defined by members of the group in their published Work 1 as you can, with references and links to the published works of the other course participants.