Medical Education’s Updates

Item-Based Testing and Psychometrics (Admin Update 2)

Item-based, standardized tests have epistemological and social bases.

Their epistemological basis is an assumption that there can be right and wrong answers to the things that matter in a discipline (facts, definitions, numerical answers to problems), and from the sum of these answers we can infer deeper understanding of a topic or discipline. (You must have understood something if you got the right answer?) Right answers are juxtaposed beside 'distractors'—plausible, nearly right answers or mistakes it would be easy to make. The testing game is to sift the right from the (deceptively) wrong.

The social basis of item-based tests is the idea of standardization, or tests which are administered to everyone in the same way for the purposes of comparison measured in terms of comparative success or failure.

Psychometrics is a statistical measurement process that supports generalizations from what is at root survey data. (An item-based test is essentially, a kind of psychological survey, whose purpose is to measure knowledge and understanding.)

Today, some standardized tests, such as PISA and TIMMS aim to evaluate higher order disciplinary skills.

Comment: When are standardized tests at their best? And/or worst?