Simulated Computer Adaptive Testing Methodology Choices for Ability Estimation with Empirical Evidence

Abstract

Computer Adaptive Testing (CAT) is a technological advancement for educational assessments which requires thorough feasibility studies through computer simulations in a bid to have strong foundations for a testing programme. This is especially germane in Africa trying to catch up with technology in the digital age and this should not be done blindly without empirical evidence. The quasi-experimental design was adopted for this study with the aim of establishing methodological choices for CAT ability estimation being the most important aspect of CAT performance evaluation of a testing program. 5000 examinees were simulated with 100 items simulate through the 3-parameter logistic model. The simulation design stipulated a fixed-length test of thirty (30) items while examinee characteristics were drawn from a normal distribution with a mean of 1 and Standard Deviation of 0. Also, controls for the simulation was were set not to either control for item exposure or to using the progressive restricted method. Data gathered were analysed using descriptive statistics (mean and standard deviation) and inferential statistics (Two-way Multivariate Analysis of Variance: MANOVA) for testing the generated hypotheses. Findings of this study will provide empirical evidence for choosing methods with respect to ability estimation for CAT as part of the efforts geared towards designing a testing programme.

Presenters

Jumoke Oladele
Postdoctoral Research Fellow, Department of Sciences and Technology Education, Faculty of Education, University of Johannesburg, Eastern Cape, South Africa

Erica D. Spangenberg
Deputy Head of Science and Technology Education; and Senior Lecturer (Mathematics Education), Science and Technology Education, University of Johannesburg, South Africa

Mdutshekelwa Ndlovu
Vice-Dean Research & Postgraduate Studies, Faculty of Education, University of Johannesburg , Gauteng, South Africa

Details

Presentation Type

Paper Presentation in a Themed Session

Theme

Assessment and Evaluation

KEYWORDS

Computer Adaptive Testing, Simulation, Ability estimation, Item response theory

Digital Media

Downloads

Presentation Handout

Presentation_Hanout_Zoom_link.pdf

Presentation PowerPoint

PRESENTATION_PowerPoint.pdf