Abstract

During the COVID-19 pandemic, teaching and testing witnessed an unprecedented shift to the online environment on a global scale. Assessing the reliability of designed tests by using statistical analysis for research purposes is crucial; however, qualitative insights into teachers’ actual testing practices, which identify the reliability and validity of their own generated e-test assessments, are scarce. This study employs a thematically-analyzed qualitative case study design, including online semi-structured interviews of twenty-eight teachers from five Omani higher education institutions. The findings indicated that the concepts of reliability and validity alone do not offer realistic criteria that can be applied when designing any e-test or online assessment tool. Moreover, technological tools such as Turnitin, are unsupportive in constructing reliable and valid e-tests where multiple choice questions are frequently used. Additionally, this study considered the impact of teacher-related, environment-related, and student-related factors on the reliability and validity of e-tests. Therefore, this study shows that there is a need to develop alternative approaches to ensure that e-test assessments are reliable and valid.