Learning analytics (LA) refers to the process of collecting, evaluating, analysing, and reporting organizational data for decision making (Campbell and Oblinger, 2007). It involves the use of big data analysis for understanding and improving the performance of educational institutions in educational delivery. Open and distance learning (ODL) institutions present an ideal context for the use of LA as, with their large student numbers and the increasing use of the internet and mobile technologies, they already have a very substantial amount of data available for analysis with analytics

Using Learning Analytics to Assess Student Learning in Online Courses

Method

Data Measures

Sample assessment data measures (as referenced in Table 1) were collected from a preservice instructional technology course taught at a southeastern university in the United States. The course was taught in a 15 week time period and had 7 modules. Each module included a variety of instructional components including an elearning module, a quiz, and hands on projects. There were 18 students in this online course and Table 2 below provides the names of the different modules in this course and the different assessments that were used.

table2

Results

Comprehension-Type Assessment

Two different example analyses are presented for comprehension type assessment. Time spent 1

on quiz 1, quiz1 score and frequency (number of times) of access were tabulated and a whisker

plot was drawn (see Figure 4). The maximum score that a student could earn in quiz 1 was 10

points. The quiz included 10 multiple choice items with four item responses. Since this course was

offered 100% online, the students had the option to take this quiz open book. There was no time

limit set for them to complete the quiz. The whisker plot provided representation of variables by

providing the median values for each one of them. In this visual below, the median time spent on

the quiz is between 0.03 and 0.35 (hours), and the median quiz 1 score is between 8 and 9 points,

and the median for the number of times accessed was 4. Analysis like this can provide instructors

with useful information on students’ behavior. For example if you look at the element “number of

times accessed” the figure shows that he lower quartile is 3 times and the upper quartile is 6 times meaning half of the students (50%) accessed the quiz between 3 to 6 times. The average score is 8.44 with the minimum score being 6 and the maximum score being 10. Such information could be useful and meaningful for instructors, as it could be used to benchmark students ‘scores and practices (i.e. time spent on a quiz) to analyze and understand students’ performance in a course.In each of the three areas (time spent on quiz, Quiz 1 score, and number of times accessed), thedots outside the box represent outliers.

Implications and Future Directions

Martin and Ndoye: Learning Analytics for Assessment

The goal of this article is to report the results of a small case study of 18 students in an online environment. Some of the data available from the 18 students’ use of the online learning platform are exported and quantitative and qualitative visualizations of learning analytics data are presented. Furthermore, the information and feedback provided through the use of learningAnalytics data for assessment in online and blended learning could be of great importance to all stakeholders.

As an instructor, taking the time to review the learning analytics data on student activity and assessment was meaningful though the class size was small. It helped identify the students whowere very active in the online class and were spending a lot more time in the LearningManagement System. It also helped identify students who were not as active and who I needed to reach out to both in terms of performance and engagement. Analyzing the data helped with the implementation of similar courses and in designing the assessments in a way that is more beneficial to the students. As an instructor, if I noticed that there was a student who was struggling with a particular module, I reached out to the student to provide additional support. I was also able to reach out to inactive students early on, rather than waiting until the end of the semester to provide support and develop remediation strategies.

While instructors may use such information for effective online teaching, students can also use it to enhance their learning. Instructional designers may use this information to recommend best practices in online course design. Administrators may use this information to design successful online programs. Educational researchers may use this framework to analyze data from the various online assessments within the learning analytics framework. Bringing together these points of view will help improve online teaching and learning.

Future directions for this study will be to conduct research on the effectiveness of these tools. The authors also plan to implement the various data analyses techniques in large enrollment classes in the future.

@ 

Florence Martin

University of North Carolina Charlotte, florence.martin@uncc.edu

Abdou Ndoye

Qatar University, abdou.ndoye96@gmail.com