Produced with Scholar

Technology Mediated Learning Analysis

Project Overview

Project Description

'Parse' a learning technology - what is its underlying theory of learning and how is this reflected by the way it works in practice? When discussing the theory of learning read and cite (with links) the theorist works (Work 1) of other course participants.

Icon for Learning Analytics

Learning Analytics

Why Learning Analytics?

For education to be accountable, both teachers and schools need to be invested in continuous improvement. If student success is the goal, then determining what leads to success should be a valued part of attaining that goal. However, understanding the various factors that lead to either learning success or failure is a challenge of grand proportions.

The assessment movement that focused educators on measuring learning outcomes is an example of a broad scale effort to both hold education accountable as well as improve teaching and learning to foster student success. Birthed by technology’s ability to capture and organize large data sets related to education, learning analytics are now another commonly used tool to improve teaching and learning. Learning analytics are defined as, “…the measurement, collection, analysis, and reporting of data about learners and their contexts, in order to understand and optimize learning and the environments in which it occurs” (Scapin, 2015, slide 14).

Big Data and the History of Learning Analytics

The internet, widespread computer and smartphone use, and social media are major producers of user data. The growth and abundance of this user-generated information is often referred to as “big data”. Big data is defined as, “Extremely large data sets that may be analyzed computationally to reveal patterns, trends, and associations, especially relating to human behavior and interactions” (Scapin, 2015, slide 3).” It is also referred to as data sets that are too large to be analyzed by traditional means. Analytics systems were developed to manage and organize this data to make it useful for understanding and responding to online human behavior and interactions. Long and Siemens (2011) refer to the information collected by analytics as “data trails” (p. 32). The corporate world has long been investing time and money in analyzing big data.

Similar to the enormity of data accessible by corporations, educational institutions now also have access to what Romero and Ventura (2010) have aptly described as “ a gold mine of educational data” (p. 1). The use of big data analytics in education is a fairly new development and has mostly occurred in the 21st century (Ferguson, 2012). Research communities and conferences focusing on big data in education can only be traced for about a decade (Siemens & Baker, 2012). 

Academic Analytics and Learning Analytics: There’s a difference

The terms academic analytics and learning analytics can be easily confused, but they are very distinct types and uses of big data. While both are used in education, academic analytics are based heavily on business analytics and are intended to inform administration. Academic analytics are commonly used for decision making in areas such as finance, operations, and organizational efficiency. They can be used to make decisions around resource allocation, can increase productivity, and can decrease the time required to respond to institutional challenges (Long & Siemens, 2011, p. 36). Academic analytics are often used to get a broad picture of an entire institution or region.

On the other hand, learning analytics generally provide data from a particular course or program and they are used to inform and improve teaching and learning specifically. Instead of being used by administration, learning analytics are used by teachers, students, and course designers. A popular use of learning analytics among instructors is identifying students who are at risk of failing or doing poorly. Learning analytics can similarly be used by students themselves to assess their progress, learning, and sometimes to compare themselves to their peers in a course. 

What Learning Analytics Measure

The data that feeds learning analytics originates from many places. The following is an incomplete list:

  • Time spent in the entirety of a course
  • Time spent in each course activity
  • Social interaction
  • Use of course resources
  • Assignment completion
  • Assignment grades
  • Detailed assignment grades such as which questions students got right or wrong
  • Number of discussion posts
  • Number of logins

When Instructors Use Analytics

Retrieved from CollegeStats.org https://s3.amazonaws.com/infographics/Data-Mining-Analytics-Education-800.png

Learning analytics are most commonly used to identify learners at risk of doing poorly in a course, including those that are at risk of failing. They allow instructors to make interventions that can lead to student success by helping students to meet learning outcomes. An example of this would be the use of learning analytics to shape personalized learning experiences including content, pace, objectives, and method of instructional delivery. When an instructor uses learning analytics, their role expands to include analyst as well as facilitator. Their teaching becomes more student-focused because their actions are based on students' actions and they are providing personalized feedback and interventions. For example, if a teacher can quickly look at analytics for a course and identify which students are not performing well or not participating enough, the teacher can reach out to those students, offer suggestions, resources, or encouragement, provide alternative or additional learing materials, or set up one-on-one help. 

Learning analytics can also be used by instructors to group students and to find the most frequently made mistakes (Romero & Ventura, 2010). In this way, analytics can help teachers provide in-the-moment changes to content or pedagogy classwide. Learing analytics can also be used to monitor collaborative student work and provide strategies on how to collaborate better in group projects to either individuals or groups (Leeuwen, Janssen, Erkens, & Brekelmans, 2014).

Learning analytics are also used to predict individual students’ future behavior and performance, a use that can be controversial. By evaluating “historical student data” analytics can also be used to create “predictive models” for both successful and unsuccessful student types (Scapin, 2015, slide 28). However, in their oft-cited article about learning analytics, Long and Siemens (2011) promoted the idea that learning analytics are most useful when they actually transform students and learning:

It is not sufficient to treat big data and analytics as useful only for evaluating what learners have done and for predicting what they’ll do in the future. Analytics in education must be transformative, altering existing teaching, learning, and assessment processes, academic work, and administration. (p. 38)

When Students Use Analytics

Learning analytics can also be used by students, and when course designers and instructors make this possible, the goal is to increase achievement. Verbert, Duval, Klerkx, Govaerts, and Santos (2013) found that student use of dashboards can increase grades and retention. They outline four stages of effective use of analytics by students in their “process model” (p. 1501): awareness, reflection, sensemaking, and impact. In short, referring to the data leads to awareness, reflection and sensemaking produces questions and answers about the data, and impact brings meaning and implemented behaviors (p. 1502). In their research, they also discovered that students find dashboards more useful when they provide a more complete and full picture of student activity (p. 1506). 

Data Visualization

To make data useful for both teachers and students, leaning analytics often use dashboards and graphs. This is referred to as data visualizations. They are intended to make the use of learning analytics less time consuming and more efficient than looking at raw data. In some applications, the goal of data visualizations is for the teacher to quickly be able to tell which students are at-risk and in need of an intervention.

There are different ways that learning analytics can be set up for students. In some models, all students receive instant access to data visualizations of their learning progress. In other models, alerts inform students when they are doing poorly. Alerts can be set to happen automatically and be received by all students, or they can be an option whereby students need to opt-in to receive alerts. Taken a step further, alerts can also be tied to required tasks that need to be completed (Scapin, 2015, slide 29).

The video below of Moodle Learning Analytics highlights the following features used by one university: 

  • Learner access (Instructors): list of learners with their last access date/time
  • Outline report (Instructors): how many times they accessed resources, which ones they’ve accessed, grades
  • Activity report (Instructors): See how many students have accessed or completed an activity, or search for a particular student and activity
  • Completion tracking (Instructors and students): see their progress for activity completion
  • Dashboard (Instructors): See the last seven days of activity including forum posts and assignment completions.
  • Correlation reports and Learning engagement trends reports (Instructors)
  • Personalized Learning Experience (Students): send automatic notifications to students based on certain actions or “triggers”, such as low test scores or when a student hasn’t accessed the course in a while.
Media embedded September 6, 2015

For other examples of learning analytics and how data is visualized, the following videos for BlackBoard and Adobe Presenter 9 are included below:

BlackBoard: 

Media embedded September 6, 2015

Adobe Presenter 9: 

Media embedded September 6, 2015

 

Learning Theory and Learning Analytics

In some ways, because learning data can be used to “shift patterns of behaviors in desirable ways” (Wagner & Ice, 2012, p. 38), analytics can be considered to have at their core a behaviorist view of learning. Again from the learner perspective, if adult learners are using analytics to reflect and create new learning schemes, then analytics might also be related to Mezirow’s Transformative Learning Theory. Similarly, the reflection on their own learning could be indicative of Flavel's Metacognition Theory. If their reflection leads to motivation to perform better, then Self Determination Theory pertains, as the combination of visible dashboard analytics and formative feedback can motivate some students both intrinsically and extrinsically. 

If we view learning analytics as a tool used by educators to shape and personalize strategies for particular students via interventions, then learning analytics can be viewed through the lens of constructivism. In this particular application, the interventions are generally formative feedback. If it's taken a step further, and personalized instruction is differentiated by the different ways students interact with and understand the world, then Gardner's Multiple Intelligences Theory can also apply. 

Criticism of Learning Analytics

Privacy is a top concern among critics of learning analytics. What rights do students have regarding the data they generate? Regarding consent, Ferguson (2012) states, “There is no agreed method for researchers to obtain informed and ongoing consent to the use of data, and there are no standard procedures allowing learners to opt out or to have their analytic record cleared” (section 9.4). Besides the alerts features in some models, the rest of student participation in big data collection happens with their consent. Should students get to choose whether their analytics are visible to the instructor? Should the instructor only be able to view anonymous data? Does a balance need to be achieved between the individual who might feel “digitally stalked and manipulated” and the organization who might depend on data for its survival (Wallace, 2014)?

Profiling and biases are another concern. Will data identifying a student as a low achiever stay with the student from course to course? What kinds of biases could this fuel? Will they be encouraged to take classes below their aptitude level? Will they be discouraged from taking challenging classes?

Questions of responsibility have also been raised. If the outcome of analytics recommends that students take certain actions to mitigate their coursework or grades, what happens if they ignore those recommendations? Where does the responsibility fall if they follow the recommendations but are still unsuccessful?

There's also a concern that analysis of learning data may lead to inaccurate or questionable conclusions and recommendations. Another concern is oversimplification. Booth warns that without careful implementation and a focus on learning assessment, learning analytics runs the risk of, “becoming a reductionist approach for measuring a bunch of ‘stuff’ that ultimately doesn’t matter” (2012, p. 52). Booth calls for learning analytics to be used as a part of a larger assessment whole and acknowledges that analytics can’t see or measure all learning (p. 52). Long and Siemens (2011) point out that learning analytics can’t measure online students’ behavior outside of the LMS, for example, their use of online library databases, social media, or internet searching. They also don’t capture offline behavior such as physical trips to the library, academic advisors, or writing center. 

 

Conclusion

It is important for educators and administrators to remember that the intelligence in analytics, “is in the interpretation of data by a skilled analyst” (Scapin, 2015, slide 33). In other words, educators need to be trained and willing to use analytics appropriately. Administration needs to be provide instructors with adequate knowledge and resources to use learning analytics successfully. It is important that those managing and using analytics focus on what they need to know to make decisions about teaching and learning, and what particular data is needed to best provide the needed knowledge (Long & Siemens, 2011, p. 32). Big data provides teachers, students, and administrators with enormous amounts of data, which used correctly, can be immensely helpful and foster student success. Learning analytics will likely continued to be fine-tuned and in turn be increasingly useful but concerns such as privacy, biases, and misuse need equal attention.


References

Booth, M. (2012, July 18). Learning analytics: The new black. EDUCAUSE Review, 52-53. Retrieved from http://er.educause.edu/

Ferguson, L. (2012). Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5/6) 304–317. doi: 10.1504/ijtel.2012.051816

Learning Solutions Magazine. (n.d.). Learning analytics in Adobe Presenter 9: Tips for early intervention. Retrieved from http://www.learningsolutionsmag.com/articles/1277/learning-analytics-in-adobe-presenter-9-tips-for-early-intervention

Long, P., & Siemens, G. (2011). Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, 31-40. Retrieved from http://er.educause.edu/

Romero, C., & Ventura, S. (2010). Educational data mining: A review of the state-of-the-art. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 40(6), 601-618. doi: 10.1109/tsmcc.2010.2053532

Scapin, R. (2015, April 27). Learning analytics in education: Using students’ big data to improve teaching [Slides]. Retrieved from http://www.slideshare.net/rscapin/learning-analytics-47463622

Siemens, G., & Baker, R. S. (2012). Learning analytics and educational data mining: Towards communication and collaboration. Proceedings of the 2nd international conference on learning analytics and knowledge (ACM 2012). Retrieved from http://users.wpi.edu/~rsbaker/LAKs%20reformatting%20v2.pdf

Van Leeuwen, A., Janssen, J., Erkens, G., & Brekelmans, M. (2014). Supporting teachers in guiding collaborating students: Effects of learning analytics in CSCL. Computers & Education, 79, 28-39. doi:10.1016/j.compedu.2014.07.007

Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist, 57(10), 1500-1509. doi: 10.1177/0002764213479363

Wagner, E., & Ice, P. (2012). Data changes everything: Delivering on the promise of learning analytics in higher education. EDUCAUSE Review, 33-42. Retrieved from http://er.educause.edu/

Wallace, M. (2014, November 20). Privacy by design: Humanizing analytics [Video file]. Retrieved from https://www.youtube.com/watch?v=8JLzs_xVKxY