“I had a student that was struggling and I could not figure out what she was struggling with. But IXL's Analytics allowed me to see that what she couldn't do was add and subtract integers—she could multiply and divide them—but not add and subtract. So without that data, I would never have been able to figure out where she was struggling”

Nancy Foote, teacher

Sossaman Middle School, Higley, AZ

Testimonial on https://www.ixl.com/analytics/:

The following Text illustrates software for differentiated instructions by presenting the web-paged www.ixl.com with its associated analysis tools which offer a possible solution to the introductory question. The underlying educational paradigm of www.ixl.com as well as the design-principles of the software are contrasted with Problem Based Learning (PBL) (see Hmelo-Silver, Duncan & Chinn, 2007 for an overview).  

www.ixl.com offers test-questions to skills required in math, language arts, science and social studies for K1-12 based on the common core: 

Figure 1: Example of seventh grade math skills (https://www.ixl.com/math/grade-7/percents-of-numbers-and-money-amounts)

The webpage uses a gamification approach by letting students extensively practice the required skills. This allows the student to earn “smart scores” (see Figure 2) and at the same time generates a big amount of data which will then enable the teacher to spot problem areas of each student and helps individualize teaching. (following film explains the possibilities of the analytics: https://youtu.be/5y-ymj1C9Jc)

Figure 2: example of a question with an increase in smartscores for correct answers. (https://www.ixl.com/math/grade-7/percents-of-numbers-and-money-amounts)

Although I do not exactly know the logic behind the analytics it is most likely that it is some kind of computer adaptive testing (see Cope and Kalantzis 2016, p. 4). Furthermore it can be assumed that every single question is deconstructed into its single components and every necessary definite mathematical operation. All the involved operations are known and with the huge amount of generated data the pattern of the mistakes will become obvious. This might help to find the answer to my introductory question: “how do I get to know what I don’t know”. As a consequence this might help a teacher find the root cause of a students’ problem and enable differentiated instruction – as shown in the testimonial above. Although this is a sophisticated and powerful tool, it still follows a didactic pedagogy (Cope and Kalantzis, 2016).

In terms of the current course on e-learning affordances which is more in favor of reflexive pedagogy (Cope and Kalantzis, 2016) it would be interesting to examine if and how methods like Problem Based Learning (PBL) (watch https://youtu.be/4Xr8AQWH75M for an example of PBL in a math – class) could offer similar insights into the learning progress and uncover possible problem areas of a student. According to Hmelo-Silver et al. (2007) PBL is highly scaffolded and therefore could basically allow similar analysis (although possibly not at such a detailed level). One important question is if such insights have to be generated individually by a teacher / expert with his experience and his accurate sighting of misconceptions. This is an important question because if individual human expert feedback is necessary it means that the instructional unit is less scalable and requires more human resources to achieve an adequate quality. Without being able to give a final and evidence based answer I would argue that in a PBL environment finding root-causes of problem areas depends largely on the investigation of someone who has mastered the topic as well as the rules and procedures of a discipline and is able to evaluate appropriately. This is mainly due to the fact that the process and outcomes of PBL are more complex and less predictable. Students’ results or knowledge artefacts might be viable but not entirely correct. Shortcomings in detecting these errors might be harmful for students’ learning process as they might not notice their misconceptions in important key areas. 

More detailed search for available scientific results and analysis software is necessary to give a more elaborate answer if tools in the domain of natural language processing or an equivalent for mathematical reasoning are available to allow for systematic analysis and uncovering of students’ misconceptions and problem areas. Similarly, it has to be examined how self-evaluation and peer feedback in software such as cgscholar has to be designed in order to either really uncover problem areas of a student or at least avoid wrong believes of mastery. 

References

Cope, B., and Kalantzis, M. (2016). Conceptualizing e-learning. In B. Cope and M. Kalantzis (Eds), e-learning ecologies. New York: Routledge (fortcoming). Retrieved: February 4th 2017

Hmelo-Silver, C. E., Duncan, R. G., & Chinn, C. A. (2007). Scaffolding and achievement in problem-based and inquiry learning: A response to Kirschner, Sweller, and Clark (2006). Educational psychologist, 42(2), 99-107. Retrieved: February 4th 2017

IXL: Homepage: www.ixl.com | Youtubechannel: https://www.youtube.com/user/IXLLearning