Produced with Scholar

Work 1: Educational Theory Literature Review

Project Overview

Project Description

Take one of the theories or theoretical concepts introduced in this course. Look ahead into the course learning module to get a sense of upcoming ideas—don’t feel constrained to explore concepts introduced early in the course. Or explore a related theory or concept of your own choosing that is relevant to the course themes.

Theoretical and Empirical Literature Review: Your work must be in the genre of a literature review with at least 5 new scholarly sources (peer reviewed journal articles or scholarly books) that you have not previously used in this or other courses. Of course, in addition to these five, you will reference previously used sources and other media. In the references section, you should add an asterisk in front of every new scholarly source.

Convey in your introduction how your topic aligns with the course themes and your experience and interests. Outline the theory or define the concept referring to the theoretical and research literature and illustrate the significance of the theory using examples of this concept at work in pedagogical practice, supported by scholarly sources.

Rubric: Use the ‘Knowledge Process Rubric’ against which others will review your work, and against which you will do your self-review at the completion of your final draft. You will find this rubric at the end of this document, and also in CGScholar: Creator => Feedback => Rubric.

Word length: at least 2000 words

Media: Include at least 7 media elements, such as images, diagrams, infographics, tables, embedded videos, (either uploaded into CGScholar, or embedded from other sites), web links, PDFs, datasets or other digital media. Be sure these are well integrated into your work. Explain or discuss each media item in the text of your work. You should refer to specific points of the video with timecodes or the particular aspects of the media object that you want your readers to focus on. Caption each item sourced from the web with a link and be sure to cite all media sources in the references list.

References: Include a References “element” or section with the scholarly articles or books that you have used and referred to in the text, plus any other necessary or relevant references, including websites and media.

Important Note: The First Draft means a complete first version of your Work!

Icon for Know Thyself

Know Thyself

The Use and Efficacy of Self-Assessments for Learning

Image 1. John Dewey: American philosopher, psychologist, and educational reformer whose still influential ideas made him one of the most prominent scholars of the 20th century. (image credit: Thinking Pathways, 2022).

Many years ago I read Robert Pirsig's semi-autobiographical novel, Zen & The Art of Motorcycle Maintenance, one of the memorable moments from which was his recounting when, as a professor of writing and rhetoric, he eliminated graded feedback (although grades were kept) for his undergraduate students in favor of only providing substantive encouragement and/or constructive criticism so they could reflect and revise, much in accord with the educational philosophy of John Dewey (see Image 1). According to Pirsig's fictional alter-ego, this resulted in grade-conscious over-achievers and those seeking to coast through with a C alike dropping the class, leaving him with an ardent group of "B students" who just wanted to learn. These students -- the ones who had struggled the most in class at first -- ultimately persevered to grow exponentially in their enthusiasm, abilities, and confidence as the class progressed. Pirsig's retort to an administration focused more on the two-thirds of students who dropped the class than the one-third who claimed it was transformative serves as an apt indictment of standard assessments as well as a synopsis of his experiment and defense of its continuation as a best practice: 

Grades really cover up failure to teach. A bad instructor can go through an entire quarter leaving absolutely nothing memorable in the minds of his class, curve out the scores on an irrelevant test, and leave the impression that some have learned and some have not. But if the grades are removed the class is forced to wonder each day what it’s really learning. The questions, What’s being taught? What’s the goal? How do the lectures and assignments accomplish the goal? become ominous. The removal of grades exposes a huge and frightening vacuum. (Pirsig, 1974).

Although not necessarily wanting to instill in my students an existential crisis, the consequent deep self-reflection that could have an impact on students far beyond a class' content resonated with me. I was not a teacher at the time, but a recent college graduate living a relatively bohemian existence with a strong sense that I someday would be a teacher, once I felt I had something to teach, and Pirsig's sentiments here I knew needed to be put in my proverbial backpack for later.

Ironically, fast forward a few decades, and I was designing and delivering a year-long class to explore existential questions alongside students at the Darien, CT campus I helped launch for Newport Academy. Newport's students are often gifted, typically from well-resourced families, but who are also in a residential or intensive outpatient setting as they recover from addictive behavior disorders (e.g., eating disorders, self-harm, substance abuse, co-occuring disorders, etc.).  (Newport Academy, 2022). As discussed in detail in an earlier work (Wilson, 2021), this course was designed to provide AdvancED accredited alternative content in the humanities (language arts and social sciences) that (a) would not be as triggering for students of trauma as much of what was commonly assigned in such courses, and (b) would also prove engaging and relevant for a population for who for Camus' essential question really was their daily struggle: "Should I kill myself?" (Aronson, 2021).

That course included the following (Attachment 1) essential questions, design statement, and grading rubric, which I posited (Wilson, 2022) as a potential foundation for reflective assessment as described by Drs. Cope and Kalantzis (2018).

Attachment 1: Laughing at Life: Essential Questions, Design Statement & Grading Rubric

This Pirsig-inspired design and rubric worked extremely well for this class and in the context of Newport Academy in terms of students' formative engagement and ultimate buy-in to the summative assessment and final grade, as well as overall community feedback (students, staff, and families). This was somewhat unexpected considering that students also reported that this course was amongst the most academically and personally challenging material they encountered, as well as my surprise finding that over time the course still tended to naturally gravitate toward a bell-shaped curve for final grades (though no failures). My own reflection leads to the following hypothesized causal explanations for these observations:

  1. According to this rubric, demonstrating effort alone provided a baseline safety net for Newport's traumatized and disengaged student body coming in, while higher level attainment remained possible only by meeting standards that, while simpler and self-reflective, were neither less specific nor demanding than what commonly exists in many humanities courses/assignments;
  2. Due to Newport's co-clinical/educational model, we had a lot of historical and in-the-moment quantitative as well as qualitative data on students at the outset as well as the capacity and imperative to incorporate that data into highly individualized learning plans.

However, in positing these materials to my doctoral peers in this course, a credible criticism was offered against the grading rubric's potential subjectivity. It may be implied from that criticism that there is a skepticism against self-reflection generally when used for assessment as opposed to just part of the learning process. A counterargument to this criticism is that if self-reflection is not part of ubiquitous, formative assessment (Cope & Kalantzis, 2018) it risks being made an ancillary aspect of the class, as commonly seen in the plethora of "un-rubriced", highly subjective student self-reflection worksheets available online for students to execute and (maybe) hand in after an assignment is done. (see Google search results, 2022). This unfortunately common practice would appear to de-prioritize self-reflection in the eyes of students, and maybe even make the exercise seem like a pro forma waste of time, leading to its avoidance even if more robustly supported and required in some future class. That stated, however, my counterargument serves to highlight my one critical peer's explicit complaint, namely "attempts to measure or score effort ... [and]  assumptions based on known abilities ... are the very definition of subjective ... cannot be known by anyone outside of the student ... [and] are meaningless in a mastery oriented system." (comment to Wilson, 2022).

Of course, my experience demonstrates that this colleague's comments cannot always be true, but that experience also begs the question: are self-reflective assessments meaningful outside of a class on existentialism and a student body like Newport Academy's?

In other words, and to end our introduction with the question at the core of this literature review: what is the evidence for the efficacy of self-reflective assessment in improving mastery based learning for conventional course subjects in non-specialized educational environments?

Toward that end, the body of this work will be divided into four sections:

  • Definition for self-reflective assessment
  • Theories of education underlying the practice
  • Key concepts derived from its study congruent with assessment for learning
  • Gaps in the literature

 

Definition

Media embedded February 12, 2022

VIDEO 1. (JFF, 2013).

JFF (Jobs for the Future) is a $40M (Schwenke, 2013) "national nonprofit that drives change in the American workforce and education systems to achieve economic advancement for all." (JFF, 2022). In Video 1, JFF demonstrates the anecdotal advantages of incorporating student self-reflection into a course's assessment, with a clear outline of potential best practices:

  • Students report being more open to realizing potential to improve on their own than when they receive criticism for being yet to meet standards (0:00-0:10)
  • Self-assessment ≠ subjective summative self-grading, but a formative process of reflection guided by a comparison of performance to clear criteria (0:20-0:45)
  • The purpose of self-assessment is to inspire student ownership of the revision process that research suggests is essential to and enhances all forms of learning and instruction (0:50-1:08)
  • The application of these three principles is demonstrated through a step-by-step Language Arts example (2:30-3:25)
  • In conclusion, reflective self-assessment treats content and process mastery not as the goal, but as the means to beyond-the-class and beyond-the-classroom goals ostensibly aligned with JFF's mission to ensure that "workforce and education systems achieve economic advancement for all" (cf. 4:42-4:52 with JFF, 2022)

Thus JFF proposes the following definition for students' reflective self-assessment:

A practical method of helping students identify strengths and weaknesses in their own work and revise accordingly in order to gain the skills they need to become independent, self-directed, life-long learners. (JFF, 2013).

Interestingly, the University of Pittsburgh's Center for Teaching and Learning recommends this same practice for professors, as an ongoing iterative process to improve student learning through teacher self-assessment, which they define as:

[Concrete methods that] allow instructors to reflect upon and describe their teaching and learning goals, challenges, and accomplishments ... [t]he format of [which] varies and can include reflective statements, activity reports, annual goal setting and tracking, or the use of tools like the Wieman Teaching Practices Inventory. (University of Pittsburgh, 2022).

Cornell University's Center for Teaching Excellence provides similar definitions for both students' and teachers' reflective self-assessment, accompanied by the consequent following guidance (Figure 1):

Figure 1. (Cornell University, 2022).

The George Lucas Educational Foundation (Edutopia), citing practice at New Mexico School of the Arts (NMSA), further details these rationales and considerations with concrete steps for varied application according to a 3-step process that adds meat to the bones of JFF's general claim that self-assessment is comprised of "practical methods of helping students identify strengths and weaknesses in their own work and revise accordingly." Namely, Edutopia and NMSA promote:

  1. Showing examples of mastery
  2. Teaching vocabulary specific to students' craft
  3. Providing ample opportunity to practice peer critique

(Minero, 2016).

Thus the various perspectives cited in this section complement each other and, as such, provide a clear definition for reflective self-assessment with implementation details we can identify and evaluate in terms of fidelity of practice and, where fidelity is found, efficacy of outcomes. Furthermore, they promisingly comport, not only with Pirsig's initial intuition and anecdotes from his undergraduate writing and rhetoric course that introduced our topic, but recent research that seems to suggest he was correct in his ad hoc experiment, observations, and analysis; namely, that students learn most when all numeric and letter grades are withheld, leaving them with nothing but the "huge and frightening vacuum" of purely qualitative feedback. (Booth, 2017).

From this firm foundation, let's now let's look at the theories of education that support the approach we have defined and review the literature examining its application.

Theory

Image 2. (Cope & Kalantzis, 2018)

Although specifically examining the enhanced affordances provided by e-learning ecologies, in Image 2 Cope & Kalantzis (2018) ask educators to consider those affordances' impact on assessment. First, as Dr. Cope demonstrates and diagrams in Image 3 below (2018), these affordances restructure the relationships in a learning community, resulting in less teacher-initiated outflow of information/knowledge to students in favor of student-initiated information gathering and knowledge production that only flows to the teacher for review/assessment after the benefit of significant peer-2-peer interaction and feedback.

Image 3. Restructuring the "classroom" relationships. (Cope & Kalantzis, 2018).

The implications of these restructured relationships is the integration of assessment into -- rather than at the end of -- the learning experience process so that, to borrow a phrase from JFF's definition for self-assessment, "students [can] identify strengths and weaknesses in their own work and revise accordingly." (JFF, 2013). Examining Image 2 again, it is clear why leveraging e-learning ecologies' affordances -- as opposed to ignoring those advantages, restricting e-learning to mere teacher replacement (see e.g., Bushweller, 2020) or remote replication of the 20th century classroom (see e.g., Elevate K12, 2022) -- would naturally result in the form of student reflection JFF promotes. After all, three of the seven affordances, or principles, provided in Drs. Cope & Kalantzis' new learning agenda are directly and sequentially related to student reflection: 

  1. Active knowledge making, whereby students are an integral part, and not just a recipient, of a course's instructional design, forging their own meanings (which, as a further note, is predicated on and supportive of the affordances ubiquitous and differentiated learning, as well as collaborative intelligence);
  2. Recursive feedback, which supplants summative done-and-forgotten tests with ongoing formative assessment;
  3. Metacognition, or thinking about thinking, which is the very definition of reflection.

Thus, whether we choose or have the capacity to adopt e-leaning ecologies, their prospect has outlined for us a new assessment agenda, as demonstrated by the transformation outlined in Image 4 below. (Cope & Kalantzis, 2018).

Image 4. The potential transformations for assessment e-learning ecologies afford. (Cope & Kalantzis, 2018).

An examination of this emerging assessment model on the right of Image 4 makes it evident that nearly century on, we have the opportunity -- the realization of which will arguably best be afforded within an e-learning ecology -- to test educational philosopher/reformer John Dewey's assertion that opened this work; namely, that "we don't learn from experiences, we learn from reflecting on the experience." (Dewey, 1933).

Thus we come to the literature to ask: has this been tested, and, if so, is there evidence that self-assessment produces its definitionally stated end goal of more independent, self-directed, life-long learners?

Key Concepts from the Literature: Findings & Critique

Media embedded February 14, 2022

VIDEO 2. Assessments that don't formatively engage students undermine the curriculum (1:43-2:15). (University of Technology Sydney, 2017).

The research of Dr. David Boud (Video 2) -- Alfred Deakin Professor and Foundation Director of the Centre for Research in Assessment and Digital Learning, Deakin University; Professor in the Work and Learning Research Centre, Middlesex University; and Emeritus Professor at the University of Technology Sydney -- traces 40 years of the evolving use of self-assessments in education. (University of Technology Sydney, 2022). His early-career meta-analysis of the field (Falchikov & Boud, 1989) revealed several insights whose application are evident in how JFF (2013) currently defines the criteria for real self-assessment (Video 1) as well as the rationale and considerations for its use in high school and higher education (see e.g., infra University of Pittsburgh, 2022; Cornell University, 2022; Minero, 2016). Namely, the early studies that were subject to Boud's review moved the field toward those positive definitions by identifying and moving us away from the following problems with pre-1990 self-assessments:

  1. Poor design and implementation of assessments and their own evaluation resulted in poor correspondence between self- and teacher-grading;
  2. Students in introductory courses appeared to need more supports (e.g., scripts, rubrics) if they were to approximate the accuracy of assessment attained by students in more advanced courses;
  3. The more quantitative the course -- or the more course content provided clear procedures for student activity toward universally expected, objective outcomes -- the more accurate students' self assessments (i.e., studies in the sciences appeared to produce more accurate self-assessments generally than did those in the humanities);
  4. The overall challenge in students' ability to accurately self-assess that is the theme to these issues suggests that self-assessment is (a) improved when complemented by some form of external feedback; and (b) more promising for use as a formative as opposed to summative tool (i.e., as stated in Video 1, self-assessment ≠ self-grading (JFF, 2013)).

(Falchikov & Boud, 1989).

Unfortunately, even at the dawn of the 21st century, many educators and researchers appear to have missed this memo, the former continuing to use self-assessment as a less-rubriced "how do you think you did" summative tool while the latter then lambasts such poorly designed and implemented assessments for their failure to predict subsequent performance, whether on a conventional test or in the field.

For example, Ward et al. (2002) reviewed the use of self-assessments in physician continuing education and found that "despite the accepted theoretical value of self-assessment, studies have consistently shown that the accuracy of self-assessment is poor ... [due to] methodological issues that plague the measurement of self-assessment ability and ...  common pitfalls encountered in the[ir] design." However, those "methodological issues" and "common pitfalls" were the same for which Falchikov & Boud (1989) had suggested remedies well over a decade earlier. Nonetheless, Ward's (2002) sentiment continued to be echoed by others, as in Dunning et al.'s (2004) article "Flawed Self-Assessment", the arguments of which make it unclear whether the authors chose their title because they investigated self-assessments that were flawed or as an assertion that self-assessments are inherently and categorically flawed by definition. Indeed, the flaws they outline continue to be what self-assessments' proponents might argue is the pedagogical equivalent of blaming a screwdiver for not being a hammer (i.e., using a formative tool as a summative test (Falchikov & Boud, 1989; JFF, 2013; University of Technology Sydney, 2017)), even while admitting that later studies by Boud et al. (1991, 1995) appear to demonstrate proper self-assessments' value as a formative tool that encourages and empowers students "to become more autonomous agents in their education, taking responsibility for gaining and improving on their knowledge and skill" ... "particularly as some schools move to a problem-based or case-based model of instruction."

In fact, contemporaneously with Ward (2002) and Dunning et al. (2004), Dr. Boud (McDonald & Boud, 2003) was collaborating on a study of an allegedly "not fundamentally flawed" application of self-assessments. In this study over 500 students spread across 10 participating high schools in four broad academic subject areas -- Business Studies, Humanities, Technical Studies, and Science -- were divided evenly into control and experimental groups. Addressing Boud's earlier identified challenges to self-assessment (1989), teacher and student training in the design and proper contextual use of self-assessments was provided to the experimental group prior to those assessments' administration. The control group did not receive this training and did not use self-assessments. At the conclusion of the test period, both the control and experimental group took the external, criterion referenced examinations in the aforementioned subject areas that are compulsory for all high school students in the West Indies, where the study was conducted.

The results of this study are provided in Table 1 below:

Table 1. (Table II from McDonald & Boud, 2003).

As McDonald & Boud (2003) analyzed and summarized this data:

"It can be seen that the experimental group had considerably higher means in all curricular areas (Business Studies, Humanities, Science and Technical Studies) ... The t-tests and p-values for the experimental and control groups in all curriculum areas of academic achievement were statistically significant at the .01 level, indicating that the differences in performance of the treatment groups were not due to prediction errors nor sampling errors ... All differences between the means of academic achievement for the experimental and control groups had a greater effect on academic achievement in ... Humanities (d 0.54) ... than on Science (d 0.27). [Because prior studies (see infra, Falchikov & Boud, 1989) found science subjects to be more amenable to accurate self-assessment than those in the humanities,] this meant that self-assessment training significantly differentiated the treatment groups in terms of their academic achievement."

In more plain terms, when students and teachers were trained in the administration and execution of well-designed formative self-assessments, on subsequent standardized summative testing they significantly outperformed students in classes that did not use such self-assessments, regardless of content area and controlling for other factors.

These promising findings have been supported by subsequent studies.

First, Andrade & Valtcheva (2009), prefacing their review of several treatment and control studies on the use of self-assessments with struggling elementary grade reading and math students with the assertion that -- echoing Falchikov & Boud (1989) and presaging the term's subsequent definitions (e.g., JFF, 2013) -- "self-assessment must be a formative type of assessment, done on drafts of works in progress ... not a matter of determining one's own grade", concluded that when so administrated the consequent:

Blurring [of] the distinction between instruction and assessment through the use of criteria-referenced self-assessment can have powerful effects on learning. The effect can be both short-term, as when self-assessment influences student performance on a particular assignment, as well as long-term, as students become more self-regulated in their learning. (Andrade & Valtcheva, 2009).

In fact, between the treatment cohorts in these studies' significantly better performance on subsequent standardized tests and their positive feedback on the self-assessment process itself -- which in these cases, as in the McDonald & Boud (2003) study, included teacher and student training on the design and use of such assessments that went far beyond "just passing out a rubric" -- Andrade & Valtcheva (2009) felt compelled to enumerate a series of best practices:

  1. Define the criteria by which students assess their work
  2. Teach students how to apply the criteria
  3. Give students feedback on their self-assessments
  4. Help students use self-assessment data to improve performance
  5. Provide sufficient time for revision after self-assessment
  6. Do not turn self-assessment into self-evaluation by counting it toward a grade.

Andrade & Valtcheva (2009) concluded that "under these conditions, criteria-referenced self-assessment can ensure that all students get the kind of feedback they need, when they need it, in order to learn and achieve."

This conclusion -- particularly its careful caveat that it is only supported "under these [6] conditions" -- could also be said to accurately synopsize the findings of Panadero et al. (2012) in their mixed-methods study of treatment and control cohorts of grade school students performing a short geography exercise. Basically, in all regards -- observation and survey data on students' self-regulation and self-efficacy as well as a post-test standard summative assessment for whether students had learned the geography in question -- Panadero et al. (2012) found that they could not tease apart various elements that could be equated to Andrade & Valtcheva's (2009) conditions. In other words, each of rubrics, feedback, a self-assessment tool, revision opportunity, etc., alone had litte to no significant impact on any measures of student performance; however, when all of these elements were provided in concerted combination, their cumulative impact was significant. In fact, it was significant enough to cause these researchers to ponder whether length of treatment -- i.e., applying these elements over the course of an entire school year as opposed to a discreet geography exercise completed in an afternoon -- might not have yielded results that were even more profound. (Panadero et al., 2012).

Of course, the implications of these studies' design and execution is that student self-assessment done right constitutes a lot of teacher time and effort, despite its name implying that somehow teachers might delegate to students one of their major and majorly time consuming responsibilities. More specifically, teachers must first design a real and contextualized assessment. Second, that assessment must be grounded in a task-specific rubric. Third, students must be trained pre-task on how to use that rubric and self-assessment individually and in concert. Fourth, once the task begins, frequent and formidable substantive feedback must be provided to students regarding their tasks, their self-assessments, and on their process with both. Finally, all this must be done over the course of repeated iterations to accomodate the meaningful revision through which self-assessment's benefits accrue.

This is a daunting -- and perhaps deal-breaking -- list of new responsibilities for an overwhelmed US educational system that has for decades proven immune to any attempts at reform (Wilson, 2021B), which the failure of the aforementioned articles to adequately address brings us to our next section: Gaps in the Literature.

Gaps in the Literature

Media embedded February 16, 2022

VIDEO 3. (PBS Digital Studios, 2017).

Considering that Dr. Boud cited the teacher capacity problem that concluded our last section over three decades ago (Falchikov & Boud, 1989), it's surprising that over the course of his subsequent research neither he nor the authors of the many other studies and meta-analyses we examined chose at any point to give serious consideration to educational technology's capacity to remedy that issue.

Video 3 above -- itself the product of Crash Course and PBS Digital Productions, progenitors and examples of leveraging tech for education -- highlights tech's particular potential for alleviating the otherwise intractable conundrum of a single teacher providing timely, meaningful feedback and assessment for dozens or hundreds of students (see video at 3:48-9:35). 

Unfortunately, despite a few innovative studies on the promise of computer-mediated feedback in what are already non-traditional, or at least somewhat specialized, settings (e.g., music composition (Chen & O’Neill, 2020); the EFL classroom (Vakili & Ebadi, 2019); public speaking training (Van Ginkel et al., 2020)), in terms of core academic instruction in US public schools the most common tools in, uses of, and feedback for students from ed tech is nothing like what Video 3 describes. To the contrary, they remain far more rudimentary reproductions of the conventional classroom's multiple-guess summative quiz that is the antithesis of what we have come to identify as real self-assessment (cf. infra with Rahayu & Purnawarman's (2019) examination of Quizziz as well as this dialogical stream within CGScholar).

Indeed, if this is the drill-n-grill foundation being laid at present, the future of ed tech in the classroom would seem to look less like the reflective companion for self-assessment envisioned in Video 3 (see 9:38-10:24), and more like the pervasive and invasive command-and-control vision The Wall Street Journal discovered is already becoming a reality in China (see Video 4 below).

Media embedded February 16, 2022

VIDEO 4. (The Wall Street Journal, 2019).

However, it would seem that such Orwellian monitoring of students' literal every breath wouldn't be necessary if they were simply engaged in a manner that studies of self-assessment as we've defined it suggests is possible. 

Furthermore -- and ironically -- the technology necessary to make that future possibility for self-assessment a present reality seems far less advanced or expensive than the network of wearables and biometrics highlighted in Video 4. Furthermore, as evident in a simple schema from one such technology platform, the University of Illinois Urbana-Champaign's CGScholar, (see Image 5 below), the most effective tech innovation to facilitate this would shift the onus of feedback to the student peer community. Although not discussed therein, I'm certain any intrepid educator watching Video 4 cringed at the data overload being thrown at instructors to process and act upon. Indeed, note that in that context we saw precious little instruction and no peer-2-peer interaction, just a lot of independent work and teachers' computer-mediated monitoring. (The Wall Street Journal, 2019).

Image 5. The reflective, feedback infused self-assessment process in CGScholar. (Cope & Kalantzis, 2018)

Thus the gaps in the literature seem to suggest a gap in practice that is in turn due to a gap in vision: anyone who would call Quizziz and its kind an innovation in assessment or seek to enhance student performance on such tests through the people-as-product measures evinced in Video 4 clearly are not interested in getting their class "to wonder each day what it's really learning" in the manner espoused by Robert Pirsig at the outset of this article. No, instead of removing grades from the class to cause students and teachers alike to confront the "huge and frightening vacuum" of "what’s being taught? what’s the goal? how do the lectures and assignments accomplish the goal?" (Pirsig, 1974), the entire continuum from Quizziz to the extremes of Video 4 makes the classroom about nothing but the grades.

Sadly, this dichotomy between Pirsig's educational utopia -- a place that is simultaneously ideal and non-existent -- and the less-than-ideal but all-too-possible dystopian vision of Video 4 brings us to this work's conclusion. 

Conclusion

Nearly 50 years ago Robert Pirsig speculated that "if the grades are removed [a] class is forced to wonder each day what it’s really learning. The questions, What’s being taught? What’s the goal? How do the lectures and assignments accomplish the goal? become ominous. The removal of grades exposes a huge and frightening vacuum." (Pirsig, 1974).

Over the course of four decades, researchers like Australia's Dr. Boud and his many collaborators tested this idea (Falchikov & Boud, 1989; Boud & Feletti, 1991; Boud, 1995; McDonald & Boud, 2003;  University of Technology Sydney, 2017), contributing to a body of work that ultimately led to a definition of meaningful student self-assessment in education:

A practical method of helping students identify strengths and weaknesses in their own work and revise accordingly in order to gain the skills they need to become independent, self-directed, life-long learners. (JFF, 2013).

The detailed application of such "practical methods" has yielded details of best practices (Cornell University, 2022; University of Pittsburgh, 2022) that, when followed, have proven superior to other forms of assessment and education, both according to standard measures of achievement and qualitative student feedback (McDonald & Boud, 2003; Andrade & Valtcheva, 2009; Panadero et al., 2012; Vakili & Ebadi, 2019; Chen & O’Neill, 2020; Van Ginkel et al., 2020).

Furthermore, advances in technology since Pirsig's (1974) proposition, Boud's (1989) meta-analyses of nascent self-assessments, and even early 21st century lamentations of self-assessment's labor intensivity for teachers (Ward et al., 2002; Dunning et al., 2004) demonstrate how plausible, efficient, and educationally transformative widespread implementation could be. (Cope & Kalantzis, 2018).

However, gaps in the literature reflect the gaps in practice: from the rudimentary Quizziz (Rahayu & Purnawarman, 2019) to the cutting edge of China's bid to become the pioneer for AI in education (The Wall Street Journal, 2019), most educators and technologists do not seem interested in creating transformative education that engages students, but tighter controls to compensate for the lack of engagement endemic to the same didactic practices John Dewey railed against a century ago (Dewey, 1933). 


References

Academic (Scholarly & Trade) Articles and Sites 

* Andrade, H., & Valtcheva, A. (2009). Promoting learning and achievement through self-assessment. Theory Into Practice, 48(1), 12–19. https://doi.org/10.1080/00405840802577544

Aronson, R. (2021, December 13). Albert Camus. Stanford Encyclopedia of Philosophy. Retrieved March 6, 2022, from https://plato.stanford.edu/entries/camus/

* Booth, N. (2017). What is formative assessment, why hasn’t it worked in schools, and how can we make it better in the classroom? Impact (2514-6955), 1(1), 1–7.

* Boud, D., & Feletti, G. (1991). The challenge of problem-based learning. Kogan Page.

* Boud, D. (1995). Enhancing learning through self assessment. Kogan Page.

* Bushweller, K. (2021, September 15). Teachers, the robots are coming. but that's not a bad thing. Education Week. Retrieved March 6, 2022, from https://www.edweek.org/technology/teachers-the-robots-are-coming-but-thats-not-a-bad-thing/2020/01

* Chen, J. C., & O’Neill, S. A. (2020). Computer-mediated composition pedagogy: Students’ engagement and learning in popular music and classical music. Music Education Research, 22(2), 185–200. https://doi.org/10.1080/14613808.2020.1737924

Cope, W., Kalantzis, M., Francis, K., Tzirides, A., Mattingly, S. (2018). Assessment for Learning.  Retrieved February 15, 2022, from https://cgscholar.com/bookstore/works/copy-of-assessment-for-learning-version-1?category_id=higher-education-modules&path=higher-education-modules%2F160

* Cornell University. Self-assessment. Center for Teaching Innovation. (n.d.). Retrieved February 12, 2022, from https://teaching.cornell.edu/teaching-resources/assessment-evaluation/self-assessment

* Dewey, J. (1993). How we think: A restatement of the relation of reflective thinking to the educative process. Henry Regnery.

* Dunning, D., Heath, C., & Suls, J. M. (2004). Flawed self-assessment. Psychological Science in the Public Interest, 5(3), 69–106. https://doi.org/10.1111/j.1529-1006.2004.00018.x

Elevate K12. Live Streaming Instruction: Elevate K12. (n.d.). Retrieved February 12, 2022, from https://www.elevatek12.com/.

* Falchikov, N., & Boud, D. (1989). Student self-assessment in Higher Education: A meta-analysis. Review of Educational Research, 59(4), 395–430. https://doi.org/10.3102/00346543059004395

* JFF. Jobs For the Future. (n.d.). Retrieved February 12, 2022, from https://www.jff.org/.

* JFF. Self-assessment: Reflections from students and teachers. YouTube. (2013, August 22). Retrieved February 12, 2022, from https://youtu.be/CkFWbC91PXQ

* McDonald, B., & Boud, D. (2003). The impact of self-assessment on achievement: The effects of self-assessment training on performance in External Examinations. Assessment in Education: Principles, Policy & Practice, 10(2), 209–220. https://doi.org/10.1080/0969594032000121289

* Minero, E. (2016, October 4). 4 steps of student self-assessment. Edutopia. Retrieved March 6, 2022, from https://www.edutopia.org/practice/mastering-self-assessment-deepening-independent-learning-through-arts

Monroe, J. (2022, February 23). A healing center for teens and families. Newport Academy. Retrieved February 12, from https://www.newportacademy.com/

* Panadero, E., Tapia, J. A., & Huertas, J. A. (2012). Rubrics and self-assessment scripts effects on self-regulation, learning and self-efficacy in secondary education. Learning and Individual Differences, 22(6), 806–813. https://doi.org/10.1016/j.lindif.2012.04.007

* PBS Digital Studios. Educational technology: Crash course computer science #39. (13 Dec 2017). Retrieved February 16, 2022, from https://www.youtube.com/watch?v=zTi3_l5h5PQ

* Rahayu, I. S., & Purnawarman, P. (2019). The use of Quizizz in improving students’ grammar understanding through self-assessment. Proceedings of the Eleventh Conference on Applied Linguistics (CONAPLIN 2018). https://doi.org/10.2991/conaplin-18.2019.128

* Schwencke, K. & Tigas, M. (2013, May 9). Jobs for the future inc - nonprofit explorer. ProPublica. Retrieved February 12, 2022, from https://projects.propublica.org/nonprofits/organizations/61164568

* The Wall Street Journal. How China is using artificial intelligence in classrooms | WSJ. (1 Oct 2019). Retrieved February 16, 2022, from https://www.youtube.com/watch?v=JMLsHI8aV0g

* University of Pittsburgh. Self Assessment. University Center for Teaching and Learning. Retrieved February 12, 2022, from https://teaching.pitt.edu/resources/assessment-of-teaching-self-assessment/.

* University of Technology Sydney. David Boud P 1. YouTube. (2017, May 8). Retrieved February 12, 2022, from https://youtu.be/rkI0sVmMIQQ

* University of Technology Sydney. Profiles.uts.edu.au. Retrieved February 12, 2022, from https://profiles.uts.edu.au/David.Boud/about.

* Vakili, S., & Ebadi, S. (2019). Exploring EFL learners` developmental errors in academic writing through face-to-face and computer-mediated dynamic assessment. Computer Assisted Language Learning, 1–36. https://doi.org/10.1080/09588221.2019.1698616

* Van Ginkel, S., Ruiz, D., Mononen, A., Karaman, C., Keijzer, A., & Sitthiworachart, J. (2020). The impact of computer‐mediated immediate feedback on developing oral presentation skills: An exploratory study in virtual reality. Journal of Computer Assisted Learning, 36(3), 412–422. https://doi.org/10.1111/jcal.12424

* Ward, M., Gruppen, L. & Regehr, G. Measuring Self-assessment: Current State of the Art. Adv Health Sci Educ Theory Pract 7, 63–80 (2002). https://doi.org/10.1023/A:1014585522084

Wikimedia Foundation. (2022, February 16). John Dewey. Wikipedia. Retrieved February 12, 2022, from https://en.wikipedia.org/wiki/John_Dewey

Popular Media and Author's Prior Work

Google. (n.d.). Google search. Retrieved February 12, 2022, from https://www.google.com/search?q=self%2Breflection%2Bfor%2Bstudents&rlz=1C1UEAD_enUS944US944&source=lnms&tbm=isch&sa=X&ved=2ahUKEwjq3MTEuPr1AhWzHzQIHfh_Db4Q_AUoAXoECAEQAw&biw=1368&bih=712&dpr=2.

Pirsig, R. (1974). Zen and the art of motorcycle maintenance: An inquiry into values. William Morrow and Company.

Wilson, J. (2021, December 16). Laughing at Life: Exploring Existential Questions through (mostly) Funny Films - Lesson Plan 5 of 14: Big Fish. CGScholar. Retrieved February 12, 2022, from https://cgscholar.com/community/profiles/jwilson6/publications/241625.

Wilson, J. (2021, October 10). Ch-Ch-Ch-Changes: A business management perspective on US school-level ed reform implementation. CGScholar. Retrieved February 12, 2022, from https://cgscholar.com/community/profiles/jwilson6/publications/238199.

Images

Routines for self reflection. THINKING PATHWAYS. (n.d.). Retrieved February 12, 2022, from https://thinkingpathwayz.weebly.com/routinesselfreflection.html