Produced with Scholar

Assessment Theory

Project Overview

Project Description

Write a wiki-like entry defining an assessment concept. Define the concept, describe how the concept translates into practice, and provide examples. Concepts could include any of the following, or choose another concept that you would like to define. Please send a message to both admins through Scholar indicating which you would like to choose - if possible, we only want one or two people defining each concept so, across the group, we have good coverage of concepts.

Icon for Process Oriented Performance and Product Based Assessment

Process Oriented Performance and Product Based Assessment

Background

Process Oriented Performance and Product Based Assessment goes by a number of name variations such as performance based assessment, authentic assessment, porfolio assessment, and many others. Essentially, all these names are attempting to capture variations on the same theme within this type of assessment. For the purposes of this work, this form of assessment will be referred to as process oriented performance assessment.

Process oriented performance assessment has a long history. Prior to the 1920's, process oriented performance assessment or variations of it were used as the primary means of assessment. It took the form of essays, student products, and oral presentations. In the 1920s, in an effort to improve efficiency in assessments, mutiple choice and short answer questions became the dominent form of assessment in schools. Even after the 1920s though, variations on process oriented performance assessment still remained the primary means of assessment in skills based course work such as in painting, sculpture, welding, and sowing. These programs would continue to use variations on this type of assessment as the primary means for assessing learning. (Wolf, 1995) In the 1990s, process oriented performance assessment began to enjoy a resurgence in popularity to a degree. With the advent of "21st century learning", "authentic learning", and "deeper learning", process oriented performance assessment took center stage as a primary form of assessment for these approaches to educational learning.

Explication of the Theory

Process oriented performance assessment is assessment of learning by observing and documenting the process of learning as it occurs and combining this with the production of a performance or an artifact resulting from the learning process. This type of assessment is both formative and summative in nature and can be assessed by self, peers, the instructor, or a committee of individuals or any combination of these.

Process oriented performance assessment uses “authentic tasks that assess what a student knows and can do.” (Caffrey, 2009) With this type of assessment, students are required “to perform a task rather than select an answer from a ready-made list.” (Sweet, 1993)

Process oriented performance assessment by definition includes assessment of the student's learning process as well as the exhibition of an artifact created as evidence of learning in the form of a product or performance.

These two elements of process oriented performance assessment, taken together, offer a tangible means in which to assess learning. The specific rubrics or guidelines for what to assess against remain open to flexibility and creativity based on the needs of the course and the students. This type of assessment is involved, it takes time and effort, and is open to a certain degree of subjectivity but can be especially useful in assessing deeper, more authentic learning.

Process oriented performance assessment is often paired with new, alternative forms of learning that go by a few different names such as Deeper Learning or 21st Century Skills Learning. Bloom's Taxonomy is often referenced when considering the levels of thinking and learning and how they can be assessed. It is argued that multiple choice and short answer assessment tools will only assess the lowest levels of thinking whereas deeper, more authentic learning accompanied by process oriented performance assessment will assess the higher levels of thinking. Process oriented performance assessment is “better suited for measuring “21st Century Skills,” such as creating, critical thinking, problem solving, collaboration, creativity, and innovation.” (Caffrey, 2009) Illustrated in the graphic below are the lower and higher level thinking skills outlined in Bloom's Taxonomy.

Image Source: http://www.curriculet.com/blog/wp-content/uploads/Blooms-Taxonomy.png

Similarly, Webb's Depth of Knowledge levels also illustrate this point. DOK Level 1 can be assessed via multiple choice, short answer, and fill in the blank. As you progress forward, the higher the DOK level, the more innacurrate a standard multiple choice, fill in the blank or short answer test will be at providing an accurate picture of learning. It is these higher levels of learning illustrated in Webb's Depth of Knowledge and in Bloom's Taxonomy that can be successfully assessed using process oriented performance assessment.

Documentation of the Learning Process

Documenting the learning process can be both formative and summative, depending on how it is applied.

Documenting the learning process is formative when it is also used to critique the current state of learning while learning is taking place. Self-reflection can be a form of this. There will always be a certain degree of self-assessment occuring due to the nature of creating an artifact. Should there also be purposeful, documented assessment by self, peers, teachers, or others, the learning process would yield better results as an assessment and learning tool.

This documentation can also be used for summative assessment when the documented learning is included as part of the final assessment that has taken place such as in the example of evaluating a student’s portfolio.

Exhibition of an Artifact

The other element to process oriented performance assessment is the exhibition of an artifact in the form of a product or performance. This exhibition is concerned with both the celebration of learning but also the demonstration of the learning which has occurred. This may or may not be followed up with a formal graded assessment by self, peers, teachers, or others, depending on the specific design and structure of the course. However, the exhibition can be seen as both a demonstration and assessment (formal or informal) of the learning that has taken place as a result of the particular course of study.

Immediate Application of Process Oriented Performance Assessment

Process oriented performance assessment is believed to have a positive effect on teaching and learning. In "Performance Assessment and the New Standards Project: A Story of Serendipitous Success", Elizabeth Spalding notes the potential for performance based assessments to positively influence curriculum and instruction. She believes that "performance based assessments and portfolios look like our best hope for providing meaningful information about the performance capabilities of students and for bringing about institutional change." (Spalding, 2000)

It is believed that learning of a deeper, more relevant, and more authentic nature can only be assessed by taking into account the process of learning as well as the performance or product produced by this type of learning. This type of learning cannot be accurately assessed via a multiple choice exam. Conversely, while the type of assessment should not determine the approach to teaching and learning, learning that is authentic, that reflects a deeper understanding, and that teaches critical thinking, collaboration, problem solving, creativity, and innovation calls for this type of assessment to be used.

The essential difference between traditional learning assessment and process oriented performance assessment is that traditional assessment focuses on questions regarding what can be recalled through memory of facts and figures and, to some degree, understanding. This assessment is confined to assessing lower level thinking skills. Process oriented performance assessment attempts to assess how well students can take the facts and figures of lower level thinking and apply them to real, creative challenges and solutions primarily in the form of creating or doing something. In this model, formative assessment and the learning process become two sides of the same coin and summative assessment is essentially a demonstration of the learning and the formative assessment that has already taken place.

The graphic below illustrates some lower level thinking activies before ballooning into a number of activites that require higher level thinking skills. Traditional forms of assessment such as multiple choice are not equipped to assess these types of higher level thinking skills and activities accurately.

Credit: K. Aainsqatsi

The methods for putting this type of assessment into practice are varied and are goverened primarily by the goals of learning. There are certain elements that are common to all process oriented performance assessment. Those elements being pre-established expected outcome guidelines (pre-established by teacher and/or students), documentation of the learning process, and a final artifact that demonstrates how well a student has grasped and can apply what was learned.

Implementing process oriented performance assessment is dependent not only on the elements listed above but also on a course of study that is willing to go beyond lower level thinking and challenge students to create, think critcally, analyze, and apply. It does not make sense to assess lower levels of thinking with process oriented performance assessment. Given this fact, one must begin with an appropriate approach to learning such as project-based learning, inquiry-based learning, deeper learning, etc. These types of learning approaches all begin with a basic, relevant question or challenge and work backwards from there to expound on and produce an appropriate artifact or performance that will answer the question or challenge presented.

Project Examples

The following list provides examples of some projects that have been done. These can all be found listed as project resources at the Buck Institute for Education website. This list provides examples of some of the specifics involved in these projects including rubrics, stated outcomes, and evidences of learning.

A Most Amazing Race: Folktales From Around The World by Adrianne Lewicki Manning of Teach21 Project Based Learning

Today's Machines by Aaron Commerson, Nicole Tempel of High Tech High

Blocking Sound by Robert Lyons of Teach21 Project Based Learning

Connecting Math To Our Lives by Lynn Rody of iEARN

Create The Perfect Wedding Reception of www.ohiorc.org

Create and Deliver a Winning Video of www.ohiorc.org

Compare Greco-Roman Society with American Society by Brian O'Connell of Teach21 Project Based Learning

Culture Shock by Thisbe Cooper of Teach21 Project Based Learing

School Examples

There are a number of schools that are currently using process oriented performance assessments. A few have been highlighted below:

Metropolitan Arts & Tech High School

Metropolitan Arts & Tech High School in San Francisco, CA uses process oriented performance assessment in their College Readiness Portfolio Defenses. This concept can be seen in the following two videos:

The first video, highlighting the College Readiness Portfolio Defense at Metropolitan Arts & Tech, is hosted on TeachingChannel.org. It is presented from the student, Yvonne’s, perspective. In it, she, her teachers, and the assessing panel talk about this schools unique approach to Process Oriented Performance Assessment in the form of the College Readiness Porfolio Defense. As Yvonne's teacher says in the video: "The defense is it. Everything leads up to the defense. All the work that we do is mapping backwards from the final moment which is when a student stands up in front of a panel of assessors to say 'I am ready for college'."

The panel uses a rubric to guide them in their assessment of Yvonne's portfolio defense. The student uses supporting artifacts, essays and brochures produced by herself, documented self-reflection and self- assessment, and the final presented defense as evidence of her learning. In this instance, the panel and the student are using evidence of both the process of learning as well as evidence based on the final performace to demonstrate and assess the students learning.

Yvonne Student Profile

The second video highlighting Metropolitan Arts & Tech’s College Readiness Portfolio Defense is hosted on TeachingChannel.org. It is presented from the teacher and assessing panel’s perspective and incorporates the same Portfolio Defense found in the Yvonne Student Profile video. It's emphasis is on the process of using the assessment tool to guide the panel in their assessment preparations and decisions.

Calibration Assessing Defense Portfolios

High Tech High School

High Tech High is a school known for its project-based learning models. The video below highlights the schools approach to exhibition and the celebration of learning.

School As A Living Museum

High Tech Middle School

Finally, the Teaching Channel produced this video about “engaging students in work that matters” (video: 12:59) highlighting High Tech Middle School in San Deigo, CA. The key point in this video relevant to assessment is how learning and process oriented performance assessment are so interrelated. It is difficult, impossible even, to separate one from the other.

Engaging Students In Work That Matters

Strengths and Weaknesses of Process Oriented Performance Assessment

As with any form of assessment, there are strengths and there are weaknesses. The value of any assessment tool lies in what it's intended purpose is. The goal of process oriented performance assessment is to accurately assess higher levels of thinking and learning on a smaller, more personal level. When used with this goal in mind, it is an excellent form of assessment. When used to assess for a different purpose, such as normative comparisons of student's learning, it would very likely not produce the hoped-for results as it is not designed for that purpose.

Strengths

Process oriented performance assessment yields more detailed information than multiple choice tests. There is a greater body of knowledge that can be gleaned from this type of assessment since the trial and error of the documented learning process combined with the final product or performance will demonstrate to a much greater degree what was learned, how it was learned, why it was learned, and the depth of the learning. According to Adamson & Darling Hammond, “…research shows that well-designed performance assessments yield a more complete picture of students’ abilities and weaknesses, and can overcome some of the validity challenges of assessing English language learners and students with disabilities." (Adamson & Darling Hammond, 2010)

There is also greater student engagement through process oriented performance assessment. Student engagement can be increased as they reflect on their process of learning as well as demonstrating or documenting that process and the higher order thinking that went into it. [We can] “engage students more in their own learning and interests” through the inclusion of “reflection and demonstration of thinking processes.” (Tung, 2010)

Additionally, when instruction and assessment are combined and integrated into the currirulum, this promotes a greater culture of collaboration amoung students and teachers. [Process oriented performance assessment] “encourag[es] schools to build professional collaborative cultures through integrating curriculum, instruction, and assessment.” (Tung, 2010)

Process oriented performance assessment is also an excellent professional development opportunity for teachers in that it “increase[s] intellectual challenge in classrooms and support[s] higher-quality teaching.” (Adamson & Darling Hammond, 2010) [Teachers] “become more knowledgeable about how to evaluate and teach to challenging standards.” (Adamson & Darling Hammond, 2010)

Finally, as stated earlier, Process Oriented Performance Assessment is “better suited for measuring “21st Century Skills,” such as critical thinking and problem solving skills, collaboration, creativity and innovation.” (Caffrey, 2009) Multiple choice tests will engage lower level thinking skills such as matching, organizing, comparing, and defining. Process oriented performance assessment is much more effective in assessing higher lever thinking skills.

Weaknesses

There can be some problems with using process oriented performance assessment. Madaus and O’Dwyer point out in A Short History of Performance Assessment: Lessons Learned, “Performance assessments are generally less efficient, difficult to administer, more time-consuming, not easily standardized, and additionally test a considerably smaller sample from the overall knowledge domain.” (Madaus and O’Dwyer, 1999)

Process oriented performance assessment is more time consuming and less efficient. One of the primary reasons that schools moved away from this type of assessment in the early 1900s and why standarized testing became the norm from that time on is because process oriented performance assessment is less efficient and more time consuming. This form of assessment struggles to come up with a means of assessing as many students as possible in one shot. It is a much more personal and personalized assessment that requires time for documentation, reflection, discussion, exhibition planning and preparation, presentation, and time for repeated failed attempts and gradual successes. It requires time of the student, class peers, the teacher, other teachers or administrators, and anyone who is assessing or evaluating any part of the learning process. Multiplied exponentially by the number of students, it is easy to understand why this form of assessment can be more time consuming. It is also easy to see why standardized multiple choice and fill in the blank assessments were and still are attractive when assessing large numbers of students at once.

Process oriented performance assessment is not easily standardized. The strength of this type of assessment is also its weakness. It will assess students on a highly personal level and is useful largely in classroom environments. It becomes unwieldy if applied at a multi-school or larger level. This is primarily due to the intent of this form of assessment. Process oriented performance assessment is intended to assess the learning of the student. It is not intended as an normative assessment practice whose goal would be to compare schools, teachers, or students against each other. It is also not easily standardized due to the subjective nature of the assessors involved, ranging from self, to peers, to teachers, and to assessing bodies of various makeups.

Process oriented performance assessment tests a smaller sampling of the knowledge domain. The learning that can be assessed with this form of assessment tends to be focused on specific, relevant knowledge and information. While the knowledge needed to succeed may overlap into other subject matters, students will focus primarily on the knowledge that is relevant to the success of their particular challenge. What is sacrificed with breadth of knowledge is regained with depth of understanding. This can prove to be a difficult balance to maintain and why this form of assessment cannot be standardized at the highest levels of education. Learning that is personal and real to the student can likely only be assessed accurately in such a way that is also personal and real.

Implications of Process Oriented Performance Assessment Translated Into Practice

In an online article entitled Lenz: The 21st Century Skills-Deeper Learning Connection In Assessment, Bob Lenz, Founder and Chief of Innovation at Envision Schools, states that the learning of 21st century skills and assessing those skills are inextricably linked when you engage students directly in assessing their own progress by reflecting on and documenting their own learning. Student reflection, documentation, and self assessment are common ways that contribute to being able to assess learning via Process oriented performance assessment. Lenz goes on to say:

“The reflection step in this on-going learning cycle is where assessment happens. Reflection invites students and teachers to recognize growth and accomplishments as well as identify opportunities for improvement and development. It is not separate from the learning process: it is an integral step on the path to deeper learning. Having students play an active role in this step is distinctive for two reasons:

(1) The assessment process itself helps students develop critical thinking and analysis skills and (2) The process also helps students internalize knowledge, turning what and how they learn into a well of resources they can use in the future.” (Lenz, February 12, 2014)

Self assessment and reflection are only one type of assessment that falls under the umbrella of process oriented performance assessment. Assessing the process and product of learning can come in many forms including:

  • Observation: Observation is used as a form of assessment by self, peers, teachers, or assessment body. It can be formal, informal, and is generally useful for obtaining immediate feedback. For example, students can be observed by their peers or teacher or others as they work on solving a problem, developing a skill, collaborating amongst peers, etc. Students can also be observed in the exhibition of an artifact of their learning.
  • Student Journals (or Blogs): Journals or blogs are useful and similar to portfolios in that students hone their critical thinking and analysis skills through their own writing and reflections. Journals and blogs can also be useful when incorporated into a portfolio as part of a student's portfolio showcase. The journal or blog is essentially a means of self-reflection.
  • A culminating artifact: This part of the assessment is generally summative in nature. It can be complemented by formative assessment evidence if desired. It is primarily the demonstration of learning by means of a completed project, performance, or product. There are numerous options available, limited only by creativity and available resources. Examples could include: artwork, reports, inventions, documentary videos, musical compositions and performances, portfolio defenses, dance recitals, model constructions, maps, puppet show, robotics demonstrations, supporting and reporting on a cause, etc.
  • Portfolios: Portfolios are a method of documenting the process of learning. They can also be used as part of an exhibition of a product or performance.

In an excerpt from Measurement and Evaluation in Human Performance, Fourth Edition with Web Study Guide, by James R. Morrow, Jr., Allen W. Jackson, James G. Disch, and Dale P. Mood, they state that the following are two basic types of portfolios:

  • “Working portfolio—A repository of portfolio documents that the student accumulates over a certain period of time. Other types of process information may also be included, such as drafts of student work or records of student achievement or progress over time.
  • Showcase or model portfolio—A portfolio consisting of work samples selected by the student that document the student’s best work. The student has consciously evaluated his or her work and selected only those products that best represent the type of learning identified for this assessment. Each artifact selected is accompanied by a reflection, in which the student explains the significance of the item and the type of learning it represents.”

Morrow, Jackson, Disch, and Mood go on to state:

“A rubric (scoring tool) should be used to evaluate portfolios in much the same manner as any other product or performance. Providing a rubric to students in advance allows them to self-assess their work and thus be more likely to produce a portfolio of high quality. Portfolios, since they are designed to show growth and improvement in student learning, are evaluated holistically. The reflections that describe the artifact and why the artifact was selected for inclusion in the portfolio provide insights into levels of student learning and achievement.”

Part of the consideration for assessing culminating artifacts is how the final performance or product is celebrated and exhibited. The specifics of how each project or performance is exhibited will vary by project. Music performances will have a temporary demonstration period. Products like artwork or models have the potential to live on after the exhibition. At times, a performance can be translated into a recording in order to create an archive of the student’s work. Exhibition provides for demonstration, motivation, and future inspiration for this type of learning and assessment.

Jill Davison elaborates, in the online article “Exhibitions: Demonstrations of Mastery in Essential Schools”, (2007), about how exhibition of a product or performance is utilized for four key purposes:

  • “to ensure engagement among students, staff, and the larger community
  • to assess student learning and, therefore, school effectiveness authentically
  • to raise the stakes for students and a school in ways that support and advance an Essential school’s goals
  • to create schools that express the Common Principles (Demonstration of Mastery) in everything that they do”

Davidson continues:

“Just as an exhibition is an authentic piece of work from a particular student, exhibitions at a school are an authentic reflection of that school, dependent on a school culture, not just the product of a particular class. Schools that do this work successfully know that they need to devote time for the effort and habits of mind that exhibitions demand, provide suitable professional development, and use assessment methods that provide information that fuels continuous opportunities for reflection and improvement. Real tasks that are linked to real outcomes comprise exhibitions. Students and educators identify and agree in advance on what constitutes success in ways that are reliable and valid.” (Davidson, 2007)

Chris W. Gallagher, in "Reclaiming Assessment: A Better Alternative to the Accountability Agenda", concurs:

“engagement, not accountability, should both be the means to and end of school improvement. Public demonstrations (exhibitions) of mastery constitute an engagement system for the entire school and serve a higher purpose as demonstrations of what’s possible to communities, policy makers, government officials, district personnel, other schools, and the media.”

21st Century Tools For Process Oriented Performance Assessment

Due to the depth and breadth of skills and knowledge that can be explored via project based or similar types of learning, the 21st century tools that can be used to aid in process oriented performace assessment are difficult to narrow down in one comprehensive list. For this reason, only four examples are highlighted below.

  1. Photography, Video recording and editing software: Digitial still and video photography and accompanying editing software are a very common means of documenting the learning process and / or exhibiting learning with a culminating artifact. For obvious reasons, a camera is an excellent tool for recording stills, video and audio for later reference, archival, or manipulation. Cameras on smartphones and tablets make video and photography extremely common place. Apple, Inc., says that its iPhone camera is the world's most popular camera. With it's iMovie App, the iPhone has the potential to make engaging, high quality videos such as Richard Dunn has done here. The possibilities go up from there with $1000 cameras now supporting 4K video. Adobe PhotoshopAdobe Premier or Apple's Final Cut Pro make all the advanced post production capabilites students could ever need available to them. Even a simple device such as a Chromebook has the ability to create quality photo and video through Google Chrome Webstore Apps such as Pixlr and WeVideo. The attached video below is an example of a culminating artifact of a thematic dance choreography project done by the Dance Department at Lyndon Institute in Lyndon Center, Vermont. A higher resolution version can be seen by clicking here.
    Thematic Dance Project, Lyndon Institute, VT
  2. Sound recording and production equipment: Similar to digital photo and video recording, sound recording and editing is an excellent way to record audio for use in assessing both the process and the performance of music or presentations or speeches or conversations. Sound recording can also be useful as an embedded aspect of video recording.
  3. Social media tools: The possibilities are fairly limitless when it comes to using social media for process oriented performance assessment specifically in the self, peer, and teacher review process. Twitter, Facebook, Google+, Instagram, Pinterest, YouTube, Soundcloud, and LinkedIn are some examples. Social media creates a web of interconnected relationships that provide an excellent platform for showcasing artifacts for assessment by self and others. For example, post a tweet on twitter for class members to view a discussion within a Google+ community that incorporates Youtube videos and soundcloud audio clips that are then reviewed and commented on by other members which are made up of the actual class's students. Meanwhile, in another class, students post photos on Instagram and pin them and other currated webcontent on pinterest and then send out a tweet to the rest of the class to comment or add their own content to the discussion and collection. 
  4. Blogs and ePortfolios: ePortfolios can take the form of a website created on platforms such as Wordpress, Weebly, or Google Sites. Blogging platforms are also an excellent way to build an ePortfolio. Text and various multimedia such as photos, weblinks, documents, audio, and video can be used in creating a blog. Blogging platforms can stand alone as the Kidblog platform does or blogs can be incorporated into other technologies such is the case with Blogger with Google+.
    Kidblog Homepage

Conclusion

In conclusion, process oriented performance assessment is concerned with assessing a student's learning by observing and documenting the process used to accomplish learning as well as by assessing a student's learning based on a final product or performance. Many specific forms of assessment and rubrics can be used in a formative and summative nature to complete the full assessment picture such as self assessment and reflection, peer, teacher, or independent assessing body analysis, portfolios, journals, etc. This type of assessment is linked very closely to higher level, deeper, more authentic types of learning. While exhibition of an artifact is not necessarily an automatic outflow of higher level learning and while it must be intentional, the product or performance that results from this type of learning is able to be assessed whether it's exhibited or not. The assessment of the process of learning is also a natural outflow of deeper, more authentic types of learning through the use of reflection and documentation simply because the process of completing a product or performance can be observed, reflected upon and evaluated by self, peers, teachers, and others. These types of assessments have the challenges of being more time consuming and less efficient then other types of assessment but they are also able to more accurately portray a truer picture of a student's learning on a given topic.


References

Adamson, F., & Darling-Hammond, L. (2010, April). Beyond Basic Skills: The Role of Performance Assessment in Achieving 21st Century Standards of Learning. Stanford, CA: Stanford Center for Opportunity Policy in Education.

Caffrey, E. D. (2009). Assessment in Education and Secondary Education: A Primer.Washington: Congressional Research Service.

Davidson, Jill. (2007). Exhibitions: Demonstrations of Mastery in Essential Schools

Gallagher, Chris W. (2007) Reclaiming Assessment: A Better Alternative for the Accountability Agenda.

Lenz, Bob. (2014) LENZ: THE 21ST CENTURY SKILLS-DEEPER LEARNING CONNECTION IN ASSESSMENT.

Morrow, James R., Jr., Jackson, Allen W., Disch, James G., Mood, Dale P., (2011). Measurement and Evaluation in Human Performance, Fourth Edition with Web Study Guide

Spaulding, Elizabeth. (2000). Performance Assessment and the New Standards Project: A Story of Serendipitous Success.

Sweet, D. (1993, September). Performance Assessment. (Office of Research, Office of Educational Research and Improvement (OERI) of the U.S. Department of Education.) Education Research Consumer Guide: http://www2.ed.gov/pubs/OR/ConsumerGuides/perfasse.html

Tung, R. (2010). Including Performance Assessments in Accountability Systems: A Review of Scale Up Efforts. Center for Collaborative Education and the Nellie Mae Education. 

Wolf, Richard M., (1995). The role of performance assessment in studies of educational achievement. Reflections On Educational Achievement , 277