### Project Overview

• Publisher: EPSY 556 (2014) Analysis of Advanced Instructional Technologies
• Start: Sep 23, 2014
• Due: Oct 14, 2014

#### Project Description

Analyze advanced instructional technologies in practice in a learning context. It’s quite possible, or even likely, that an ensemble of technologies may be used in a certain educational practice. This could be a description of a practice in which you are or have been involved, or plans you have to implement a practice using instructional technologies, or a case study of an interesting practice someone else has applied and that you would find beneficial to research and analyze. Use as many of the theory concepts defined by members of the group in their published Work 1 as you can, with references and links to the published works of the other course participants.

# Development

The Monterey Institute for Technology and Education (MITE), a non-profit corporation, was originally founded in 2003. The principle project of MITE is the NROC Project, a "community-guided effort focused on new models of digital content development, distribution and use." NROC (National Repository of Online Courses) is funded by The William and Flora Hewlett Foundation, the Bill & Melinda Gates Foundation, and by nationwide NROC member institutions.

In the last ten years, NROC membership has grown and now includes a wide variety of middle schools, high schools and colleges that represent over 6 million students nationwide. NROC produces and compiles high quality technology-based learning tools to support three key projects - NROC Math, HippoCampus, and EdReady. The majority of this paper will focus on EdReady, but this introduction will first explore NROC Math and HippoCampus because the three projects address different educational needs and support one another.

Currently, NROC Math includes two online free math courses. In the K-12 arena, there is a Common Core aligned Algebra I course that can be used as a stand alone course or to supplement a curriculum. For higher education, there is a self-paced Developmental Math course that prepares students for college-level math courses.

HippoCampus is the second element of the NROC online support resources. A more complete description from the NROC website is as follows:

HippoCampus.org is a free, core academic web site that delivers rich multimedia content--videos, animations, and simulations--on general education subjects to middle-school and high-school teachers and college professors, and their students, free of charge. Teachers project HippoCampus content during classroom learning and assign it for computer labs and homework. Students use the site in the evenings for study and exam prep. Users do not need to register or log in to use the site.

Hippocampus includes over 5700 free videos in four key subject areas - Math, Natural Science, Social Science and Humanities. Below are a list of the subjects and the NROC partnerships that provide videos, animations and simulations for use by HippoCampus students and teachers.

Hippocampus Subjects

EdReady builds upon the offerings in NROC Math and HippoCampus and provides college math preparation resources for middle school, high school and college age students. EdReady's three audiences are students, teachers and institutions. The services provided to each are as follows:

 Students "Students can test for college readiness, see study options, and get a personalized study path to fill in their knowledge gaps." Teachers "Teachers can personalize learning online and in the classroom to impact student success." Institutions "Institutions use EdReady to better prepare students for academic and career success, and to accelerate students through developmental education requirements."

EdReady provides students - middle school through college - with resources to study math from pre-algebra through trigonometry and calculus. Students can use these tools to prepare for college entrance or in a classroom setting. College age students can assess their math skills and identify if their skills match the requirements of various colleges. To prepare for the ACT or SAT, students can assess their current skills, determine gaps and then use the EdReady resources - videos, worked examples and practice problems - to improve. Alternatively, students can use EdReady to prepare for remedial college math placement tests such as the Accuplacer or COMPASS. Lastly, students from middle school through college can use EdReady to supplement their school curriculum. After assessing their skill levels, from pre-algebra through calculus, they can access videos, worked examples, and practice problems to improve their knowledge.

The Venn Diagram below shows how EdReady leverages the math resourcs in NROC Math and HippoCampus. NROC's 3 projects - NROC Math, EdReady, and HippoCampus - each have specific areas of specialty but also overlap and are used to support one another. For example, the NROC Math Algebra I and Developmental Math videos are included in HippoCampus and EdReady. Additionally, the Khan Academy and other math videos could support the EdReady or HippoCampus student.

Venn Diagram of Resources from EdReady, HippoCampus, and NROC Math

In summary, EdReady is NROC's free web-based tool that can help students prepare for or succeed in college-level math classes. EdReady can support an individual student or an institution. The remainder of this paper will principally focus on an individual student's experience with EdReady as explained in this video available on the EdReady website.

After a discussion of EdReady's functionality and the supporting learning theories, this paper will explore an individual student's experience with the tools and discuss instutional support through NROC membership. Lastly, this paper will identify potential future enhancements to EdReady, especially those that could leverage connections between NROC Math, HippoCampus and EdReady.

# Structure and Functionality

EdReady states that the tools can be used to support educators and learners from middle school through college entry - as soon as they begin considering their math readiness for college. While the tool could be used as young as middle school and could be used by educators and learners, the primary users of EdReady will be 16-to-19-year old students who are on the verge of college entry. For these students, EdReady focuses students on the achievment of one or more of the following three goals.

Goal 1: A student can identify colleges he or she may be interested in and then check his or her math readiness for each college because "readiness" can differ among colleges.

Goal 2: A student can take a diagnostic test for remedial college math placement tests. Separate diagnostic assessments are provided for Accuplacer and COMPASS. A third general remedial math placement diagnostic test is also provided. After completing the diagnostic test, a student can follow a customized study plan including video lessons, examples and practice problems to prepare for the remedial math placement assessment.

Goal 3: A student can take a diagnostic assessment for either the ACT or SAT and then follow a customized study plan including video lessons, examples and practice problems to prepare for either test.

Watch this video for a Student Overview of EdReady.

Screencapture of Overview Video of EdReady for the Student

EdReady is designed as an item-based formative assessment, wherein a student's ability or knowledge is measured based upon "right or wrong" answers to questions - an item - or a group of items (Blevins, 2014 and Burke, 2014). In Goals 2 and 3, the EdReady tools use online diagnostic formative assessments and online self-paced learning systems including multimedia learning to support students goals to improve math skills or prepare for the summative assessments of the ACT, SAT, Accuplacer or COMPASS. While summative assessments determine whether an answer is right or wrong, formative assessments provide students feedback so that they can learn from the assessment process and improve their skills. Below is an example question from the SAT diagnostic assessment. At first glance, it appears to be a typical summative assessment question. However, the EdReady introductory materials encourage a student to choose "I don't know" rather than to guess. EdReady then uses the "I don't know" and incorrect answers to create a customized study plan for the student.

Sample Question from the Diagnostic Formative Assessments in EdReady

EdReady utilizes the principles of differentiated instruction systems and self-paced instruction systems. "Differentiated instruction systems can be defined as any form of teaching and/or learning where students are engaged and learning at their academic level with the use of technology (Roberts, 2014). Self-paced instruction systems typically include some combination of the following elements:

• small instructional units,
• mastery of units to an established criterion,
• frequent testing,
• immediate feedback of test results,
• student proctors,
• heavy reliance on reading materials to achieve instructional goals,
• lectures and discussions as enrichment materials and motivational devices rather than
• instructional tools, and, of course,
• student pacing (Keller in Tatum and Lenel, 2012).

Below is EdReady screenshot that highlights how the tool utilizes differentiated instruction and self-paced instruction systems. This particular student has achieved an overall mastery of 80 out of 100. The units that the student has mastered are shown on the right - percents, etc. - and additional units would be listed as the student scrolled down. These areas of mastery may have been demonstrated in the initial diagnostic assessment or in subsequent assessment that occurred after EdReady online computer-based tutoring. The student can establish an overall goal and monitor goal attainment along the top of the screen. Currently, this student has achieved an 80 out of 100 and each tick mark between those numbers is a general math concept. For example, the third tick mark is "Systems of Equations and Inequalities" and the underlying skills are "Graphing Systems of Linear Equations", "Graphing Systems of Inequalities", and several other subtopics that are not shown on this screenshot. The student may want to "Check Your Mastery" of the umbrella topic or "Check Your Knowledge" of any of the subtopics. At any time the student can chose to "View Resources" for either of the subtopics.

Differentiated and Self-Paced Learning Systems in EdReady

If the student choses "View Resources" he or she would then begin to use the multimedia learning resources - videos, of EdReady which begin wih a Warm Up and then proceed through a video presentation, worked examples - possibly from Khan Academy. The student can then complete practice problems and a review, which tests the student's understanding. If the student prefers, he or she can go directly to any of these resources.

# Supporting Learning Theories

From the standpoint of traditional learning theories, EdReady can be viewed as the following:

• Didactic Instruction (i.e. videos and guided examples are teacher-to-student instruction)
• Constructivist Framework (i.e. student is building his/her learning)
• Behavioristic Measurement of Success (i.e. right and wrong answers)
• Differentiated Instruction (i.e. self-paced instruction and student chooses Resources)

EdReady can also be examined from the standpoint of a more modern learning theory - New Learning's Seven Affordances. In Drs. Kalantzis and Cope's New Learning: Elements of a science of education they discuss how technology has altered the state of learning and by doing so has provided seven affordances to the learner (Kalantzis and Cope, 2012). EdReady is a good example of a technology that provides these affordances .

Dr. Kalantzis and Dr. Cope - New Learning's Seven Affordances

Ubiquitious Learning - EdReady's web-based free tools provide anytime, anywhere resources for students.

Recursive Feedback - EdReady begins with a formative assessment and then continues to provide formative assessments as the student progresses through each stage of achieving his or her goals.

Multimodal Meaning - EdReady assessments are offered as text, video and audio. The "Resources" are varied and include a variety of videos with the option for audio and/or closed captioning. The supporting text is offered in English or Spanish with the option for text to speech.

Active Knowledge Making - To a lesser extent, EdReady supports active knowledge making. While the student is not creating a piece of work - written or video - that show insights based on his or her research, the student is interacting with the resources to increase knowledge in the specific mathematical topics to achieve his or her goals. On a more detailed level within the "Resources", EdReady provides the student with an explanation of whether an answer if correct or not and why.

Collaborative Intelligence - When EdReady is used by an institution - rather than an individual - collaborative intelligence could occur if students peer tutor is incorporated with the EdReady tools.

Metacognition - EdReady begins with an assessment so that a student understand his or her initial competency. In this assessment, EdReady encourages the student to choose "I Don't Know" to gain an accurate representation of his or her ability. EdReady provides feedback in each step of the process so that the student is aware of their increasing competency.

Differentiated Learning - EdReady students actively engage with the materials increasingly choosing which topics to explore, how to explore them and to what depth.

# Case Study

EdReady's functionality has been explored in relation to three student goals.

2. Prepare for Common Placement Exams.
3. Prepare for the ACT or SAT Math Test.

For these goals, a student completes a diagnostic assessment and determines which skills he or she has mastered and which he or she needs to master. It is in the latter - mastering additional math skills - that the power of EdReady is unleashed. This case study will be an examination of the EdReady resources. This case study will illustrate how the traditional teaching theories of didactic one-way teaching and behaviorist right-wrong answers are combined with the progressive teaching theories of constructivism and differentiated instruction. This examination of the tool will also show how these varied learning theories are presented within the modern advances of self-paced instruction and multimedia learning.

When a student chooses "View Resources" for a particular topic, he or she is provided with the variety of resources shown below. These can be explored in a linear fashion beginning with Warm Up and finishing with Review. Alternatively, the student can choose resources the individual needs for his or her particular situation. For example, a student may choose to watch Presentation and Worked Examples and then go to Review to determine his or her level of understanding. The student can also choose to come back to these resources at any time. This flexibility provides the student with the opportunity to self-pace the instruction.

The Warm Up which uses a behavioristic right-wrong answer methodology. This section includes several questions to see if the student has the requisite math knowledge to understand the topic that will be presented and explored in the following sections. In this particular case, there are six practice problems. If answered correctly, the student would progress to Presentation. If the student did not answer the Warm Up questions correctly, the student would be instructed to return to the appropriate prior topic(s). In this way, the tool guides the student so that his or her specific needs are met, i.e. differentiated learning.

Assuming the student understands the Warm Up questions, he or she would move to the Presentation section next. In Presentation, a high quality engaging video explains the mathematical concepts. The Presentation includes the option of Closed Captioning in either English or Spanish text to further support the student's needs. The video, audio, and multi-language closed captioning are additional examples of differentiated instruction.

After the Presentation video, the student would progress to Worked Examples. In this particular case, NCOR uses six Khan Academy videos to present several aspects of this math topic. The Presentation and Worked Examples sections are multimedia-learning methods which combine high quality video, audio, and written text.

Following the Presentation and six Worked Examples, the student would progress to the Practice section. In this particular situation, the student would complete ten practice problems to measure his or her understanding of "Graphing Systems of Linear Equations."

As the student completes each Practice problem, he or she receives feedback as to which answer was correct and why.

The last step in this process is to Review the topic. In this case, the student answers ten Review questions to measure his or her understanding of "Graphing Systems of Linear Equations."

After the student has completed the Review, he or she gets immediate feedback of success and for any question - right or wrong - the student can review the question and answer. As a final step, the system differentiates the learning needs of this particular student and provides guidance for further instruction.

Every screen in the EdReady "View Resources" section includes the button "Topic Text" which allows the student to access a textbook-like text which includes an overview of the topic, topic terms, and glossary. Each section of this "textbook" is available in either English or Spanish.

One real-world success story of EdReady can be found at the University of Montana which conducted an initial test in the summer of 2013 with positive results.

Early pilot results at UM showed that 86 percent of EdReady participants raised their math placement scores and therefore were able register for college level coursework this fall. A total of 41 students out of 60 no longer needed to start college with a remedial math course.

The project included high school, traditional-aged college students, and non-traditional students, all of whom progressed.

Based on this success, the University of Montana chose to "launch the program statewide making it available for all high school and post-secondary students" ("Dennis and Phyllis Washington Foundation Provides $2.4 Million for Statewide Online College-Prep Curriculum"). # Reflections and Recommendations EdReady is an excellent and easy-to-use web-based tool. The website is well designed and the supporting resources are excellent. The presentations and worked examples are short clear and engaging. The multiple worked examples are important because "a worked example, by reducing or eliminating search, reduces extraneous cognitive load and so facilitates learning" (Sweller, 2005). The depth of resources, closed captioning and dual language texts are excellent to support a variety of learners' needs. With EdReady, a student could certainly set and achieve his or her goal or complete tasks assigned by his or her teacher. EdReady can be used by an individual student or by an institution. For an individual, EdReady is an open (free) web-based tool that supports college-level math preparation. An institution can choose to become an NROC-member organization. Membership includes: • Unlimited use of all NROC content and tools • Teaching Resources to support instruction • Professional Development opportunities • Implementation Support for local adaptations and installations • Community collaboration The cost for sustaining NROC membership is not provided on the website. However, an interactive map to find member institutions is available. For member organizations, NROC offers two levels of customization - standard and extendable. In addition to a variety of customizable features, NROC membership allows teachers, college counselors or other administrators to access detailed EdReady student report data. EdReady Student Summary Reports While EdReady is an excellent tool, it could be better. One improvement might be to incorporate a computer adaptive system (Fentem, 2014 and Williams, 2014) that measures a student's ability and then provided a more or less challenging problem based on the student's success. An adaptive system could support a student's ongoing progress but still lead to the knowledge level that would be needed for the college placement or SAT and ACT assessments. Along with a computer adaptive system, another improvement could be more detailed analytics for the student or teacher to analyze the areas of success and challenge. These learning analytics (Ertz, 2014) would be especially important if EdReady continues to encourage students to use the tool to support classroom instruction. Having spoken to a NROC representative at the ISTE 2014 conference, I know that other improvements are underway. I believe they plan on expanding to include English and Writing (grammar and punctuation) so that a student could prepare for those sections of the ACT or SAT. Later developments to be considered could be adding resources to prepare for the ACT Science test. # Conclusion EdReady clearly identifies the three goals that a student could achieve by using resource. 1. Find Colleges. Check your math readiness. 2. Prepare for Common Placement Exams. 3. Prepare for the ACT or SAT Math Test. EdReady delivers on these three goals. It goes beyond those goals to provide any student from middle school through college an online assessment and tutoring resource from pre-algebra through trigonometry. The diagnostic assessments, study plan and support materials can be used by educators and students to assess and develop math skills in middle school, high school, GED programs, or college. For the student who has been accepted into college, EdReady provides math preparation to improve the student's score on college placement tests. With EdReady, the student may be able to avoid the extra time and cost of remedial math classes in college. One student highlighted this benefit of EdReady. Student Perspective Prior to college acceptance, EdReady could provide students an excellent ACT or SAT tutoring without the expense of the tutor. The high quality of these free resources could help students with limited funds remain competitive in the college application process. With college being the gateway to better careers, EdReady could be a piece of the puzzle for parents or college counselors who are searching for resources to help students achieve acceptance to college and move efficiently through college math classes. NROC is working to add more tools to support aspiring college students. Leveraging off the math success of EdReady, NROC plans to add English assessment and preparation tools in 2015. With these tools students could prepare for the English and Writing (i.e. grammar and punctuation) sections of the ACT and SAT. In later enhancements to the tool, the Science sections of the ACT could be added. To date, NROC focused their efforts on math for the ACT and SAT and developmental math for the college placement tests. Given that math tends to be more problematic for most students, their prioritization was well-placed and, as the University of Montana has found, their efforts have been quite fruitful. # References Amodei, M. (2014), Formative Assessment: Many educators and experts believe that formative assessment is an essential part of effective teaching because it drives instruction, Sep 22, 2014, retrieved on Sep 27, 2014, https://cgscholar.com/community/profiles/amodei2/publications/44480 Blevins, A. (2014), Wiki Entry for Item Based Testing and Item esign, A. Blevins, Sep 22, 2014, retrieved on Sep 27, 2014, https://cgscholar.com/community/profiles/ameliablevins/publications/45379 Dennis and Phyllis Washington Foundation Provides$2.4 Million for Statewide Online College-Prep Curriculum. (n.d.). Retrieved October 12, 2014, from http://news.umt.edu/2014/03/030614edrd.aspx

Ertz, J. (2014), Learning Analytics, Sep 22, 2014, retrieved on Sep 29, 2014, https://cgscholar.com/community/profiles/jtertz/publications/44501

Giles, K. (2014), Item-Based Testing and Design, K. Giles, Sep 23, 2014, retrieved on Sep 27, 2014, https://cgscholar.com/community/profiles/annalisa-burke/publications/44497

Kalantzis, M., & Cope, B. (2012). Chapter 10: Measuring Learning. In New learning: Elements of a science of education (2nd ed., pp. 324-333). Port Melbourne, Vic.: Cambridge University Press.

Roberts, J. (2014), Differentiated Instruction Systems, Sep 22, 2014, retrieved on Sep 27, 2014, https://cgscholar.com/community/profiles/jessica-roberts/publications/44463

Sweller, J. (2005), Implications of Cognitive Load for Multimedia Learning. In The Cambridge Handbook of Multimedia Learning. Cambridge: Cambridge University Press.

Tatum, B.C. and Lenel, J.C. (2012), A Comparison of Self-Paced and Lecture/Discussion Methods in an Accelerate Learning Format. Journal of Research in Innovative Teaching, 5(1), pp. 139-156.