Please take a moment and take the Pre-course Survey, which will instructors prepare tailored instruction to the needs of learners.

Creating a test is one of the most challenging tasks confronting an instructor. Unfortunately, many of us have had little, if any, preparation in writing tests. Well-constructed tests motivate students and reinforce learning. Well-constructed tests enable teachers to assess the students' mastery of course objectives. Tests also provide feedback on teaching, often showing what was or was not communicated clearly. Here are a few tips when creating test materials that would go along way in helping you appropriately assess your students' proficiency.

Harlen, W. & Deakin, Crick R, (2003) suggest we follow the following rules of thumb as a guide:

Length of Test

• In theory, the more items a test has, the more reliable it is. On a short test a few wrong answers can have a great effect on the overall results. On a long test, a few wrong answers will not influence the results as much. A long test does have drawbacks. If a test is too long, and particularly if students are doing the same kind of item over and over, they may get tired and not respond accurately or seriously. If a test needs to be lengthy, divide it into sections with different kinds of tasks, to maintain the student's interest.

Clear, Concise Instructions

• It is necessary to give clear, concise instructions. It is useful to provide an example of a worked problem, which helps the students understand exactly what is necessary. What seems to be clear to the writer may be unclear to someone else.

Mix It Up!

• It is often advantageous to mix types of items (multiple choice, true-false, essay) on a written exam or to mix types of exams (a performance component with a written component). Weaknesses connected with one kind of item or component or in students’ test taking skills will be minimized.

Test Early

• It is helpful for instructors to test early in the term and consider discounting the first test if results are poor. Students often need a practice test to understand the format each instructor uses and anticipate the best way to prepare for and take particular tests.

Test Frequently

• Frequent testing helps students to avoid getting behind, provides instructors with multiple sources of information to use in computing the final course grade (thus minimizing the effect of “bad days”), and gives students regular feedback. It is important to test various topics in proportion to the emphasis given in class. Students will expect this practice and will study with this expectation.

Check For Accuracy

• Instructors should be cautious about using tests written by others. Often, items developed by a previous instructor, a textbook publisher, etc., can save a lot of time, but they should be checked for accuracy and appropriateness in the given course

Proofread Exams

• On written exams, it is important to proofread exams carefully and, when possible, have another person proofread them. Tiny mistakes, such as misnumbering the responses, can cause big problems later. Collation should also be checked carefully, since missing pages can cause a great deal of trouble.

One Wrong Answer

• Generally, on either a written or performance test, it is wise to avoid having separate items or tasks depend upon answers or skills required in previous items or tasks. A student’s initial mistake will be perpetuated over the course of succeeding items or tasks, penalizing the student repeatedly for one error.

Special Considerations

• It is important to anticipate special considerations that learning disabled students or non-native speakers may need. The instructor needs to anticipate special needs in advance and decide whether or not students will be allowed the use of dictionaries, extra time, separate testing sites, or other special conditions.

A Little Humor

• Instructors have found that using a little humor or placing less difficult items or tasks at the beginning of an exam can help students with test anxiety to reduce their preliminary tension and thus provide a more accurate demonstration of their progress.

What makes a test good or bad?

• The most basic and obvious answer to that question is that good tests measure what you want to measure, and bad tests do not.
•  A good test reflects the goals of the instruction. If the instructor is mainly concerned with students memorizing facts, the test should ask for simple recall of material. If the instructor is trying to develop analytic skills, a test that asks for recall is inappropriate and will cause students to conclude that memorization is the instructor's true goal.

 

Creating Fill in the Blank Questions

There are a number of advantages to fill-in-the-blank questions.

• They provide an excellent means for measuring specific knowledge.
• They reduce guessing by the students
• They force the student to supply the answer. In other words, teachers can get a real feel for what their students actually know.

Here are some things to keep in mind when developing these types of questions.

• Keep them simple and clear.
• Ensure there is only one correct answer for the blank.
• Use the blank judiciously.
• Keep answers brief.
• Limit the number of blanks per question.
• Don't give clues to the correct response in the question.
• Review it carefully.

The link, below, provides further details on constructing Fill in the Blank types of question. http://https://www.ispringsolutions.com/blog/8-tips-for-writing-good-fill-in-the-blank-questions-in-e-learning-courses

Advance Skills Online (2021, June 17). How to write Fill in The Blanks questions. [Video]. https://www.youtube.com/watch?v=5lu5oYAxr84

In the above video, Saad F Shawer shares some technique on how to write Fill in the Blank questions, decides when to use them, and talks about their advantages and disadvantages. He also explains the whole process of crafting Fill in the Blank questions that includes, crafting, validating, and trying them out.

Pros Cons Best Practices

• Provide a wide sampling of content  
• Minimize guessing compared with multiple choice and true/false
• Vocabulary can be assessed in a minimal time
• Construction is relatively easy
• Difficult to construct in order to measure more than simple information recall
• Difficult to construct without providing relevant clues to correct answer
• More time-consuming to score than other objective questions
• Ensure only one correct answer to avoid confusion and difficulty grading.
• Blank area should appear at the end or close to the end of a statement.
• Omit only significant words from the statement. Do not omit so many words that the statement becomes ambiguous. Poor: The Constitutional Convention met in ______________ in ______________. Better: The Constitutional Convention met in the city of _______________ in 1787.
• If possible, write completion items that have a single correct answer. Poor: Abraham Lincoln was born in ____________________. Potential answers: Kentucky, 1809, February, a log cabin, etc. Better: Abraham Lincoln was born in the state of _________.
• Use blank areas of the same length throughout the test so that the length does not reveal answer.
• Avoid lifting text from study materials or other resources to avoid answer memorization.

Once the questions are ready, you can utilize the following apps to create your test/quiz. There are a great number of other apps available online to utilize for this purpose. These are just a few samples for demonstration.

Socrat Ghadban (2020, June 15). Fill in the blank Quiz Google Forms. [Video]: https://www.youtube.com/watch?v=f7LPD7yMx5o

 

Mike Rich (2018, January 19). Schoology Assessments - Part 2: Fill in the Blank Items. [Video]: https://www.youtube.com/watch?v=f7LPD7yMx5o

The video above focuses on the three Fill in the Blank question types: traditional, drop-down, and drag-and-drop. You can explore all three types working on your own and then share with peers. 

Update:

1. Create an update: Select a passage for a level of your choice and create a fill-in the black assessment. Share it with your peers for peer-feedback.

2. Comment on at least three updates posted by participants this week.

Matching Question Definition

A matching question is two adjacent lists of related words, phrases, pictures, or symbols. Each item in one list is paired with at least one item in the other list. Matching may be considered to be a variant of multiple-choice in which more than one choice is correct. Check all that apply (multiple select) is also a variant of matching in which one of the lists (or the stem) has only one item.

Characteristics of Matching Questions

• Used for recognition of relationships and making associations.
• Can be used for a wide range of subject matter.
• Can be used to match:

Tips for writing good matching questions include:

• Provide clear directions.
• Keep the information in each column as homogeneous as possible.
• Allow the responses to be used more than once.
• Arrange the list of responses systematically if possible (chronological, alphabetical, numerical)

Good matching items can easily be converted to multiple-choice items.

The premises and responses should be homogeneous.

Advantages of Matching Questions

• Easy to write.
• Easy to score.
• Effective when assessing definitions and relationships.
• Space-saving, objective, compact method to assess learning targets.
• Can be developed to use with pictures, maps, graphs, etc.

Disadvantages of Matching Questions

• Not well suited for assessing higher-order thinking. Does not measure any type of interpretation, judgment, or application.
• Correct matches may be evidence of guessing rather than understanding (Make sure there are more definitions than items to alleviate this).
• If lists contain same number of choices, the last few choices may be giveaways.
• Students can use rote memorization to answer these exercises, especially as typical matching problems involve assessment of rote associations such as names, dates, etc. That's OK if that's what you want.
• Can be difficult to develop homogeneous premises and responses.

Constructing Matching Questions

Below are a few guidelines to help you construct matching questions:

• Directions should provide a clear basis for matching.
• Include 4 to 10 items in each list.
• Include approximately 3 extra items from which responses are chosen. That is, avoid perfect matching which allows for elimination of responses when the test-taker is unsure of the correct answer.
• Shuffle the order of the items in each column.
• Use brief phrases and as few words as possible in each list.
• Longer statements should be in the premise (left) column and shorter statements in the response (right) column.
• Format the entire question to fit on the same page.
• All possible responses should be plausible solutions.

The link below provides guidance on how to write effective questions for quizzes or tests.

http://personal.psu.edu/bxb11/QuizQuestions/quizmatching.html

Advance Skills Online (2021, June 17). How to write Fill in The Blanks questions. [Video]: https://www.youtube.com/watch?v=5lu5oYAxr84

In the above video, Dr. Saad F Shawer shares some technique on how to write effective Fill in the Black questions, decides when to use them for maximum effect, and talks about their advantages and disadvantages. He also explains the whole process of designing Matching questions that includes, crafting, validating, and trying them out.

Once you have mastered the art of crafting good Matching questions, utilize the follow applications to make your tests more interactive, which will increase student motivation and thus stimulate their learning. There are a ton of other apps available online, which you pick and choose to better suit your needs.

Sowash (2019, April 2). Creating Matching Questions using Google Forms. [Video]: https://www.youtube.com/watch?v=YiIqzwZQAv8

In this video, you'll learn how to use the grid question type to create a matching question using Google Forms. These questions are significantly more complex than a simple multiple-choice question.

Additional Resources: Reference the link below for details on Exam Questions: Types, Characteristics, and Suggestions.

Update:

1. Create an update: Select a passage for a level of your choice and create a Matching assessment. Share it with your peers for peer-feedback.

2. Comment on at least three updates posted by participants this week.

Writing Good Multiple Choice Test Questions

Multiple choice test items can be written to assess various levels of learning outcomes, from basic recall to application, analysis, and evaluation. Because students are choosing from a set of potential answers, however, there are obvious limits on what can be tested with multiple choice items. For example, they are not an effective way to test students’ ability to organize thoughts or articulate explanations or creative ideas.

​Multiple choice items are considered to be among the most versatile of all item types. They can be used to test factual recall as well as levels of understanding and ability to apply learning. Multiple choice tests can be used to test the ability to:

• recall memorized information
• apply theory to routine cases
• apply theory to novel situations
• use judgment in analyzing and evaluating

A multiple choice item consists of a problem, known as the stem, and a list of suggested solutions, known as alternatives. The alternatives consist of one correct or best alternative, which is the answer, and incorrect or inferior alternatives, known as distractors.

• The stem should be meaningful by itself and should present a definite problem. A stem that presents a definite problem allows a focus on the learning outcome. A stem that does not present a clear problem, however, may test students’ ability to draw inferences from vague descriptions rather serving as a more direct test of students’ achievement of the learning outcome.

Truckee Meadows Community College (2015, February 18). Writing Multiple Choice Test Questions. [Video]: https://www.youtube.com/watch?v=3zQLZVqksGg

In the video, above, Kathy Browster talks about the art of developing effective Multiple-choice questions in a very simple to understand way, and also talks about the use of Bloom's Taxonomy in creating Multiple-choice questions in order to make the questions more thought provocative for greater master of the subject matter.

Nancy Sulla (2012, September 18). Writing Multiple Choice Test Questions. [Video]: https://www.youtube.com/watch?v=irqXYvVnzbk

Nancy Sulla, provides insight into the art of crafting effective multiple choice test questions, also known as items. She also explains some of the several potential advantages. For example, multiple choice items can also provide an excellent basis for post-test discussion, especially if the discussion addresses why the incorrect responses were wrong as well as why the correct responses were right. 

Test Reliability and Validity are two terms that are very important to keep in mind when constructing any tests (The Graide Network, 2018):

• Reliability is defined as the degree to which a test consistently measures a learning outcome. Multiple choice test items are less susceptible to guessing than true/false questions, making them a more reliable means of assessment. The reliability is enhanced when the number of MC items focused on a single learning objective is increased. In addition, the objective scoring associated with multiple choice test items frees them from problems with scorer inconsistency that can plague scoring of essay questions.
• Validity is the degree to which a test measures the learning outcomes it purports to measure. Because students can typically answer a multiple choice item much more quickly than an essay question, tests based on multiple choice items can typically focus on a relatively broad representation of course material, thus increasing the validity of the assessment.

Advance Skills Online (2020, June 17). How to write Multiple Choice questions. [Video]: https://www.youtube.com/watch?v=RBwoJXeSDwE

In the above video, Dr. Saad F Shawer shares some technique on how to write effective Multiple-choice questions, decides when to use them for maximum effect, and talks about their advantages and disadvantages. He also explains the whole process of designing Multiple-choice questions that includes, crafting, validating, and trying them out.

Additional Resources: Reference the link below for details on Exam Questions: Types, Characteristics, and Suggestions.  

Update:

1. Create an update: Select a passage for an intermediate level and create three multiple-choice items: one stem with four distractors each question. Share it with your peers for peer-feedback.

2. Comment on at least three updates posted by participants this week. 

Short-answer questions are open-ended questions that require students to create an answer. They are commonly used in examinations to assess the basic knowledge and understanding (low cognitive levels) of a topic before more in-depth assessment questions are asked on the topic.

8 tips for designing short answer questions

Previously, we discussed how to design effective multiple-choice questions. In that article, we covered ten ways to create multiple choice questions and debated multiple choice questions vs. short answer questions.

While both formats have their merits, short answer questions can give instructors a better look at their students’ understanding of the material. Short answer questions are an excellent way to give students space to explain their answers and determine which students did not grasp the lesson.

The guide, below, is beneficial to new instructors looking to refine their approach. Click on the hyper link for more detail.

How to write short answer questions

1. Write your questions clearly and concisely and in an easily understandable way. If the question’s wording is convoluted, students will waste more time dissecting the problem than answering it.
2. Questions should be straightforward and avoid misleading language. The goal of examinations is to gauge your students’ level of understanding. Remember that “tricky” questions have no place in a proper exam.
3. Avoid using any new terms or language when writing short answer questions. All wording used in your examination should be familiar to your students to avoid confusion.
4. Short answer questions should include memory triggers or recognizable language to help students jog their memory. This technique will be beneficial to students who suffer from exam anxiety or tend to fail under pressure.
5. When designing your short answer questions, include helpful writing prompts. Using open-ended questions will allow students to provide more context with their answers and aid in the evaluation process. Prompts should also be within the question itself, including writing suggestions or language that the student will recognize as a writing prompt.
6. Reading comprehension questions are the bread and butter of examinations. Be sure to include some reading comprehension short answer questions in your exam to encourage students to absorb the course material. Being able to read and relay information is an essential skill for both professional and personal situations.
7. Give students an opening to explain by writing “Please explain” after the question. Like the useful prompts tip, leaving room for students to demonstrate their understanding gives instructors a look at their student’s development. Any student can make an educated guess and write down a few words, but asking them to break down their answer will provide a better impression of what they’ve learned.
8. Make sure you provide enough physical space. There’s nothing worse than answering a short answer question and running out of lines. Be sure to give each short answer question an appropriate amount of page real estate. Alternatively, you can indicate how long you’d like your students’ answers to be within the short answer question.

Advance Skills Online (2021, June 17). How to write Short Answer questions. [Video]: https://www.youtube.com/watch?v=5lu5oYAxr84

In the above video, Dr. Saad F Shawer shares some technique on how to write effective Short-answer questions, decides when to use them for maximum effect, and talks about their advantages and disadvantages. He also explains the whole process of designing Short-answer questions that includes, crafting, validating, and trying them out.

Additional Resources: Reference the following link for Exam Questions: Types, Characteristics, and Suggestions. 

Update:

1. Create an update: Select a passage for an intermediate level and create three multiple-choice items: one stem with four distractors each question. Share it with your peers for peer-feedback.

2. Comment on at least three updates posted by participants this week.

Writing Effective True-False Questions

In the most basic format, true-false questions are those in which a statement is presented and the student indicates in some manner whether the statement is true or false. In other words, there are only two possible responses for each item, and the student chooses between them. True-false questions are well suited for testing student recall or comprehension. Students can generally respond to many questions, covering a lot of content, in a fairly short amount of time. From the teacher’s perspective, true-false questions can be written quickly. They are easy to score. Because they can be objectively scored, the scores are more reliable than for items that are at least partially dependent on the teacher’s judgment. 

Murray (2010, October 19). Creating Good Assessment: True/False. [Video]: https://www.youtube.com/watch?v=9OukxUKOmic 

In the video, above, Murray discusses ways to create stronger True-False items and highlights the implications of their implementation. For instance,

• T/F questions assess only the lower cognitive levels.
• There is a 50% chance of getting the question correct, there is a high possibility of random error.
• Random error decreases the reliability of assessment.
• T/F should be used sparingly with conjuncntion with other types of items.

Advance Skills Online (2020, June 17). How to write True False questions. [Video]: https://www.youtube.com/watch?v=ATwoh6YhLh8

In the above video, Dr. Saad F Shawer shares some technique on how to write effective True/False questions, decides when to use them for maximum effect, and talks about their advantages and disadvantages. He also explains the whole process of designing True/False questions that includes, crafting, validating, and trying them out.

Once you have mastered the art of crafting good True/False questions, utilize the myriad applications available online to make your tests more interactive, which will increase student motivation and thus stimulate their learning. There are a ton of other apps available online, which you can pick and choose to better suit your needs.

CHICAGO (2020). Writing True False Questions: 9 Quick Tips to Make Them Better. [Video]: https://www.youtube.com/watch?v=9Cv4YfTQvAs

The video, above, provides tips on writing effective true/false questions, which include the following:

• Include only one main idea in each item.
• As in multiple choice questions generally, use negatives sparingly.
• Try using in combination with other material, such as graphs, maps, written material. This combination allows for the testing of more advanced learning outcomes.
• Use statements which are unequivocally true or false.
• Avoid lifting statements directly from assigned reading, lecture notes or other course materials so that recall alone will not permit a correct answer.
• Generally avoid the use of words which would signal the correct response to the test-wise student. Absolutes such as 'none', 'never', 'always', 'all', 'impossible' tend to be false, while qualifiers such as 'usually', 'generally', 'sometimes', 'often' are likely to be true.

Additional Resources: Refence the following link for further details on Exam Questions: Type, Characterics, and Suggestions.

Update:

1. Create an update: Select a passage for an intermediate level and create three True/False items. Share it with your peers for peer-feedback.

2. Comment on at least three updates posted by participants this week.

What does OPI test for?

• An Oral Proficiency Interview (OPI) is a standardized, global assessment of functional speaking ability. Taking the form of a conversation between the tester and test-taker, the test measures how well a person speaks a language by assessing their performance of a range of language tasks against specified criteria.

The four mandatory phases of Oral Proficiency Assessment include the following elements:

• Warm-up: FORTE (family, occupation, recreation, travel, and education)
• Level checks: What a speaker can handle with ease
• Probes: What a speaker has trouble handling
• Wind-down: Calming down the speaker

 Level check questions ranges from simple short conversation to more complex questions such as description, past/present/future narration, and expression of opinion. Wind Down includes simple short conversation, and feedback for future improvement.

Yijing Chen (2012, December 5). Oral Proficiency Interview OPI. [Video]. https://www.youtube.com/watch?v=EKA-QB9YhZ0

 An ACTFL OPI reports a rating between Novice and Superior on the ACTFL scale. An ILR OPI will rate between ILR 0 (No Proficiency) and ILR 5 (Functionally Native). 

The OPI assesses language proficiency in terms of the ability of the candidate to use the language effectively and appropriately in real-life situations. It does not address when, where, why, or the way in which a speaker has acquired his/her language. The OPI is not an achievement test assessing a speaker's acquisition of specific aspects of course and curriculum content, nor is it tied to any specific method of instruction. The OPI does not compare one individual's performance to others, but each individual performance to the assessment criteria.

The link below provides a sample of the type of questions you would want to engage the learn in during a speaking practice. https://www.bestmytest.com/ielts/speaking

IELTS Advantage (2019, August 29). ELTS Speaking Mock Test - Band 8. [Video]. https://www.youtube.com/watch?v=ZDv9njERj0s

In the video above, the interviewer demonstrate how to conduct a speaking test, but also how to give constructive feedback for improvement. He breaks down the components of the speaking evaluation. Below is list of the criteria, John utilizes to assess the speaker's discourse: Pronunciation. Pronunciation is a basic quality of language learning.

• Vocabulary. ...
• Accuracy. ...
• Communication. ...
• Interaction. ...
• Fluency.

Additional Resources: Refence the following link for more details on Exam Questions: Types, Characteristic, and Suggestions. 

Update:

1. Create an update: Select a person you would like to to interview and determine the level of their proficiency. Share it with your peers for peer-feedback.

2. Comment on at least three updates posted by participants this week.

Final project is due this week. 

Project Description:

• Design a test that would assess all skill levels that you have learned in this module. The test should include Fill in the Blank, Matching, Multiple-choice, Short-answer, and True/False components. 
• This a group project. Decide on how to distribute the load, and determine a team lead, who would make sure everyone is doing what they are supposed to do. 

Rubric

Please take the End of Course Survey and help us make this workshop better for future itterations. Your comments are greatly valued.