Writing multiple-choice questions that survive randomization
Most multiple-choice questions can be shuffled with no problem. A few cannot. They are easy to miss when proofreading, because the question still reads fine. It just no longer has a correct answer.
Almost every question that breaks does so because it depends on its position. This guide lists the common patterns, shows how to rewrite each one, and explains what ExamShuffle catches automatically and what it leaves to you.
Options that refer to other options
“All of the above” and “None of the above” only make sense as the last option. If they move to position B, the question is broken. Keep them last in every version.
“Both of the above” is trickier. It refers to the two options directly above it. Even if it stays in place, shuffling changes which two options sit above it, so it now points at the wrong answers. Keeping it in position does not fix it.
Options that name other options by letter, such as “Both A and C” or “A, B and C”, break as soon as the options move, because the letters now belong to different answers.
The fix for all of these is the same: name the content, not the position.
| Instead of | Write |
|---|---|
| Both A and C | Both the mitochondria and the chloroplast |
| Both of the above | Both photosynthesis and respiration |
| A, B and C | All three of the processes listed |
The rewritten option works in any order, and it is clearer for students too. They no longer have to remember which letter went with which answer.
If you want to keep “All of the above”, that is fine. Just make sure it stays last.
Options that have a natural order
Some option lists are in order for a reason:
- Dates: 1918, 1932, 1945, 1963
- Ranges: less than 10, 10 to 50, 50 to 100, more than 100
- Steps in a process, or items ranked from smallest to largest
Shuffled, these questions can still be answered, but the student now has to sort the options in their head first. That is extra work the question was not meant to test. It is also the kind of question where shuffling options is most likely to change difficulty, which one study has suggested can happen.
Keep the options of these questions in their original order. The question itself can still move around the paper.
Questions that depend on their position
Look for these in the question text itself:
- References to letters, like “Which of options A and B is correct?” Same problem as above, but harder to spot because the letters are in the question, not the options.
- References to other questions, like “Using your answer to question 4”. Once the order changes, question 4 is a different question. Either rewrite the question so it stands alone, or put the related questions in their own section so they stay together.
- Instructions about layout, like “the answers are in chronological order”. This stops being true once the options are shuffled.
What ExamShuffle handles for you
When you upload an exam, ExamShuffle looks for the most common problems:
- It pins “All of the above”, “None of the above” and “Both of the above” in their original position. The other options shuffle around them.
- It stops shuffling a question’s options completely when one option names others by letter, such as “Both A and B”, “A and B” or “A, B and C”. Pinning would not help there, because the problem is the text, not the position. It only does this when the letters stand alone, so an option like “Fish and chips” is not affected.
Questions are shuffled inside their own section, and sections move as whole blocks. So anything that has to stay together can go in its own section.
What you still need to check
Some problems cannot be found automatically:
- “Both of the above” is pinned, but pinning does not fix it. As explained above, the two options above it change. Rewrite it to name the two answers.
- References written in words, such as “the previous option” or “neither of the first two”.
- Option lists in a natural order, such as dates. There is no reliable way for software to tell that four numbers are a timeline.
- Anything inside an image, such as a diagram with lettered labels.
On the review screen there are two controls for these:
- Don’t shuffle these options is a switch under each multiple-choice question. Turn it on to keep that question’s options exactly as written. The question still moves around the paper. Use this for dates, ranges and other ordered lists.
- Pin is a checkbox next to each single option. It keeps that one option in place while the others shuffle around it.
A quick test for any question
Read the question as if its options were in a random order. On at least one version, they will be.
- If it still makes sense, it is safe to shuffle.
- If you need the original order to understand it, rewrite it or keep its options fixed.
Questions that pass this test are usually better questions anyway. Each option is a complete answer on its own, and students have to know the material rather than work out the shape of the list.
For the rest of the process of making versions by hand, see how to make multiple versions of an exam in Word.