Does shuffling exam questions actually reduce copying?
Most teachers who shuffle exams do it for one reason: if the paper on the next desk is in a different order, copying from it is harder. That sounds obvious, but it is fair to ask whether anyone has actually checked.
There are two separate questions here:
- Do different versions make copying less useful?
- Does shuffling change a student’s score for reasons that have nothing to do with what they know?
A small amount of research covers each one. This guide sums it up in plain terms, including where it is thin.
Do different versions reduce copying?
The most direct study we know of is by Mehdi Mdaghri Alaoui, Flip Klijn and Marc Vorsatz: “Online Academic Exams: Does Multiplicity of Exam Versions Mitigate Cheating?”, published in the International Review of Economics Education. You can read it free from the Barcelona School of Economics, and the journal version is on ScienceDirect.
What they did. 463 university students took an online multiple-choice final exam in a game theory course. Nobody supervised it, and students could not go back to earlier questions. Different groups got different versions of each problem, and the same version was given to different groups at different times, so the researchers could see whether answers seemed to be passed along.
What they found.
- Students who took a version later finished it faster. That suggests information about the questions was getting around.
- But those students did not score any higher. Whatever they heard did not turn into marks.
- The number of versions made no clear difference. The authors conclude that as few as two versions can make cheating less effective.
What it does not show. This was one course at one university, taken online with no supervisor. A paper exam in a supervised room is a different situation: copying there usually means a glance at the next desk, not a message to a friend. The study is a good sign for paper exams, but it did not measure them.
What it means for you. Two versions is a sensible starting point. Making four or five probably adds work without adding much protection.
Does shuffling change students’ scores?
This question has been studied more often, mostly in university economics courses.
The clearest paper is by Della L. Sue: “The Effect of Scrambling Test Questions on Student Performance in a Small Class Setting”, in the Journal for Economic Educators (2009). Across three exams in two courses, with about 73 to 82 students per exam, half got the questions in topic order and half got them scrambled.
On some exams the scrambled version did average a little lower. But none of the differences were large enough to rule out chance. In the paper’s words, the idea that “scrambling the content order of questions in a multiple-choice test does not affect student performance on the test is not rejected.”
Two older studies point the same way. We have only read them as summarised in Sue’s paper, not in the original:
- Bresnock, Graves and White (1989) found that question order had no effect on test performance.
- Sue (2006) found that scrambling question order did not affect grades in an intermediate microeconomics course.
How to read this. “No difference was found” is not the same as “there is no difference.” These are small studies, all in economics, and a small effect could be there without them spotting it. What you can fairly say is that shuffling question order has been tested a few times and has not been found to matter.
The catch: shuffling the answer options
The same Bresnock, Graves and White study had a second finding, again as summarised by Sue. Changing the order of the answer options inside a question could change how hard the test was, as measured by scores.
So the two kinds of shuffling are not equally well supported:
| What gets reordered | What the research suggests |
|---|---|
| The questions | No effect found in three studies |
| The answer options inside a question | At least one study suggests it can change difficulty |
ExamShuffle does both by default. We think you should know that. Shuffling the options is what stops a student from copying a neighbour’s answer letters, so it is useful. But it is also the part with less research behind it.
If an exam is high-stakes and you want versions to be as comparable as possible, you can shuffle the questions and leave the options in their original order. In ExamShuffle, each multiple-choice question has a Don’t shuffle these options switch on the review screen. The question still moves, but its options stay as you wrote them.
What to do with all this
- Use two versions and hand them out so that neighbours never share one. The exam-day checklist shows how.
- Expect question shuffling to be roughly fair. No study above found an effect.
- Be more careful with option shuffling on exams where small differences in marks matter, such as finals or placement tests.
- Fix questions that break when moved, like “All of the above” in the wrong place. Writing questions that survive shuffling lists the common ones.
The limits of the evidence
All of these studies are of university students, mostly in economics, on multiple-choice exams. None looked at school pupils, essay questions or short-answer papers. That is a narrower base than the confidence with which shuffling is usually recommended.
Shuffling is still a reasonable thing to do. It costs little, and the one study that measured copying directly found that it helped. Just treat it as one sensible step, not a guarantee.