Randomization Best Practices

Keep It Simple

Consider what you are trying to accomplish when randomizing values in a question, and whether adding more complexity — by randomizing more variables, or by allowing a wider range of randomized values — is the best solution.

To deter cheating, it is not necessary that every student in your class have completely different assignments, only that there be sufficient variation to make it unproductive for any two students who want to share answers to do so. If a given question has two variables, it's usually necessary only to use five or six possible values for each variable to deter cheating.

For example, in a class of 100 students, a question with two variables that each have five possible values will have 25 variants, meaning that on average, only four students in the class have the same version of the question.

Avoid randomizing interdependent values in the question. Instead, randomize one or two base values and calculate other values based on your randomized values. This simplifies your work when testing the question to ensure that all randomizations are solvable. When your question is displayed, both the randomized and calculated values are shown in red so your students do not know which values were randomized and which were calculated.

Sometimes, rather than increasing the complexity of a question's randomization, it is more productive to create several similar questions that are essentially the same but differ in their surface features — for example, a bomb dropped from a moving plane or a cannonball shot from a moving ship. Your alternative questions can vary the randomized values that are used, and if appropriate, also use different units or significant figures. You can either create a pool of these questions on your assignment, or use these questions for additional non-credit practice with the <PRACTICE> tag.

Keeping your questions simple makes them easier to test and makes it less likely that your students will have problems with them.

Keep It Real

Make sure all of the possible values your randomizations are reasonable, are possible, make sense, and result in a solvable problem.

• Ensure that all possible values in your question result in an answerable question, and do not result in answers that cannot be entered.
• Watch out for measurements that are impossible or could not be measured, for example, negative diameters or masses that are orders of magnitude smaller than the mass of an electron.
• Do not randomize known constants like the speed of light.
• Be careful that your values do not result in division by zero or similar problems.
• When creating problems with fractions, ensure that the numerators and denominators are integers and not decimals.
• Ensure that the variations of your numeric values do not cross orders of magnitude or change the expected number of significant figures in the answer.
• Avoid edge case values that require a judgement call, for example, whether a pH of 6.9 is neutral or acidic.
• Ensure that the values you select in your question do not change the number of correct answers, or code your question in such a way as to accommodate this.

Frequently, it might make sense to pick tested values from a list instead of selecting them from a numeric range.

Think Like a Student

Try to see your question from a student's perspective, and ensure that an honest effort and understanding of the material is rewarded.

• Ensure that none of the randomized values change the difficulty of the question. For example, two randomizations of a question might evaluate the student's understanding of a procedure, but one might require much more computation than another.
• Generally, avoid giving values beyond five or six decimal places. There are several reasons for this:
  • Most known physical constants are provided to a limited number of decimal places, and some sources for these values might disagree slightly.
  • Students are usually allowed to use calculators, which do not always have the same precision as the WebAssign servers.
  • There is no need to make the mathematical computation of the answer unduly difficult, if computational proficiency is not what is being learned.
• If you display a figure, expect some students to attempt to measure it with a ruler or protractor. If the figure is not to scale or representative of the values in the question, indicate this in the question.

Test Your Questions

Randomization errors are probably the most common kind of question error for two reasons:

• Because randomization results in multiple versions of a question, errors are not immediately apparent.
• Because randomization changes the terms of the question, errors can make your question unanswerable.

Thoroughly test your questions with a wide range of randomized values to be sure that no combination of values causes problems like those described here.