Abstract

Multiple-choice is the most common test format in educational assessment. It offers many advantages but has one shortcoming: examinees might earn extra points through strategic guessing. Item-writing guidelines on response option placement have been formulated to prevent examinees from strategically using the answer keys' distribution as clues. However, several guidelines co-exist in literature, untested by mathematical methods. In this study, the answer keys' distributions that may emerge when implementing the most common guidelines on answer keys' placement (balancing and randomization) were modeled, and the effectiveness of using well-known option-position-based response strategies ("when in doubt, choose C"; balanced guessing) was analyzed through a simulation study. Gains obtained in the different scenarios were compared for tests with different lengths and examinees with different achievement levels. Results show that randomization is the only guideline not associated with potential test results validity or equity issues. A set of recommendations for test developers is provided.