Abstract

This case study aimed to change the construction of teachers' written tests so that items were designed to assess competencies in an authentic and challenging way. A small group of five psychology teachers participated in 10 sessions of an authentic assessment faculty-training program, to learn to assess problem-solving competencies for situations typically faced by professionals in the workplace. The authentic assessment training emphasized the incorporation of three main characteristics: (1) inclusion of a realistic context, (2) measurement of higher order thinking skills, and (3) development of evaluative judgment, concerning the quality of their own performance. Post-training the items' construction was analysed, according to their type and authenticity. Mixed effects logistic regression showed a statistically significant increase in open-response items, and two-way ANOVA indicated that cognitive challenge improved. The results showed written tests had: a) more open-response items of higher cognitive complexity, b) fewer items requiring closed, memorized responses, c) more use of realistic contexts to measure knowledge in a situated way, and d) improvement in curriculum alignment of tests and greater consistency in measuring competencies. These advances validate the authentic-assessment training program for a better written assessment design.