Abstract

Comparability of student evaluation of teaching (SET) administered to different groups of students is crucial in supporting sound institutional analysis and decision-making, including personnel decisions. However, the interpretation of the same SET questionnaire may change as there are differences in beliefs about teaching and learning across disciplines, and because students change their understanding of high-performance teaching over time. This study presents substantive justification and statistical reasoning for studying the comparability of SET scores across commonly used administrative variables in institutional data analysis. As an illustration, the study presents evidence of measurement invariance to support SET scores comparability by (1) type of degree program, (2) academic department and (3) over terms using data from a graduate school of education. The results reveal no support for the strict comparability of SET scores across these institutionally relevant variables. SET comparability evaluation helps ensure effective decision-making affecting faculty and curriculum development. In addition, this analytical tool enables institutions to improve SET over time. Users and those affected by SET data utilisation should develop the skills to guarantee and inquire about SET comparability for credible interpretation of SET scores, along with fair decision-making. © 2025 Informa UK Limited, trading as Taylor & Francis Group.